CN114928699A

CN114928699A - Ultrafast imaging method based on color digital camera

Info

Publication number: CN114928699A
Application number: CN202210462633.9A
Authority: CN
Inventors: 黄敏
Original assignee: Sun Yat Sen University
Current assignee: Sun Yat Sen University
Priority date: 2022-04-28
Filing date: 2022-04-28
Publication date: 2022-08-19
Anticipated expiration: 2042-04-28
Also published as: CN114928699B

Abstract

The invention discloses an ultrafast imaging method based on a color digital camera, which comprises the following steps: controlling the output wavelength range of the super-continuum spectrum light source to cover a single super-continuum spectrum ultrashort pulse of all color channels of the color digital camera; splitting the super-continuum spectrum ultra-short pulse to obtain each path of primary color ultra-short pulse with the wavelength range respectively in the wavelength range of each color channel of the camera; delay time control is carried out on each path of primary color ultrashort pulse; combining beams of the primary color ultrashort pulses to obtain a multi-primary color ultrashort pulse string; the ultrashort pulse train is used as an imaging active illumination light source to irradiate an imaging target; the irradiation target is imaged on an image sensor of the color digital camera through an imaging optical path; controlling a camera to shoot by single exposure; and acquiring gray level images of all color channels of the images shot by the camera to obtain a plurality of ultrafast imaging frames which correspond to all primary color ultrashort pulse illumination instant scenes and are arranged according to a time sequence, so as to realize multiframe ultrafast imaging.

Description

Ultrafast imaging method based on color digital camera

Technical Field

The invention relates to the technical field of ultrafast imaging, in particular to an ultrafast imaging method based on a color digital camera.

Background

Ultrafast imaging generally refers to a technology for imaging an ultrafast transient process, and the development of the ultrafast imaging is derived from a high-speed photography technology, which is a technological evolution that the high-speed photography is further expanded to picoseconds (ps) or even femtoseconds (fs) magnitude on a time scale. Early high-speed photography was generally achieved by controlling the camera imaging process exposure time through a high-speed mechanical shutter. Due to the limitation of the mechanical movement speed of the macroscopic object, the mechanical shutter is openedThe closing speed reaches 10 ^-5 The s magnitude later encounters a technical bottleneck. With the breakthrough development of electronic and photoelectric technologies in the middle of the last century, the response speed of high-speed photographic shutters is rapidly developed due to the emergence of various novel shutter technologies based on electronic, electromagnetic, electrooptical or magneto-optical technologies and the like. The faraday box based on magneto-optical technology, taught by the university of majordson's institute of technology in the last forty years, developed an ultra-high speed camera called Rapatronic camera with an exposure time as short as 10 ^-8 s (10 ns). Using a Rapatronic camera and controlling the shutter exposure time at 2X 10 ^-6 And s, the human beings firstly realize the clear imaging of the atomic bomb explosion transient process. As another example, in the last three decades, with the maturation of lsi technology, the imaging of image sensors (CCD and CMOS chips) in cameras is rapidly replacing the conventional film imaging, and the shutter control of cameras is becoming more convenient. The imaging exposure process can be controlled by the traditional mechanical shutter by controlling the opening or closing of the pixels of the imaging chip through the chip circuit. This way of controlling the exposure time of the camera by the imaging chip is called electronic shutter, which has low cost and can obtain high shutter response speed (the exposure time can be as small as 10) ^-5 Of the order of s). Currently, professional high-speed cameras based on high-speed CMOS are available as short as 10 ^-7 shutter exposure times of the order of s, whereas professional high-speed cameras based on image-enhanced CCD or CMOS (ICCD or ICMOS) can obtain as short as 10 ^-10 shutter exposure time of the order of s. In recent years, compressed ultrafast imaging (CUP) combining streak camera and image digital coding technology can be obtained as short as 10 ^-12 And the exposure time of s magnitude realizes direct imaging of the transient process of light pulse space propagation. Recently, a time resolution of 0.58ps (determined by the streak camera limit time resolution) has been available for T-CUP based on the CUP further development. In addition, based on the optical Kerr shutter realized by inducing the optical Kerr effect by the ultrashort laser pulse, the ultrafast imaging can be realized as short as 10 ^-13 Exposure times of the order of s. The high-speed photography and the ultra-fast imaging technology are based on the shutter technology to realize the control of the exposure time of the camera. In technical terms, this is a passive exposure time control technique.

In addition to passive exposure time control, active exposure time control is also widely used in high-speed photography and ultra-fast imaging. The last fifties of the century also is the Eggeton professor invented a high-speed photography technology based on an electronic flash lamp technology, and a crown-shaped structure formed in a milk dropping cup shot by the electronic flash lamp technology shows wonderful and wonderful moments when a dynamic change process of fluid is solidified. For high speed imaging, egyton believes that controlling the light illuminating the subject is more convenient than controlling the shutter: at the time flash lamps filled with mercury vapor and other gases were used, as short as 10 a have been achieved ^-6 s, and the brightness exceeds that of sunlight. Based on the flash lamp technology, Ejegton realizes high-definition imaging of microsecond (mu s) time scale transient process of the bullet passing through the apple; the idea of high-speed and high-definition imaging is obtained by controlling the light of the shot object by Ejekton, and a foundation is laid for an important technical path of subsequent ultrafast imaging, namely, the shot object is illuminated by using ultrashort laser pulses. In recent decades, with the development of ultrashort laser pulse technology, the active exposure time control ultrafast imaging technology based on ultrashort laser pulse illumination has been developed greatly, because its time resolution depends on the time width (pulse width) of the ultrashort pulse: as the laser pulse width is developed from the early mu s scale to the recent fs scale, the ultrafast imaging time resolution based on laser pulse illumination is also 10 ^-6 s has progressed to 10 ^-13 Of the order of s. On the other hand, by combining with an ultrafast imaging technology based on laser pulse illumination, the schlieren imaging technology can also obtain ultrahigh time resolution, which provides powerful technical support for transient imaging representation of dynamic characteristics of a high-speed moving object in the fluid.

For the ultra-fast imaging technology, besides the above-mentioned core technical parameter of single exposure time resolution, it is also a significant problem whether multi-frame imaging with high frame rate can be realized. Especially for ultra-fast imaging of a single trigger event, the evolution of the dynamic characteristics of the ultra-fast imaging needs to be observed by imaging multiple frames of the same event with different time delays. In fact, the realization of high frame rate multi-frame imaging of the same event in the ultra-fast imaging process often has higher technical challenge than the improvement of the single-frame imaging time resolutionSex and complexity. In particular, for passive exposure controlled ultrafast imaging, the most straightforward multi-frame imaging can be achieved by controlling the shutter to repeatedly open and close at a certain time delay. Therefore, for multi-frame high-speed photography with single shutter exposure, the limit frame rate is determined by the limit capability of the shutter repetitive switch. In general, the extreme time delay of a single shutter repetitive switch is limited by the mechanical or electrical characteristics of the entire shutter system at high speed, which is often much longer than the exposure time of the shutter. For example, a current civil camera based on electronic shutter technology is available 10 ^-5 s exposure time, but its limit frame rate is limited by the transmission bandwidth between the imaging chip and the information storage system, which can only reach about 1000 frames per second (fps); for mechanical shutters, limited by the stability of the high-speed motion of the macro-mechanical components, the limit frame rate is much lower than for electronic shutters. Of course, some professional high-speed cameras can greatly increase the limit frame rate to 10 by using the memory of the camera to cache information and reduce the size of picture pixels ⁶ On the order of fps, but such high frame rates are achieved at the expense of significantly increased camera manufacturing costs and sacrifice in image resolution.

In fact, when the single shutter frame rate cannot meet the detection frame rate requirement during high-speed photography, an imaging frame rate much higher than the single shutter frame rate can be obtained by a multi-camera (shutter) parallel photography mode in which a specific shutter time delay is set. For example, when the united states military uses a Rapatronic camera to image the explosion process of the atomic bomb, 10 shutter opening time delays are set to be 10 ^-5 Parallel shooting by Rapatronic camera of s to obtain frame rate of 10 in atomic bomb explosion process ⁵ fps 10 consecutive transient frames. However, this method of obtaining a high frame rate in a multi-camera parallel manner is premised on a linear increase in device cost. In recent years, with the advent of the compressed ultrafast imaging (CUP) technique, high frame rate ultrafast imaging based on the passive exposure technique has been newly developed. The ultrafast imaging method combining the image digital coding technology and the compressed sensing image processing technology breaks through the limitation that the traditional high frame rate technology depends on the development of hardware equipment, and the ultrafast imaging frame rate and the frame number are obviously improved by introducing the image processing leading edge algorithm. First generation CUPOne shot imaging has been available 10 ¹¹ frame rate of fps and frame number of 350; while the frame rate of T-CUP one-shot imaging is increased to 10 ¹³ fps, and frame number can also exceed 300. However, the multi-frame ultrafast imaging technology relies on a stripe camera in hardware, and the equipment cost is high. Although the improved CUP can reduce equipment cost to some extent by using an ultrafast electro-optical deflector instead of a stripe camera, its index is still a gap compared to the stripe camera based technology. On the other hand, although optical Kerr shutters based on ultrashort laser pulse induction can achieve as short as 10 ^-13 s, but this method is relatively complicated in multi-frame imaging technology to achieve a high frame rate. Theoretically, the method needs to add one more delay pulse for each additional frame, and how to distinguish different frames in the detection system is more difficult technically. In short, how to develop a more general technical scheme for multi-frame ultrafast imaging based on passive exposure control greatly reduces the equipment cost is a technical problem to be solved urgently.

For ultra-fast imaging based on active exposure control, early multi-frame high-speed photography was mainly achieved by repeated flashes of a flash lamp. For example, by using a stroboscopic light source capable of generating high-repetition-frequency light pulses and controlling stroboscopic and shutter synchronous, Egerton teaches to shoot multiple frames of images of transient processes forming crown-shaped structures in a milk dropping cup, and the highest frame rate of the images exceeds 10 ⁴ fps. Similarly, ultra-fast imaging based on ultra-short pulse active illumination can achieve multi-frame acquisition by repeated illumination of multiple pulses with a specific time delay. However, because of the extremely short pulse width of the ultrashort pulse, in order to obtain multi-frame imaging with a frame interval close to the pulse width, the following two problems need to be solved: (1) how can multiple ultra-short pulses (equal-height pulse trains) with precise time delays with similar characteristics be obtained? (2) How do imaging frames formed by different illumination pulses differentiate recordings? Problem (1) can be achieved by performing multi-path isocandela splitting and independent delay control on the input pulse. However, if the delay pulses of different paths are not specifically modulated or operated, the imaging frames formed by different delay pulse illuminations will be overlapped, and information overlapping is easy to occur, so that no image is formedThe method clearly distinguishes each imaging frame and cannot meet the requirements of many practical ultra-fast detection. Therefore, in the actual ultrafast imaging technique, the researcher needs to consider the problem (2) with importance. In high-speed photography, since the frame rate is not particularly high, frame image shooting for each illumination pulse can be realized by controlling the strobe in synchronization with the shutter. However, in ultrafast imaging, the frame rate is very high, and the shutter repetition switching frequency is often difficult to reach the very high repetition frequency of the active illumination ultrashort pulse. In fact, if the shutter repetition switching frequency can be increased to meet such a requirement, it is possible to directly realize ultrafast imaging by passive exposure control without realizing ultrafast imaging by active exposure control. Thus, for ultra-fast imaging with active exposure control, differentiated capture of high frame rate multi-imaging frames is generally achieved by modulating the illumination pulse train. This modulation can be done by manipulating some inherent physical quantity (polarization, wavelength, etc.) of the optical pulse to obtain a specific code. For the case of only two frames, two frame discrimination can be achieved by having the two illumination pulses have mutually perpendicular polarizations; for the situation of multiple frames, the wavelength of the illumination pulse can be regulated, so that different time delay pulses have different wavelengths, and then the multiple frames are distinguished by the spectrum distinction in the imaging process. For example, continuous timing total optical mapping photography (STAMP) technology, wavelength discrimination of different time delay pulses is realized by a scheme of generating dispersion by a wide spectrum pulse, and the frame rate can reach 4.4 multiplied by 10 ¹² fps. In the STAMP technique, the image acquisition end uses a special spatial mapping device to spatially separate the different wavelength pulses with specific time delay, so as to form images with different time delay frames in different areas of the image sensor. The use of the special frame separation equipment module leads to the complicated equipment scheme of the wavelength-division active exposure multi-frame ultrafast imaging technology such as STAMP, and the equipment is not easy to match with a standard imaging system while the manufacturing cost is higher, so that the equipment is difficult to be widely popularized.

For multi-frame ultrafast imaging, besides core technical parameters such as imaging time resolution and frame rate, imaging spatial resolution is also a technical parameter which needs to be focused. Modern optical high resolution imaging generally meets the requirements for different resolution application scenarios through optical microscopes of different performance specifications. However, the current multi-frame ultrafast imaging technology cannot be directly matched with various mature microscopic imaging technologies, and therefore, some technical advantages obtained by long-term development in the field of microscopic imaging cannot be inherited, particularly the advantages in the aspects of imaging resolution and quality. The spatial resolution of the current multi-frame ultrafast imaging system is still significantly lower than the limit of the mainstream microscope, which is significantly related to that the current multi-frame ultrafast imaging system cannot be directly constructed based on the standard microscopic imaging system: due to the particularity and complexity of the technology, high-spatial resolution imaging cannot be realized by using a high-numerical-aperture objective lens, and a Kohler illumination technology cannot be combined (an optical microscope adopts Kohler illumination to realize uniform illumination, so that excellent microscopic resolution capability and imaging quality can be obtained).

On the other hand, various modern optical imaging devices commonly achieve digital acquisition of images through an image sensor of a digital camera. However, the current multi-frame ultrafast imaging system cannot directly realize multi-frame ultrafast imaging through a digital camera configured in a standard manner, and can be realized only after different frame information is spatially separated through a camera front-end device. The space mapping equipment in the STAMP technology realizes the space separation of delay frames corresponding to different wavelengths through grating dispersion; in the CUP technology, a stripe camera is used for realizing the spatial offset of different delay frames, and the reconstruction of the spatial offset frames is realized through an image digital coding technology. The special equipment at the front end of the camera not only greatly improves the equipment cost of a multi-frame ultrafast imaging system, but also limits the improvement of the limit space-time resolution. As in the STAMP technique, the dispersion grating used for frame space separation leads to an increase in imaging dispersion and a loss in imaging spatio-temporal resolution; the operation of the streak camera in the CUP technology depends on the deflection of photoelectrons, and coulomb interaction generated under high density easily causes image distortion and hinders the improvement of imaging space-time resolution.

In summary, how to develop a technical scheme that can be matched with a standard imaging system, is based on direct imaging of a digital camera, and has high application universality for various active or passive exposure multiframe ultrafast imaging methods, the technical space for further improving various core technical indexes can be provided while the equipment complexity and the construction cost are greatly reduced, and the method is a core technical problem to be solved in the field of ultrafast imaging at present.

Disclosure of Invention

The invention provides an ultrafast imaging method based on a color digital camera, aiming at solving the problems of the defects and shortcomings of the prior art.

In order to realize the purpose of the invention, the technical scheme is as follows:

an ultrafast imaging method based on a color digital camera, the method comprising the steps of:

s1: controlling the output wavelength range of the super-continuum spectrum light source to cover a single super-continuum spectrum ultrashort pulse of all color channels of the color digital camera;

s2: splitting the super-continuum spectrum ultrashort pulse to obtain each path of primary color ultrashort pulse with the wavelength range respectively in each color channel wavelength range of the color digital camera;

s3: the delay time between pulses is controlled for each path of primary color ultrashort pulse, so that the imaging frame rate of a subsequent multi-frame ultrafast imaging process is controlled;

s4: combining beams of the primary color ultrashort pulses to realize the common-beam propagation of the primary color ultrashort pulses in space, and obtaining a multi-primary color ultrashort pulse string with specific time delay between a beam of pulses;

s5: the multi-primary ultrashort pulse string is used as an imaging active illumination light source to irradiate an imaging target, and the multi-primary ultrashort pulse string active exposure time control of the imaging target is realized;

s6: an imaging target irradiated by the multi-primary-color ultrashort pulse train is imaged on an image sensor of the color digital camera through an imaging light path;

s7: controlling a shutter of the color digital camera to expose for a certain time at a single time to realize shooting of an ultra-short pulse train lighting instant scene imaged on an image sensor;

s8: and acquiring gray level images of all color channels of images shot by the color digital camera to obtain a plurality of ultrafast imaging frames which correspond to all primary color ultrashort pulse illumination instantaneous scenes and are arranged according to time sequence, so as to realize multiframe ultrafast imaging.

Preferably, in step S1, the synchronous control system is adopted to send out a trigger signal to control the supercontinuum light source to output a single supercontinuum ultrashort pulse, and send out a trigger signal with a certain time advance to the color digital camera, so as to control the opening and subsequent closing of the camera shutter.

Further, step S2, splitting the super-continuum ultrashort pulse by using a beam splitting dichroic mirror group to obtain each channel of primary color ultrashort pulse corresponding to each color channel band of the color digital camera; the said each way of base color ultrashort pulse carries on the centre wavelength selection and spectral width control through the band-pass filter respectively, make the wavelength range locate in the color digital camera each color channel wavelength range separately, realize the regulation and control to the ultrashort pulse time width of each way of base color while reducing the color crosstalk among each way of base color; the energy of each path of primary color ultrashort pulse is controlled through a continuously adjustable neutral filter to obtain a certain energy proportion of each primary color pulse, so that an imaging target irradiated by a subsequent multi-primary color pulse string has an approximate white balance effect when the imaging target is imaged on a color digital camera.

Still further, in step S3, a delay line group or a delay fiber group is used to perform inter-pulse delay time control on each path of the primary-color ultrashort pulse.

Still further, in step S4, a beam-combining dichroic mirror set or an optical fiber beam combiner is used to combine the channels of the primary-color ultrashort pulses.

Further, step S5, the imaging target is illuminated by direct illumination or kohler illumination, so as to realize active exposure time control of the multi-primary ultrashort pulse train of the imaging target.

Still further, in step S6, the imaging target irradiated by the multi-primary-color ultrashort pulse train is clearly imaged on the color digital camera image sensor by using the optical microscope imaging optical path or the camera lens imaging optical path.

And step S7, based on the trigger signal with a certain time lead sent by the synchronous control system to the color digital camera, controlling the color digital camera shutter to expose once for a period of time that can completely cover the whole course of the ultrashort pulse string illumination, so as to realize the shooting of the ultrashort pulse string illumination instantaneous scene imaged on the image sensor.

Still further, in step S8, the gray scale image of each color channel of the image captured by the color digital camera is obtained through the direct color channel separation output inside the color digital camera or the external subsequent color channel separation processing.

Still further, the color digital camera types include a multispectral camera and a hyperspectral camera, that is, the color channel number range of the color digital camera is the case of three primary colors or higher than the case of three primary colors; the color channel wavelength range of the color digital camera is from a visible light wave band, or from a visible light wave band to an infrared wave band, or from an ultraviolet wave band to a visible light wave band, or from an ultraviolet wave band to an infrared wave band.

The invention has the following beneficial effects:

the invention generates a plurality of primary color ultrashort pulses with specific time delay and corresponding to each color channel (primary color) wave band of a color digital camera, uses the primary color ultrashort pulse strings as an imaging active illumination light source to irradiate an imaging target, then shoots an image formed by irradiating the imaging target on a camera image sensor through an imaging light path by the color digital camera, and obtains a plurality of ultrafast imaging frames corresponding to each primary color ultrashort pulse illumination instantaneous scene through color channel separation processing, thereby realizing multiframe ultrafast imaging. Compared with the existing active illumination multiframe ultrafast imaging technology based on the wavelength regulation and control technology, the method does not need an additional frame space separation module to realize the physical separation of frames with different wavelengths, but directly has the color (wavelength) space separation function to realize the physical separation and the image acquisition of the imaging frames with different wavelengths based on the imaging and shooting process of the universal color digital camera. That is, the most core technical innovation point of the method of the invention lies in that the multi-primary color space separation function of the image sensor of the color digital camera is ingeniously multiplexed: by generating various time delay ultrashort pulses corresponding to various imaging primary color wave bands of the color digital camera as an active illumination light source to irradiate an imaging target, the primary color space separation function of the color digital camera can be directly utilized in the transient image shooting process, and the space separation of different wavelength ultrafast imaging frames in multi-frame ultrafast imaging of active illumination in different wavelengths is realized.

The invention relates to a multi-frame ultrafast imaging method with high universality, which is compatible with various conventional optical imaging methods, obviously reduces the equipment complexity and the construction cost of a multi-frame ultrafast imaging system, and simultaneously can fully utilize the technical advantages of the conventional optical imaging method developed for a long time to improve the technical indexes of the multi-frame ultrafast imaging system. Specifically, in an active exposure ultrafast imaging system, a common method for realizing multiframe separation, such as the STAMP technology, is used for performing different-band light splitting on an ultrashort pulse illumination light source. However, in the prior art, a frame separation module is required to realize the spatial separation acquisition of different color frames, which not only increases the complexity of the system, but also causes the problem that the system cannot be directly matched with a conventional imaging system (cannot be directly established or upgraded based on the conventional imaging system) and fully utilizes the technical advantages of the conventional imaging system, thereby limiting the space for technical development, such as the improvement of the ultimate spatial resolution. Aiming at the problem, the method realizes the spatial separation of multi-primary color frames by utilizing the technical characteristic that a conventional color digital camera, such as a camera based on Bayer (Bayer) type or prism type light splitting technology, has the multi-primary color spatial light splitting capability, avoids the use of a special frame spatial separation module, is convenient for establishing a multi-frame ultrafast imaging system based on a conventional imaging system, obviously reduces the complexity and the cost of equipment, and simultaneously obtains good compatibility with the conventional imaging system, namely inherits the technical advantages and the performance indexes. For example, the main body of the multi-frame ultrafast microscopic imaging system established based on the method of the present invention is based on the existing optical microscope and the matched color digital camera, and the multi-primary color ultrashort pulse string generated from the outside is led in from the microscope illumination light source, so that the basic multi-frame ultrafast imaging can be realized. The construction of the system can not influence the light path of the microscope, and the imaging limit spatial resolution of the system can reach or approach the limit of the original microscope.

Besides being beneficial to improving the limit spatial resolution, the invention obtains the multi-primary-color ultrashort pulse string through the super-continuum spectrum light splitting to be used as an active illumination light source, thereby being beneficial to fully utilizing the bandwidth advantage of the super-continuum spectrum and improving the time resolution of the ultrafast imaging. Specifically, according to the relation of the product of the pulse time bandwidth, wider spectrum width corresponds to narrower limit pulse width. For active exposure ultrafast imaging, the ultimate temporal resolution is determined by the illumination pulse width, i.e. the spectral width. On the other hand, in active exposure multiframe ultrafast imaging, multiband light splitting of illumination pulses is a common method for realizing multiframe separation, such as the STAMP technique. In fact, such spectroscopic processing can severely impact the limit temporal resolution of ultrafast imaging: to realize N-frame imaging, the original pulse spectrum is equally divided into N parts, and the maximum spectral width of each part can only be 1/N of the original spectrum, so that the system limit resolvable time can be changed into N times of the original pulse width according to the time-bandwidth product relation, and is obviously lower than the original pulse time resolvable limit. Aiming at the problem, the invention adopts the super-continuum spectrum induced by the ultra-short pulse to replace the original ultra-short pulse as a light source to carry out light splitting corresponding to each primary color wave band of the color digital camera. Firstly, the super-continuum spectrum has super-wide spectrum, can be matched with the whole wave band covered by all primary colors of the color digital camera, and can obtain multi-channel super-short pulses corresponding to the wave bands of all the primary colors of the camera through the light splitting of the specific wave band. Secondly, the super-continuum spectrum has a much wider spectrum width than the original ultra-short pulse, even through multi-band light splitting, the super-continuum spectrum can still obtain a wider spectrum width than the original ultra-short pulse, namely, higher limit time resolution, and breaks through the limitation of the existing wavelength light splitting technology in the aspect of the limit time resolution. Finally, for the mainstream three-primary-color technology of the color digital camera, each primary-color wave band has a spectral width of about 100nm, so that theoretically the limit pulse width of each primary-color ultrashort pulse is close to 10fs, namely the multiframe ultrafast imaging of the invention can obtain the limit time resolution close to 10fs, and is obviously superior to the current wavelength-splitting multiframe ultrafast imaging method.

The invention can realize modularized construction, and each module has higher technical universality. Therefore, the method has high setting flexibility and portability, can adapt to wider ultra-fast imaging application scenes, and can solve the problem of weak portability caused by strong specificity of a common multi-frame ultra-fast imaging system to a certain extent. Specifically, the multi-wavelength frame space separation of the method is based on the conventional color digital camera technology, and the complexity and the specificity of a multi-frame ultrafast imaging system are obviously reduced, so that the method can be built through the following modules: 1) a super-continuum spectrum light source and a synchronous control module; 2) a multi-primary ultra-short pulse generation and control module; 3) a multi-primary color pulse illumination and target imaging module; 4) and the color digital camera shooting and color channel separation module. The modules can be realized by a mature technical scheme, and have high technical feasibility, high universality, high setting flexibility and application portability. For example, 1) the supercontinuum can be generated by ultra-short pulses propagating in various white light nonlinear media (dielectric, gas, or fiber, etc.); 2) the multi-primary-color pulse time delay manipulation can be realized by a delay line or a delay optical fiber group, and the pulse width manipulation can be realized by a band-pass filter group; 3) the multi-primary color pulse illumination can be realized by means of Kohler illumination or direct illumination, and the target imaging can be realized by an optical microscope (microscopic imaging) or an optical lens (macroscopic imaging); 4) the color digital camera shooting of (1) can be realized by a bayer type (each primary color is spatially separated by a bayer color filter) or a prism type (each primary color is spatially separated by prism light splitting) color camera, and the color channel separation can be realized by hardware (physical channel separation) or software (image information processing). Therefore, the method is flexible in setting, can be transplanted to different ultrafast imaging application scenes, and has more remarkable advantages in the aspect of universality compared with the conventional multi-frame ultrafast imaging method.

Drawings

Fig. 1 is a flowchart of the steps of the ultrafast imaging method described in embodiments 1 and 2.

Fig. 2 is a schematic diagram of an ultrafast imaging system as described in embodiment 1.

Fig. 3 is a schematic diagram of an ultrafast imaging system as described in embodiment 2.

In the figure, 1-1-synchronous control system; 1-2-supercontinuum light source; 2-1-a first beam-splitting dichroic mirror; 2-2-a second beam-splitting dichroic mirror; 3-1, 3-2 and 3-3-band pass filters; 4-1, 4-2, 4-3-neutral filters; 5-1, 5-2, 5-3-delay lines; 6-high reflection mirror; 7-a converging lens; 8-1, 8-2, 8-3-delay fibers; 9-1-a first combined dichroic mirror; 9-2-a second combined dichroic mirror; 10-a fiber optic adapter; 11-3-in-1 optical fiber combiner; 12-an achromatic lens; 13-a scattering sheet; 14-a converging lens; 15-half mirror; 16-a microscope objective; 17-imaging the object; 18-an imaging lens; 19-a camera lens; 20-1-prismatic color digital camera; 20-2-bayer color digital camera; 21-a beam splitting prism; 22-1-black and white image sensor; 22-2-bayer array color image sensor; 23-a camera image information processing module; the 24-1, 24-2 and 24-3 cameras directly separate and output the gray level images of the color channels; 25-processing of synthesizing RGB color image from gray level image of each primary color in camera; 26-the camera outputs the RGB color image; 27-RGB color images; 28-carrying out color channel separation processing on the RGB image outside the camera; 29-1-carrying out interpolation processing on the RAW RGB format image in the camera; 29-2-carrying out interpolation processing on the RAW RGB format image outside the camera; 29-3-performing interpolation processing on RAW format images of all color channels outside the camera; 30-the camera directly outputs the RAW RGB format image; 31-RAW RGB format image; 32-1, 32-2, 32-3-each color channel RAW format image; 33-1, 33-2, 33-3-each color channel gray scale image (each primary color ultrafast imaging frame); 34-arranging the ultrafast imaging frames according to the sequence of the occurrence of the events according to the ultrashort pulse string time sequence; 35-a sequence of chronologically ordered ultrafast imaging frames.

Detailed Description

The invention is described in detail below with reference to the drawings and the detailed description.

Example 1

As shown in fig. 1 and 2, an ultrafast imaging method based on a color digital camera, the method includes the following steps:

s1: the synchronous control system 1-1 is adopted to send out a trigger signal to control the supercontinuum light source 1-2 to output a single supercontinuum ultrashort pulse with a wavelength range covering all color channels of the color digital camera 20, and a trigger signal with a certain timing lead is sent to the color digital camera 20 to control the opening and subsequent closing of a camera shutter.

The super-continuum spectrum light source 1-2 outputs super-continuum spectrum ultrashort pulses with single pulse energy of 1mJ and pulse width close to 2ps, wherein the wavelength range covers all color channels of the color digital camera 20, and the super-continuum spectrum ultrashort pulses are irradiated by the high-energy femtosecond laser pulses through a white light medium method. The pulse has more remarkable time chirp, and the pulse width can be further compressed through dispersion compensation operation such as grating or chirped mirror and the like; meanwhile, the method has low coherence, is beneficial to weakening the speckle effect of coherent light illumination, and improves the spatial resolution and quality of ultrafast imaging.

In the running process of the ultrafast imaging system, the synchronous control system 1-1 is adopted to send out a trigger signal to control the supercontinuum light source 1-2 to output a single supercontinuum ultrashort pulse, and a trigger signal with a certain time lead is sent to the color digital camera 20 to control the opening and the subsequent closing of a camera shutter. In addition, a trigger signal port is reserved in the synchronous control system 1-1 for controlling the trigger time of a single trigger event to be detected; when a single trigger event needs high-precision time delay triggering, the ultra-short pulse light source can be obtained by performing time delay control and high-speed photoelectric conversion after ultra-short pulse light splitting sampling is output by the ultra-continuous spectrum light source 1-2; the high-energy femtosecond pulse of the pump supercontinuum light source 1-2 can be directly used as an excitation source of a single trigger event after light splitting and time delay control.

S2: splitting the super-continuous spectrum ultrashort pulse by using a beam splitting dichroic mirror group 2 to obtain each path of wide-spectrum primary color ultrashort pulse corresponding to each color channel wave band of the color digital camera 20; the said each way of base color ultrashort pulse carries on the central wavelength to choose and control the frequency spectrum width through the band-pass filter 3 separately, make the wavelength range in the color digital camera each color channel wavelength range separately, realize the regulation and control to the ultrashort pulse time width of each way of base color while reducing the color crosstalk among each way of base colors; the energy of each path of primary color ultrashort pulse is controlled through the continuously adjustable neutral filter 4 respectively to obtain a proper energy proportion of each primary color pulse, so that an imaging target irradiated by a subsequent multi-primary color pulse string has an approximate white balance effect when the imaging target is imaged on a color digital camera.

In this embodiment, the beam splitting dichroic mirror group 2 is used to split the super-short pulse with super-continuum spectrum in the visible light band, so as to obtain three primary color pulse lights corresponding to the wavelengths of the three color channels of red (R), green (G), and blue (B) of the color digital camera 20-1. The beam splitting dichroic mirror group comprises a first beam splitting dichroic mirror 2-1 and a second beam splitting dichroic mirror 2-2 arranged behind the first beam splitting dichroic mirror 2-1; the first beam splitting dichroic mirror 2-1 is used for reflecting a blue light wave band (wavelength range 390-490nm) of a visible light wave band supercontinuum and transmitting the other wave bands; the second beam-splitting dichroic mirror 2-2 is used for reflecting the green light band (wavelength region 490-580nm) and transmitting the red light band (wavelength region 580-690nm) in the rest bands.

In the embodiment, the adjustment of the light splitting sequence of the three primary colors of red, green and blue can be realized by changing the configuration of the transmission and reflection wavelengths of the two dichroic mirrors of the beam splitting dichroic mirror group 2; in this embodiment, the light splitting may also be realized by using an RGB beam splitter prism, which emits the incident supercontinuum directly along different directions according to three primary color bands of the color digital camera 20-1, so as to realize more compact RGB three-primary color pulse light splitting.

Through the light splitting operation, three primary color pulse lights with similar spectral widths (spectral widths) and matched with three primary color wave bands of the color digital camera 20-1 can be obtained, and a prerequisite is provided for further carrying out accurate adjustment on the spectral widths (pulse widths) and energy of the primary color pulses: the ultra-short pulses of each primary color respectively carry out center wavelength selection and spectral width control through the band-pass filter 3; the energy of each path of the primary color ultrashort pulse is controlled through the continuously adjustable neutral filter 4.

Specifically, since the broadband supercontinuum output by the supercontinuum light source 1-2 has a relatively significant time chirp, the red, green and blue primary color pulses split by the beam splitting dichroic mirror group 2 also have a relatively significant time chirp. Therefore, by adjusting the central wavelength and the spectral width of the pulse light with the primary colors of red, green and blue, the pulse width of the pulse light with the primary colors of red, green and blue can be compressed, and the ultrashort pulse with the primary colors of red, green and blue with hundred fs magnitude and similar pulse width can be obtained. For example, the three primary color pulse lights of red, green and blue respectively pass through the 10nm bandwidth band-pass filters 3 with center wavelengths of 650nm (R path 3-3), 540nm (G path 3-2) and 450nm (B path 3-1), so as to obtain three primary color pulses with a full width at half maximum (FWHM) pulse width of about 200 fs. It should be noted that, because the time bandwidth product of the ultra-short pulse at the transform limit is constant, the too narrow spectral width of the RGB three primary colors may cause the pulse width of the three primary color pulse to widen (in this embodiment, the widening phenomenon of the three primary color pulse can be observed by using the 3nm bandwidth band pass filter 3). In addition, too narrow a spectral width of the RGB primaries also results in a significant reduction of the pulse energy of each primary. In addition, for the broadband supercontinuum pulse with significant chirp (far from the transformation limit) in the present embodiment, the reduction of the spectral width of the RGB three-primary-color pulse leads to the improvement of the coherence of the pulse light, which is disadvantageous for reducing the laser speckle effect and improving the imaging quality.

On the other hand, the central wavelengths of the three primary color pulse lights are set at positions of the red, green and blue wave bands close to the center, and the relatively small spectral width is set, so that the color crosstalk of three color channels of the color digital camera 20 during the ultrafast imaging detection can be remarkably reduced, and the signal-to-noise ratio and the sensitivity of the imaging of each color channel are improved. This is because, for the wavelength response curves of the three color channels of the color digital camera 20, in the central wavelength region of a certain primary color band, the corresponding primary color light has the highest response sensitivity, and the color channel only responds to the corresponding primary color light and does not respond to the other two primary color lights. In contrast, in the wavelength region adjacent to the two primary color bands, the wavelength response curves of the two primary color lights are overlapped to some extent, so that any one of the two color channels responds to the two primary color lights, and further color crosstalk, that is, crosstalk between imaging frames, is caused, and the imaging contrast is reduced.

In addition, for multi-frame active exposure ultrafast imaging technology, the illumination pulses of different frames should achieve a specific irradiation light intensity at the imaging target, so that the plurality of captured imaging frames have a consistent and appropriate reference brightness. In this embodiment, the energy of the ultrashort pulse with three primary colors of red, green and blue should be respectively adjusted to appropriate values (meeting a specific ratio) to achieve that the imaging target irradiated by the multi-primary-color pulse train has an approximate white balance effect when imaged on the color digital camera. Specifically, the ultrashort pulses with three primary colors of red, green and blue respectively pass through a continuously adjustable neutral filter 4; the imaging brightness state of a shot target is judged by a color digital camera to obtain the reference imaging brightness required by realizing good single-frame imaging (under the condition of insufficient illumination brightness, the imaging brightness when one primary color pulse with the minimum energy is used for illumination is used as the reference imaging brightness), and then the continuous adjustable neutral filters 4 of each path (or the other two paths) are accurately adjusted, so that the reference imaging brightness can be obtained when each primary color pulse irradiates the imaging target 17 respectively, and the accurate control of the energy proportion of the ultrashort pulse of the primary colors of red, green and blue is realized.

S3: and the delay line group 5 is adopted to control the delay time among the pulses of each path of primary color ultrashort pulse, so that specific accurate time delay is obtained among the ultrashort pulses of each path, and the accurate control of the imaging frame rate in the subsequent multi-frame ultrafast imaging process is realized.

For multi-frame ultra-fast imaging, the time delay between different frames is a parameter which needs to be accurately controlled. In this embodiment, the delay line group 5 is used to precisely control the time delay between the ultrashort pulses with three primary colors, red, green and blue. Wherein, the red, green and blue three primary color light paths are all provided with a high-precision delay line (the displacement precision is 1 μm, the time delay adjustment with the precision higher than 10fs can be realized), and the precise time delay between the three primary color ultrashort pulses can be obtained by precisely adjusting the time delay of each two primary color ultrashort pulses. And the delay line group 5 reduces one path, and can also finish the accurate adjustment of the time delay among three paths of pulses. It should be noted that, in order to make the multiple frames of instantaneous pictures collected by the multiple frames of ultrafast imaging distinguishable in time, the minimum time delay set by the system should match the pulse width of the three primary color pulses. Specifically, for a gaussian envelope pulse, the peak time delay of two adjacent pulses should be greater than the full width half maximum pulse width of two pulses to realize the separation of the two pulses in the time domain. For example, for a 200fs pulse width pulse obtained by a 10nm filter, the inter-pulse delay is greater than 200fs, so that the acquired ultrafast imaging frames can be separated in the time domain. On the other hand, the maximum time delay which can be set by the system is determined by the length of the delay line. For example, the delay line stroke used in the present embodiment is 5 cm. And considering the return of the optical path, the maximum 10cm delay optical path can be obtained by the delay optical path, and the corresponding time delay is 333 ps. In addition, when the optical path is arranged, the propagation direction of incident light and emergent light of each path of delay line should be parallel to the motion direction of the delay line, so that the propagation direction of the emergent light is unchanged after the time delay is adjusted.

S4: and a beam-combining dichroic mirror group 9 is adopted to combine beams of each path of primary color ultrashort pulse, so that the common beam propagation of each path of primary color ultrashort pulse in space is realized, and a multi-primary color ultrashort pulse string with specific time delay among one beam of pulse is obtained.

In this embodiment, the beam combination dichroic mirror group 9 is adopted to combine beams of the ultrashort pulses of each primary color, so as to implement the collimated and co-beam propagation of the ultrashort pulses of each primary color in space. To realize the common-beam propagation of the ultrashort pulses of three primary colors, red, green and blue, the spatial positions and propagation directions of the three primary-color pulse beams should be consistent, and a good beam-combining state can be maintained at a longer propagation distance (e.g., the propagation distance of the whole imaging system). In fact, the beam combination is the inverse process of the beam splitting, so that the collimated beam combination for the three primary color ultrashort pulses can be realized by using the same beam combination dichroic mirror set 9, similar to the beam splitting process of step S2. The beam-combining dichroic mirror group comprises a first beam-combining dichroic mirror 9-1 and a second beam-combining dichroic mirror 9-2; the first beam-combining dichroic mirror 9-1 (with the parameters of the second beam-splitting dichroic mirror 2-2) reflects the green light band (wavelength interval 490-580nm) and transmits the red light band (wavelength interval 580-690 nm); the second beam-combining dichroic mirror 9-2 (the parameters of which are consistent with those of the first beam-splitting dichroic mirror 2-1) reflects the blue light wave band (wavelength range 390-490nm) and transmits the rest wave bands. First, the red light path determines the propagation direction of the combined light beam by fine adjustment of the reflection direction of the high-reflection mirror 6 (mounted on a high-precision optical adjustment frame). Then, the green light path realizes the collimation and beam combination with the red light path through the reflection of a first beam combination dichroic mirror 9-1 (installed on a high-precision optical adjusting frame). The spatial position of the first beam-combining dichroic mirror 9-1 is precisely adjusted to enable the red light beams and the green light beams to coincide at the exit end of the dichroic mirror, and then the transmission direction of the green light beams is precisely adjusted by using a lens frame of the first beam-combining dichroic mirror 9-1 to enable the green light beams and the red light beams to be combined at the far end, namely the spatial positions and the transmission direction of the two light beams are consistent (the two-step adjustment can be circularly carried out to achieve a good collimation beam-combining effect). Finally, the blue light path is reflected by a second beam-combining dichroic mirror 9-2 (installed on a high-precision optical adjusting frame) to realize the collimation and beam combination with the red light path and the green light path. Similarly, in the adjusting process in this step, the spatial position of the second dichroic beam combiner 9-2 is precisely adjusted, so that the blue light beam coincides with the red and green light beams at the exit end of the dichroic beam combiner, and then the transmission direction of the blue light beam is precisely adjusted by using the lens holder of the second dichroic beam combiner 9-2, so that the blue light beam and the red and green light beams are combined at the far end, that is, the spatial position of the three light beams is consistent with the transmission direction (similarly, the two-step adjustment can be circularly performed, so as to achieve a good three-path light collimation beam combination effect). By the beam combination operation, the collimation co-beam propagation of the ultrashort pulse with the red, green and blue three primary colors with specific time delay, namely the ultrashort pulse train with the three primary colors can be realized.

S5: the multi-primary ultrashort pulse string is used as an imaging active illumination light source, a Kohler illumination mode is adopted to irradiate the imaging target 17, and the multi-primary ultrashort pulse string active exposure time control of the imaging target 17 is achieved.

In this embodiment, the multi-primary pulse illumination and target imaging module is implemented based on a conventional optical microscope, which has a coaxial epi-illumination system. The tricolor ultrashort pulse string is used as an imaging active illumination light source and is incident to a Kohler illumination system of a conventional optical microscope to illuminate an imaging target, and the control of the active exposure time of the tricolor ultrashort pulse string of the imaging target is realized. Generally, optical microscope illumination sources all have a specific divergence characteristic, rather than a collimated light source. Therefore, the collimated tricolor ultrashort pulse light beam is adjusted to be a divergent light beam, and a divergent origin is arranged near the primary light source, so that the primary light source incident illumination system of the microscope can be replaced, and good coupling with the imaging system of the microscope is realized. Specifically, the collimated three-primary-color ultrashort pulse light beam is converged by the achromatic lens 12, becomes a divergent light beam after being focused, and is incident on a converging lens 14 (in some optical microscopes, the function of the converging lens is realized by a more complex converging lens group) of the imaging illumination system as a microscope illumination light source, and then is reflected by the half-mirror 15 and then propagates along a main optical axis of the microscope, and then is incident on the microscope objective 16 through a rear aperture, and finally irradiates the imaging target 17 through the objective, so that three-primary-color ultrashort pulse active illumination of the imaging target 17 is realized. The focal length of achromatic lens 12 is determined by the characteristics of the microscope imaging illumination system and the sample illumination requirements. If a high illumination brightness is required, a relatively large focal length can be selected, i.e. a relatively small numerical aperture of the illumination light is obtained. In this embodiment, the focal length of achromatic lens 12 achieving such illumination condition is selected to be 6.5 cm. Under such conditions, the high illumination intensity will be relatively localized in the central region of the field of view of the microscopic imaging. In contrast, to achieve a wider illumination brightness distribution that is more evenly distributed across the field of view, a relatively smaller focal length, i.e., a relatively large illumination light numerical aperture, may be selected. In the present embodiment, the focal length of the achromatic lens group 12 (realized with a 10 × long working distance microscope objective lens) for realizing such an illumination condition is selected to be 1.7 cm. Under such conditions, the illumination intensity distribution in the field of view is wider and more uniform, and the microscopic imaging resolution is improved while speckle is reduced, but the intensity is reduced. On the other hand, the longitudinal position of the achromatic lens 12 is determined by the light converging focal position, and the converging focal position is determined by the original light source position of the microscope illumination system, or by the light source position condition for achieving good kohler illumination. In addition, in order to improve the incoherent property of the illumination light (improve the imaging spatial resolution and reduce the influence of laser speckle on the imaging quality), a thin optical scattering sheet 13 (having surface scattering and non-volume scattering properties so as to reduce the influence of the scattering process on the pulse width and energy of the three-primary-color pulse) can be arranged near the convergence position of the three-primary-color pulse light beams (namely the focus position of the achromatic lens 12), so that the coherence of the three-primary-color pulse light beams is reduced after the three-primary-color pulse light beams penetrate through the scattering sheet 13.

The imaging illumination setting can realize the condition of approximate Kohler illumination, so that the ultrafast microimaging system obtains excellent active illumination effect, and the limit optical resolution capability of the microobjective 16 is fully utilized. The method for introducing the three-primary-color ultrashort pulse train into the microscope illumination system has universality, and is suitable for illumination systems of various conventional optical microscopes, namely, the coaxial epi-illumination system, other epi-illumination systems and transmission illumination systems.

S6: the imaging target 17 irradiated by the multi-primary color ultra-short pulse train is clearly imaged on the image sensor 22 of the color digital camera 20 by adopting an optical microscope imaging optical path.

The microscope imaging system of the present embodiment is an infinity corrected optical system, and clear microscopic imaging of an imaging target 17 irradiated by a three-primary-color ultrashort pulse train on an image sensor of a color digital camera is realized through an optical microscope imaging optical path formed by a microscope objective 16 and an imaging lens 18. In the imaging process, coaxial epi-tricolor ultrashort pulses with specific time delay sequentially irradiate an imaging target 17 on a focal plane of a microscope objective 16, reflected and scattered light of the coaxial epi-tricolor ultrashort pulses is collected by the microscope objective 16 and then reversely propagates along an optical axis of an imaging optical path, and finally, clear microscopic imaging on an imaging sensor 22-1 (an imaging lens imaging surface) of a color digital camera 20-1 is achieved through convergence of an imaging lens 18. Because the whole microscopic imaging system has good achromatic property and the three primary color pulses have coaxial property, the imaging surfaces corresponding to the primary color pulses respectively irradiated by the three primary color pulses of the imaging target 17 have approximately consistent property. As shown in this embodiment, the optical imaging process of the method of the present invention is compatible with conventional optical imaging equipment, and therefore, the method can be directly built based on conventional optical imaging equipment, such as optical microscopes in various forms, various types of optical imaging lenses, and even optical telescopes.

S7: based on the trigger signal with a certain time lead sent to the color digital camera 20 by the synchronous control system 1-1, the shutter of the color digital camera 20 is controlled to be exposed once for a period of time capable of completely covering the whole course of the ultrashort pulse string illumination, so that the shooting of the instant scene of the ultrashort pulse string illumination imaged on the image sensor 22 is realized.

The color digital camera described in this embodiment is a prism-type color digital camera 20-1, and its image sensor is a black-and-white image sensor 22-1, the number of which is three (3CCD or 3 CMOS). The gray scale image formed by irradiating the imaging target 17 with ultrashort pulses of each primary color is shot respectively by using a prism type color digital camera 20-1 for a certain time of single exposure. Specifically, based on a trigger signal which is sent to the color digital camera by the synchronous control system 1-1 and has a certain time advance, the shutter of the prism-type color digital camera 20-1 is controlled to be opened, so that the black-and-white image sensors 22-1 on the image planes of the primary color paths are exposed for a period of time which can completely cover the whole illumination process of the primary color pulses, and the shooting of the instant scene formed by irradiating the imaging target 17 with the primary color ultrashort pulses imaged on the black-and-white image sensors 22-1 is realized. The beam splitting prism 21 of the prism type color digital camera 20-1 on the image plane of the microscopic imaging light path can realize beam splitting of three primary colors of red, green and blue, so that instantaneous microscopic scenes formed by irradiating the imaging target 17 with three primary color pulses are imaged on the black-and-white image sensor 22-1 corresponding to the primary color path respectively. That is, the transient image captured by the monochrome image sensor 22-1 of each primary color channel is a transient imaging frame corresponding to different active exposure times for a single trigger event in multi-frame ultrafast imaging. In the multi-frame ultrafast imaging method of the present invention, the most central technology is that the wavelength range of each time delay ultrashort illumination pulse is set within the wavelength range of each color channel of the color digital camera 20, respectively — a plurality of ultrafast imaging frames recording different instantaneous scenes are encoded by the wavelength of each primary color ultrashort pulse, and the encoding wavelength corresponds to each color channel of the color digital camera, so that the color digital camera having the capability of multi-primary color space separation itself can directly realize the space separation (wavelength decoding) of these common-beam multi-primary color ultrafast imaging frames. In this embodiment, the beam splitter prism 21 of the prism-type color digital camera 20-1 realizes spatial separation of three color channels, that is, spatial separation of different instantaneous ultrafast imaging frames by the multi-frame ultrafast imaging system, and further direct image acquisition of each ultrafast imaging frame can be realized by the corresponding black-and-white image sensor 22-1.

In this embodiment, the shutter opening time of the prism color digital camera 20-1 for image acquisition is controlled by an external trigger signal sent by the synchronous control system 1-1, so as to ensure that the whole illumination process of the illumination pulse train can be captured when the prism color digital camera 20-1 performs transient imaging frame shooting. Before the illumination pulse train illuminates the imaging target 17, the shutter of the prism-type color digital camera 20-1 should be fully opened (even if the electronic shutter is used, the shutter needs a certain time from triggering to pixel full-open state, and the full-open time of the rolling shutter needs to be longer than that of the global shutter), then the shutter is kept opened for a period of time covering the whole illumination process of the pulse train, and the shutter is closed after the illumination of the illumination pulse train is finished. To ensure the above-mentioned operation sequence of the ultrafast imaging system is normal, the synchronous control system 1-1 is set to issue the trigger signal to the prism-type color digital camera 20-1 at a time earlier than the trigger signal to the supercontinuum light source 1-2 by a sufficient time period (30 ms is a typical required value for the present rolling shutter technology digital camera; 1ms is sufficient for the present global shutter technology digital camera). Thus, based on the trigger signal output to the prism-type color digital camera 20-1 by the synchronous control system 1-1, the electronic shutter system of the prism-type color digital camera 20-1 controls the three-way black-and-white image sensor 22-1 on the image plane to start working and expose for enough time (which needs to be longer than the trigger advance time plus the pulse train time), so as to realize the complete acquisition of the imaging frame of each primary color. However, if the exposure time of the camera is too long, the adverse effect of ambient light on imaging will be aggravated, resulting in too high brightness of the static background in the imaging frame, and significantly reducing the transient image contrast.

The prism type color digital camera 20-1 of the present embodiment records the imaging information of each color light of the road base by a black and white image sensor 22-1 through a prism 21 light splitting technology, so as to form three images corresponding to each color channel with the same view field and the same view angle. Compared with a single-image sensor color imaging technology based on Bayer color filter array light splitting (CFA, 4 multiplied by 4 color filter array, which is composed of 8 green, 4 blue and 4 red pixels), the prism light splitting-based three-image sensor color imaging technology has the advantages that high isolation is easier to obtain by spectral response curves among color channels, namely, color crosstalk between adjacent primary color intervals is weaker, the prism light splitting realizes transmission and reflection light splitting of different wave bands through an optical hard film, compared with a Bayer array transmission type color filter, wavelength-dependent absorption light splitting is realized through each primary color photoresist (soft polymer dye), and an RGB band-pass wave band with a steeper band edge is easier to obtain. Therefore, compared with a single-sensor color digital camera based on a Bayer color filter array, the multi-sensor color digital camera based on the prism realizes multi-frame ultra-fast imaging, and is favorable for obtaining lower color crosstalk between adjacent waveband imaging frames, thereby obtaining better imaging quality. In addition, theoretically, the limit light energy utilization rate of an R channel and a B channel in RGB (red, green and blue) light splitting of the Bayer color filter array can only reach 25%, and the limit light energy utilization rate of a G channel can only reach 50%. And the limit light energy utilization rate of each channel of the prism RGB light splitting can reach 100%. Therefore, compared with the Bayer array light splitting technology, the color digital camera using the prism light splitting technology can obviously improve the energy utilization rate of the illumination pulse and obtain higher imaging brightness. In addition, in the original imaging process of the color digital camera based on the bayer color filter array, each pixel can only generate an accurate value of one color of red, green and blue, and the values of the other two colors are obtained by performing spatial color interpolation algorithm estimation through the camera image information processing module 23(ISP), so that color noise, such as moire, zipper effect, false color, and the like, may be generated, and the color imaging quality is reduced. The accurate values of the red, green and blue colors of each pixel of the color digital camera based on the prism beam splitting technology are recorded in the original imaging process and are obtained without interpolation. The quality advantage of the color digital camera based on the prism beam splitting technology in the aspect of color imaging has definite technical significance for realizing lossless or low-noise reduction of each transient primary color imaging frame.

S8: the gray level images 33 of all color channels of the image shot by the color digital camera are obtained through the direct separation output 24 of the gray level images of all color channels in the prism type color digital camera 20-1 or the external subsequent color channel separation processing 28 of the RGB color image output by the prism type color digital camera, a plurality of ultrafast imaging frames 35 which correspond to the instant scene of the ultrashort pulse illumination of all primary colors and are arranged according to time sequence are obtained, and multi-frame ultrafast imaging is realized.

In this embodiment, after the prism color digital camera 20-1 finishes acquiring the corresponding ultrafast imaging frame by each black-and-white image sensor 22-1, the prism color digital camera 20-1 can generally output the acquired image information in two ways:

1) the inside of the camera directly separates and outputs 24 red, green and blue color channel gray scale images 33 shot by the three-way black-and-white image sensor 22-1 through three signal channels, namely, the synchronous separation output of three ultrafast imaging frames 33 is directly realized.

2) The camera first performs RGB color image synthesis 25 on the grayscale image 33 of each color channel captured by the three-way monochrome image sensor 22-1 through the internal image information processing module 23(ISP), and then outputs 26 the RGB color image 27 obtained by superimposing instantaneous scenes of the frame sequentially irradiated by three-primary-color ultrashort pulses on the imaging target 17.

Then, the image output by the prism type color digital camera 20-1 is subjected to subsequent processing to obtain three primary color ultrafast imaging frames 33 corresponding to the three primary color ultrashort pulse illumination instantaneous scenes, and then the ultrafast imaging frames are arranged 34 according to the sequence of the occurrence of the events to obtain a sequence 35 of ultrafast imaging frames arranged according to the time sequence, so that the ultrafast transient evolution process of a single trigger event is presented. As described above, the color digital camera 20-1 based on the prism technology can output a captured image in two common ways. Therefore, in order to obtain multiple frames of ultrafast transient images arranged in time sequence of occurrence of events, the output images should be processed correspondingly according to the image output mode of the color digital camera:

1) for the situation that the camera directly and synchronously separates and outputs 24 three primary color ultrafast imaging frames 33 (each primary color ultrafast imaging frame 33 realizes direct color channel separation in the camera through a hardware mode), after three ultrafast imaging frames 33 output by the camera and corresponding to three primary color ultrashort pulse illumination instant scenes are obtained, the ultrafast imaging frames 33 can be directly arranged 34 according to the sequence of occurrence of events based on the time delay relation of the active exposure process of each primary color ultrashort pulse.

2) For the situation that the camera outputs 26 a single frame of RGB color image 27 (each color channel gray-scale image 33 has synthesized an RGB format color image 27 inside the camera, that is, three primary color ultrafast imaging frames 33 are superimposed in the same RGB color image 27), the color channel separation processing 28 of the external RGB color image needs to be performed on the frame of RGB color image 27 output by the camera to obtain three ultrafast imaging frames 33 corresponding to the three primary color ultrashort pulse illumination transient scenes, and then the ultrafast imaging frames 33 are arranged 34 according to the sequence of events based on the time delay relationship of the active exposure process of the ultrashort pulses of each primary color.

In this embodiment, the images output by the color digital camera are subsequently processed in the two ways to obtain the ultrafast imaging frame sequence 35 arranged in the sequence of the occurrence of the events. The time resolution of the multi-frame ultrafast imaging system is determined by the pulse width of the primary color pulse. For example, if the limit spectrum width of the three-primary color channel is fully utilized, the three-primary color ultrashort pulse will obtain the limit pulse width of about 10fs, i.e. the multi-frame ultrafast imaging system will obtain the limit time resolution of about 10 fs. In addition, the imaging frame rate (FPS) of the multi-frame ultrafast imaging system of the present invention is determined by the time delay (Δ t) between the ultrashort pulses of each primary color (FPS ═ 1/Δ t). For example, for the 10fs limit pulse width pulses described above, when the transient imaging frame between pulses is in a critically resolvable situation in the time domain with equal Δ t of 10fs between pulses, the FPS of the system will be as high as 10 ¹⁴ fps. On the other hand, the spatial resolution of the multi-frame ultrafast imaging system of the present invention is mainly determined by the imaging spatial resolution of the optical microscope responsible for target imaging. For example, when the optical microscope of the system uses an objective lens with a numerical aperture of 0.95, the theoretical spatial resolution in the visible wavelength band can be up to about 0.3 μm. For the current multi-frame ultra-fast imaging technology, the time and space resolution indexes are at the forefront of the field.

For the three-sensor (3CCD or 3CMOS) color digital camera based on the prism spectroscopy in this embodiment, each color channel is configured with a black-and-white image sensor for collecting the component values of the corresponding primary colors of each pixel in the RGB color image. In contrast, in a single-sensor color digital camera based on the bayer color filter light splitting technology, each color channel performs pixel (spatial) discrete sampling based on a bayer array by using the same sensor, and then obtains a complete three-primary-color component value of each pixel through interpolation processing. For the multi-frame ultrafast imaging technology of the invention, the color digital camera based on the prism beam splitting technology can provide higher convenience in technical process for the method of the invention: the color digital camera based on the prism beam splitting technology can directly realize the separation output (independent of image information processing) of the ultrafast imaging frame of each primary color on a pure hardware (sensor) level, and the color digital camera based on the bayer color filter beam splitting technology generally needs additional image information processing to realize the pixel interpolation or the separation output of the ultrafast imaging frame of each primary color. Therefore, considering the convenience of the technology, and the aforementioned significant advantages of lower color crosstalk, higher light energy utilization rate, no interpolation color noise, etc., a color digital camera adopting the prism spectroscopy technology is a preferred technical solution as the imaging camera of the method of the present invention.

In addition, when the color digital camera is not based on the prism type RGB three-sensor technology, but is based on other RGB three-sensor (layer) color imaging technology, such as the RGB three-layer photosensitive sensor technology Foveon X3 with sharp color representation, the above-mentioned embodiment of the multi-frame ultrafast imaging method based on the prism type color digital camera 20-1 is still applicable. Similarly, based on the Foveon X3 color imaging technology, direct separate acquisition and output of three primary color ultrafast imaging frames can be realized in the hardware process of image acquisition of the red, green and blue three-layer image sensor without depending on additional image information processing, so that the method also has the advantages of high processing efficiency, high imaging quality, high multi-frame isolation and the like of the embodiment.

Example 2

As shown in fig. 1 and 3, steps S1 and S2 of this embodiment are the same as the corresponding steps of embodiment 1 in technical details, and will not be described in detail here. The steps S3 to S8 in this embodiment are as follows:

s3: the delay time between pulses is controlled by the delay optical fiber group 8 to obtain specific accurate time delay between the ultrashort pulses, so as to realize accurate control of the imaging frame rate in the subsequent multi-frame ultrafast imaging process.

For multi-frame ultrafast imaging, the time delay between different frames is a parameter that needs to be accurately controlled. In this embodiment, the time delay between the three primary color ultrashort pulses of red, green and blue is realized by propagating the delay fibers 8 of different lengths through the three primary color ultrashort pulses. Wherein, the red, green and blue primary color light paths are all provided with a delay fiber 8 with preset length. Unlike a delay line with high displacement accuracy but relatively small limit optical path length, the delay fiber can realize relatively large delay time, such as a delay optical path length in the meter (m) order, and simultaneously save the optical path space. By arranging three delay fibers 8 with different precise lengths, precise time delay among three primary color ultrashort pulses can be obtained. For example, by setting the lengths of the three delay silica fibers to be 1, 2 and 3m respectively, considering that the optical path difference between the pulses is 1m × the refractive index of the silica core and the refractive index of the silica in the visible light band is about 1.46, the time delay between the three primary color ultrashort pulses with about 4.87ns can be calculated. Specifically, the collimated three primary color ultrashort pulses are converged by a converging lens 7 and then are centrally collimated and coupled into a delay fiber 8, and the position of the converging lens 7 is accurately set so that the three primary color ultrashort pulses have equal optical paths when being coupled into the end face of the fiber. Wherein, to realize good coupling, the diameter of the focused spot of the convergent light should be smaller than the diameter of the fiber core of the optical fiber, and the numerical aperture of the convergent light should be smaller than the numerical aperture of the optical fiber. In the present embodiment, the focal length of the condensing lens is selected to be 4 cm; because the three primary color ultrashort pulses have higher peak power, in order to avoid strong light damage to the optical fiber or induce strong nonlinear effect, the delay optical fiber is selected to be a multimode silica optical fiber with a large fiber core diameter of 200 mu m.

In addition to the advantages of obtaining a large pulse delay and achieving a compact optical path arrangement, the method of obtaining a delay by using the delay fiber 8 also has the advantage of easily achieving a standardized delay arrangement: a series of standard delay optical fibers 8 (such as 0.1, 0.11, 0.12, 0.15, 0.2, 0.5, 1, 2, 5 and 10m) are manufactured by taking a series of specific multiples of preset reference length (such as 0.01m) as a length standard, and the standard delay optical fibers 8 are flexibly matched and connected for use (the optical fibers are connected through a standard optical fiber adapter 10), so that a standardized time delay configuration scheme which has a large number of different pulses and takes specific time delay (the corresponding time delay of the optical path of the quartz optical fiber with the reference length of 0.01m as a unit) as a unit can be conveniently realized. It should be noted that, due to chromatic dispersion (such as typical material dispersion and modal dispersion) of the optical fiber, the pulse width of an ultra-short pulse with a certain bandwidth transmitted in the optical fiber is often broadened (determined by the sign of the dispersion coefficient of the transmission medium in the corresponding wavelength band and the chirp characteristics of the pulse), and the broadening degree is in positive correlation with the length of the optical fiber. Therefore, compared with a delay line in which light generates time delay in air propagation (the air dispersion coefficient is far lower than the solid transparent medium dispersion coefficient), the delay fiber in which light generates time delay in solid transparent medium propagation is more suitable for multiframe ultrafast imaging occasions which have low requirements on time resolution (time resolution of ps to ns magnitude) and large pulse time delay. On the other hand, due to pulse broadening caused by fiber dispersion (pulse temporal coherence weakening), and transverse mode mixing caused by multimode fiber (pulse spatial coherence weakening), coherence of ultra-short pulses propagating in the fiber is reduced, which is beneficial for speckle suppression in ultra-fast imaging and improvement of imaging quality.

S4: and a 3-in-1 optical fiber beam combiner 11 is adopted to combine the primary color ultrashort pulses of each path, so that the common beam propagation of the primary color ultrashort pulses of each path in space is realized, and a multi-primary ultrashort pulse string with specific time delay among one beam of pulse is obtained.

In the present embodiment, the three primary color ultrashort pulse beams propagating in the three primary color delay fibers 8 are spatially combined by a 3-in-1 (3 × 1) fiber combiner 11 to realize three primary color ultrashort pulses with specific time delay. Specifically, the three delay fibers are first connected to three input ends of the 3-in-1 fiber combiner 11 through the fiber adapters 10, so that the three ultrashort primary-color pulses propagated in the delay fibers 8 can be directly transmitted into the 3-in-1 fiber combiner 11, thereby realizing the combining of the three pulsed lights, and becoming an ultrashort three-primary-color pulse train propagated in the combining fibers of the 3-in-1 fiber combiner 11. And then the tricolor ultrashort pulse train is emitted out through the output end of the 3-in-1 optical fiber beam combiner 11 to form a free space transmission tricolor ultrashort pulse train light beam with a certain divergence angle (determined by the numerical aperture of the optical fiber). The 3-in-1 optical fiber beam combiner 11 can directly realize good beam combination of multiple primary color time delay pulse beams transmitted in the optical fiber, and does not need to perform multi-dimensional spatial position and direction precise adjustment to realize good beam combination unlike the multi-primary color beams transmitted in space, so that the technology is more convenient. This is also another advantage of using fiber for pulse delay. Even if the 3-in-1 optical fiber beam combiner 11 is not used, the outlets of the three paths of delay optical fibers 8 with coating layers removed at the tail sections are directly flushed, aligned and closely stacked together, and after the three paths of primary color pulse beams exit the optical fibers and are divergently transmitted for a distance which is obviously larger than the diameter of the optical fibers, a good three-path light combining state can be formed.

S5: the multi-primary ultrashort pulse string is used as an imaging active illumination light source, and the imaging target 17 is irradiated in a direct illumination mode, so that the multi-primary ultrashort pulse string active exposure time control of the imaging target 17 is realized.

In the present embodiment, the three-primary-color ultrashort pulse train is used as an ultrafast imaging active illumination light source to directly irradiate the imaging target 17. In addition, the ultrafast imaging system of the present embodiment is set to the camera lens 19 imaging mode, and its typical imaging distance (m order) is much longer than the typical working distance (mm order) of the optical microscope. That is, as an illumination light source of the imaging system, the three-primary-color ultrashort pulse train needs to travel a relatively long distance before irradiating on the imaging target 17. Therefore, the three-primary-color ultrashort pulse train beam with a specific divergence angle output by the optical fiber needs to be converted into a collimated or near-collimated beam, so that the beam still has high irradiation light intensity after being transmitted for a long distance and has the space-time distribution of an irradiation light field similar to a plane. Specifically, the divergent three-primary-color ultrashort pulse string emitted from the 3-in-1 optical fiber beam combiner 11 is collimated by an achromatic illumination converging lens 14 which is 1 time of focal length away from the output end face of the optical fiber. The diameter of the collimated three primary color ultrashort pulse train beam is determined by the focal length of the achromatic lens 14 and the numerical aperture of the multimode fiber: a larger collimated beam diameter can be achieved by increasing the focal length of achromatic lens 14, given the fiber numerical aperture. It should be noted that increasing the diameter of the collimated beam reduces the intensity of the illumination. In addition, the axial position of the achromatic lens 14 can be further adjusted to obtain a divergent or convergent illumination beam according to the actual illumination requirements of the imaging target 17, thereby realizing different far-field illumination conditions. For example, if high illumination brightness is desired, the achromatic lens 14 may be axially positioned away from the fiber output end to focus the illumination beam to meet the illumination brightness requirement. It should be noted that for collimated lighting conditions, the light beam has a planar light field spatial-temporal distribution; and when the light beam is in a divergent or convergent illumination condition and is not positioned at a convergent focus, the light beam has a spherical or near-spherical light field space-time distribution. In the present embodiment, the focal length of achromatic lens 14 placed at 1 x focal length is 10cm, and the diameter of the obtained collimated beam is about 4 cm. In addition, in order to improve the incoherent property of the illumination light (reduce the laser speckle effect of the irradiated surface, weaken the random imaging noise, and improve the spatial resolution and quality of the imaging), a transmission-type thin optical scattering sheet 13 (having a surface scattering property and reducing the influence of scattering on pulse width and energy) can be arranged at the beam combining optical fiber emitting end of the 3-in-1 optical fiber beam combiner 11, so that the coherent property of the tricolor pulse beams is reduced after the tricolor pulse beams penetrate through the scattering sheet 13.

In this embodiment, the collimated three-primary-color ultrashort pulse train beam is used as a light source of the imaging illumination system to directly illuminate the imaging target 17 directly, thereby realizing active exposure time control of the multi-primary-color ultrashort pulse train of the imaging target 17. Unlike the coaxial illumination system of the optical microscope in embodiment 1, the illumination spot of the illumination system in this embodiment has a relatively large size, and can realize imaging of a relatively macroscopic ultrafast transient process. In addition, the direct illumination mode is not limited by the imaging light path of the system, so that more flexible illumination setting can be realized. For example, relative to the direction of motion of the imaging target 17, the imaging illumination angle can be flexibly set according to illumination needs: if the imaging target 17 is set to be vertical to the moving direction, multi-frame ultrafast imaging of the ultrafast dynamic process with fluid characteristics can be obtained by similar schlieren imaging technology; if the imaging target 17 is set to form a certain included angle with the moving direction of the imaging target 17, a multi-frame ultrafast transient image with a three-dimensional view angle and moving in a three-dimensional space of the imaging target 17 can be obtained. In addition, in such a direct illumination mode, the relationship between the optical axis direction of the imaging lens 19 and the illumination light wave vector direction can be flexibly adjusted according to actual needs. Therefore, under the imaging and lighting conditions of the embodiment, the relationship among the optical axis direction of the imaging lens 19, the direction of the illuminating light wave vector, and the moving direction of the imaging target 17 can be flexibly set, so that the multi-frame ultrafast imaging method has high applicability and portability in various application fields and use scenes.

S6: the imaging target 17 illuminated by the multi-primary ultra-short pulse train is imaged sharply on the image sensor 22 of the color digital camera 20 by using the camera lens 19.

In the present embodiment, the imaging module of the ultrafast imaging system employs the camera lens 19 imaging mode (the most common form of optical imaging in industrial and civil scenes) used for conventional optical imaging of cameras. That is, the imaging light path of the ultrafast imaging system is constituted by a single variable focus camera lens 19 having good achromatic properties. In a specific imaging process, collimated three-primary-color ultrashort pulses with specific time delay sequentially irradiate an imaging target 17 (good focusing on the imaging target 17 can be realized by switching to a continuous light source for illumination and adjusting the zoom position of a lens to obtain optimal imaging quality), and transmitted and scattered light of the collimated three-primary-color ultrashort pulses is collected by a zoom imaging lens 19 and then propagates along the optical axis of an imaging optical path, and finally is imaged on an image sensor 22-2 (near the imaging lens imaging focal plane) of a color digital camera 20-2. Because the zoom imaging lens 19 has good achromatic property and the three primary color pulses have coaxial property, the imaging surfaces corresponding to the primary color pulses of the imaging target 17 respectively irradiated by the three primary color pulses have approximately consistent property. The optical imaging process of the method can be compatible with conventional optical imaging equipment, so that the method can be directly built based on the conventional optical imaging equipment, such as optical lenses, microscopes in various forms and even telescopes.

S7: based on the trigger signal with a certain time lead sent to the color digital camera 20 by the synchronous control system 1-1, the shutter of the color digital camera 20 is controlled to be exposed once for a period of time which can completely cover the whole course of the ultrashort pulse string illumination, so as to realize the shooting of the ultrashort pulse string illumination instant scene imaged on the image sensor 22.

The color digital camera described in this embodiment is a bayer color digital camera 20-2, and the single image sensor (CCD or CMOS) thereof is a bayer array color image sensor 22-2. In this embodiment, based on the trigger signal output to the camera by the synchronous control system 1-1, the shutter system of the bayer color digital camera 20-2 controls the exposure of the bayer array color image sensor 22-2 on the image plane for a specific duration, and acquires a single-frame transient color image formed by superimposing the three primary-color ultrashort pulses sequentially irradiating the imaging target 17. In the present embodiment, the bayer single-sensor color digital camera 20-2 is used to realize imaging recording of a transient scene formed by irradiating the imaging target 17 with ultrashort pulses of each primary color. Specifically, the bayer color filter array of the color image sensor 22-2 located at the end of the imaging optical path can realize spatial separation of three primary colors, red, green, and blue, so that the transient scene formed by irradiation of the imaging target 17 with three primary color pulses is imaged on the sensor regions corresponding to the respective color channels of the bayer color filter, respectively. That is, the transient image captured by each color channel of the bayer color filter on the bayer array color image sensor 22-2 is an ultrafast imaging frame with different active exposure times corresponding to a single trigger event in the multi-frame ultrafast imaging. In the multi-frame ultrafast imaging method of the present invention, the most central technology is that the wavelength ranges of the ultra-short illumination pulses with different time delays are respectively set within the wavelength ranges of the color channels of the color digital camera, so the bayer array color image sensor 22-2 with the multi-primary color space separation capability can directly realize the space separation of the common-beam multi-primary color ultrafast imaging frames. That is, a plurality of ultrafast imaging frames recording different temporal scenes are encoded by the wavelength of the ultrashort pulse of each primary color, and the encoding wavelength corresponds to each color channel wavelength of the bayer array color image sensor 22-2. For the embodiment, the bayer color filter of the bayer array color image sensor 22-2 implements spatial separation of three color channels by the camera, that is, physical spatial separation (wavelength decoding) of different instantaneous ultrafast imaging frames by the multi-frame ultrafast imaging system is implemented, and then image acquisition of each ultrafast imaging frame can be implemented through subsequent image information processing (information spatial separation).

In the present embodiment, the shutter opening time of the bayer color digital camera 20-2 for image acquisition is controlled by an external trigger signal sent by the synchronous control system 1-1, so as to ensure that the bayer color digital camera 20-2 can capture the whole illumination process of the illumination pulse train when performing transient imaging frame shooting. Before the illumination pulse train irradiates the imaging target 17, the shutter of the bayer color digital camera 20-2 should be completely opened, then kept opened for a period of time covering the whole irradiation process of the pulse train, and then closed after the illumination of the illumination pulse train is finished and the imaging target 17 is irradiated. To ensure the above-mentioned operational sequence of the ultrafast imaging system is normal, the timing of the trigger signal sent to the bayer color digital camera 20-2 by the synchronous control system 1-1 is set to be earlier than the timing of the trigger signal sent to the supercontinuum light source 1-2 by a sufficient time period (30 ms is a typical requirement value for the current rolling shutter technology digital camera; 1ms is sufficient for the current global shutter technology digital camera). Thus, based on the trigger signal output to the bayer color digital camera 20-2 by the synchronous control system 1-1, the electronic shutter system of the bayer color digital camera 20-2 controls the bayer array color image sensor 22-2 on the image plane to start to operate and expose for a sufficient time (which needs to be longer than the trigger advance time plus the pulse train time), so as to realize the complete acquisition of a single frame color image formed by superimposing three transient scenes (three primary color pulses sequentially irradiate the imaging target 17 and are overlapped and imaged on the bayer array color image sensor 22-2 in the camera shutter exposure process). It should be noted that, if the exposure time of the camera is too long, the adverse effect of the ambient light on the imaging will be aggravated, resulting in too high brightness of the static background in the imaging frame, and significantly reducing the transient image contrast.

The color digital camera of the embodiment realizes the spatial separation acquisition of the three primary colors of RGB on the same bayer array color image sensor 22-2 by the bayer color filter light splitting technology. The color imaging technical scheme which is most widely applied to the current color digital camera has lower manufacturing cost than other color imaging technical schemes (such as a multi-sensor scheme of a prism type light splitting technology), and meanwhile, the compact camera space design can be realized, and the space size of an optical imaging system based on the color digital camera is favorably reduced. Therefore, even though there is a certain disadvantage in color reduction and the like in practical applications compared with other multi-sheet (layer) sensor technologies, the bayer-type color digital camera still occupies the absolute dominance of the market. For the multi-frame ultrafast imaging system based on color digital camera of the present invention, the advantages and disadvantages of the color digital camera used will be inherited by the system. Although bayer-type color digital cameras can cause interpolation color noise (ultrafast imaging frame gray scale noise), higher color crosstalk (ultrafast imaging frame-to-frame crosstalk), lower light energy utilization (ultrashort pulse illumination efficiency), and additional color channel (ultrafast imaging frame) separation processing procedures, the obvious manufacturing cost advantages, high technical maturity and continuity, and significant spatial representation advantages thereof can greatly offset the above technical disadvantages, and replace color digital cameras of multi-sensor technology in multi-frame ultrafast imaging applications requiring high economy and compact spatial representation.

S8: the gray level image 33 of each color channel of the image shot by the color digital camera is obtained through the color channel separation processing 28 of the RGB image output by the Bayer digital camera 20-2, a plurality of ultrafast imaging frames 35 which correspond to the instant scene of the ultrashort pulse illumination of each primary color and are arranged according to time sequence are obtained, and multi-frame ultrafast imaging is realized.

After the bayer array color image sensor 22-2 has acquired the ultrafast imaging single frame color image, the bayer color digital camera 20-2 may output the acquired image information in two ways:

1) an image information processing module 23(ISP) in the camera performs interpolation 29-1 on a RAW RGB format image 31 collected by the bayer array color image sensor 22-2, and then outputs 26 an RGB color image 27 obtained by superimposing instantaneous scenes of the frame, which are sequentially irradiated by three-primary-color ultrashort pulses onto the imaging target 17 (the RGB color space color image has an RGB format pixel form, each pixel of which includes values of three primary color components of RGB, and is output in a specific picture storage format, such as TIFF, PNG, BMP, and the like).

2) The camera directly outputs 30 single-frame RAW RGB format images 31(RAW RGB format images are original data of a color image sensor converting collected optical signals into digital signals, and each pixel of the RAW RGB format images only has data of one color; for a bayer array color image sensor, when the RAW RGB format is arranged in a bayer array, the RAW RGB format is also called a bayer RGB format, which is data obtained directly by transmitting light through a bayer color filter array, and is the most common one of the RAW RGB formats).

Next, each color channel separation processing 28 is performed on the single frame image output by the bayer color digital camera 20-2 to obtain three primary color ultrafast imaging frames 33 corresponding to the three primary color ultrashort pulse illumination transient scenes, and then each ultrafast imaging frame is arranged 34 according to the sequence of the occurrence of the events to obtain a sequence of ultrafast imaging frames 35 arranged according to the time sequence, so as to present an ultrafast transient evolution process of a single trigger event. Specifically, in an ultrafast image frame captured by the bayer-type color digital camera 20-2, transient scenes in which three primary color pulses respectively irradiate the imaging target 17 are superimposed in the same color image frame. That is, although the bayer-type color digital camera realizes physical spatial separation of three color channels of RGB, it does not realize information spatial separation of three color channels of RGB. Therefore, to actually realize the storage and presentation of a plurality of ultrafast imaging frames, the image single frame captured by the bayer color digital camera 20-2 needs to be subjected to the color channel separation processing 28 to obtain the ultrafast imaging frame 33 formed by the individual irradiation of the imaging target 17 with the primary color pulses. As described above, the bayer color digital camera 20-2 may output captured image information in two ways in general. For the two output modes, the image processing processes for subsequently acquiring the ultrafast imaging frames of the primary colors and arranging the ultrafast imaging frames according to the time sequence are also different.

1) For the case that the camera outputs 26 the RGB color image 27, by performing digital image information processing on the single frame RGB color image 27, that is, performing separation processing 28 on RGB color channels thereof, ultrafast imaging frames 33 corresponding to the instant scenes where ultrashort pulses of each primary color irradiate the imaging target 17 can be obtained, and then based on the time delay relationship of the active exposure process of ultrashort pulses of each primary color, the ultrafast imaging frames are arranged 34 according to the sequence of events.

2) In the case of the camera outputting 30RAW RGB format images 31, the separation process for the image RGB color channels can be achieved in two ways. Firstly, RGB color channel separation processing 28 is carried out on an RAW RGB format image 31 to obtain an RAW format image 32 of each color channel, and then interpolation processing 29-3 is carried out on the RAW format image 32 of each color channel to obtain each ultrafast imaging frame 33; secondly, firstly, the RAW RGB format image 31 is interpolated 29-2 to obtain an RGB color image 27, and then the RGB color channel separation processing 28 is performed to obtain each ultrafast imaging frame 33. By the above two modes, the three primary color ultrafast imaging frames 33 superimposed in the RAW RGB format image 31 can be restored. Then, based on the time delay relationship of the active exposure process of the ultrashort pulse of each primary color, the ultrafast imaging frames are arranged 34 according to the sequence of the occurrence of the events.

In this embodiment, the ultrafast imaging frame sequence 35 arranged in time sequence can be obtained through both the above two processing procedures, so as to clearly present multiple ultrafast transient evolution moments of a single trigger event with a macroscopic size, that is, to implement multi-frame ultrafast imaging of the transient process.

In addition, when the color digital camera used is not based on the bayer color filter array technology but on other color filter array technologies, the above-described implementation of the multi-frame ultrafast imaging method based on the bayer color digital camera is still applicable. For example, a color digital camera based on the following novel color filter array technology can realize multi-frame ultrafast imaging by the scheme of the embodiment: X-Trans arrays (Moire reduction) that change the arrangement of the three primary color filters; the RGBE array is added with a cyan waveband color filter, and the RGBW array is added with a white waveband color filter; RGB-IR incorporating infrared band filters (noise reduction in low light conditions); far beyond the three primary colors multispectral color filter array (MSFA).

In the above embodiments 1 and 2, in addition to performing image information processing on a software layer, editing and storing according to a time sequence, and subsequent video playing or continuous image frame presentation on multiple frames of ultrafast transient images captured and output by an ultrafast imaging system, a color digital camera having an RGB color component video signal output interface (such as one equipped with a D-Sub or VGA video transmission interface) may also directly output three primary color image frames to three monitors, respectively, and implement real-time synchronous display and storage recording of all imaging frames in multiframe ultrafast imaging in a pure hardware manner. Furthermore, for a color digital camera with any color video transmission interface, as long as the output color video signals of the color digital camera are properly converted to obtain each primary color component signal in an RGB color space, real-time synchronous display and storage recording of all imaging frames in multi-frame ultrafast imaging can be realized.

The above embodiments are all directed to the case where the pixel colors of a color digital camera are based on a mixture of three color channels (three primary colors) of red, green and blue, such as a color camera based on bayer color filter array technology or prism 3 sensor technology. The three-primary-color scheme directly corresponds to the three-primary-color principle in colorimetry, has perfect theory and mature technology, has the advantages of compact structure and low cost, and is the mainstream technical scheme of the commercial color digital camera at present. However, the application scenario of the ultrafast imaging method proposed by the method of the present invention is not limited to the RGB three-primary-color imaging, i.e. the method has general technical applicability in terms of the number of color channels imaged by the color digital camera. Typically, for some professional imaging application fields, the spectral imaging capability of a digital camera based on traditional standard RGB three primary colors is far from the technical requirement of performing professional spectrum acquisition, so that a novel camera with important application value in the multispectral and hyperspectral imaging fields, such as a multispectral camera and a hyperspectral camera, is developed. For example, by extending the bayer color filter array of the image sensor 22-2 of embodiment 2 to a multi-spectral color filter array (MSFA), the color digital camera 20-2 can obtain multi-spectral or even hyper-spectral images in one shot (referred to as snapshot mosaic imaging). For the situation of multispectral imaging, the method is not only applicable, but also can obviously improve the frame number of multi-frame ultrafast imaging: the situation of three primary colors can be similar by only generating a common-beam ultrashort pulse string which is matched with each color channel of the multispectral filter array and has specific time delay, irradiating the imaging target 17 by using the ultrashort pulse string, acquiring a frame of color image formed by the pulse string actively exposing the imaging target 17 through imaging by a standard imaging system based on a multispectral camera, and finally separating 28 each color channel of the shot color image to obtain a plurality of ultrafast imaging frames 35 which are arranged according to time sequence and correspond to each color channel lighting instant scene. In short, the method can be directly popularized to the field of multispectral or hyperspectral imaging with the number of color channels far higher than three primary colors, and multi-frame ultrafast imaging based on a multispectral or hyperspectral camera is realized.

In addition, the light sources of the above embodiments are all based on the super-continuum spectrum ultrashort pulse, and have the characteristics of relatively significant time chirp and relatively low time and spatial coherence, which is beneficial to reducing the speckle effect in the ultrashort pulse illumination imaging process and improving the spatial resolution and imaging quality of ultrafast imaging. It is worth noting that such ultra-short pulse with low coherence width spectrum is difficult to obtain the pulse width at the limit of time-bandwidth conversion, so that the ultra-fast imaging system cannot obtain the limit time resolution corresponding to the spectrum width. The multiframe ultrafast imaging method is not limited to the situation that the low-coherence wide-spectrum ultrashort pulse is used as a light source, namely, the high-coherence wide-spectrum ultrashort-pulse light source is also suitable for the multiframe ultrafast imaging method. Generally, a high-coherence ultrashort pulse light source can obtain a pulse width close to the transformation limit, and thus the time resolution limit of the system can be significantly improved. Under the condition of the high-coherence ultrashort pulse light source, the limit time resolution of the ultrafast imaging system can be better than 10fs (considering the spectral envelope of Gaussian distribution) by fully utilizing the spectral width of each color channel of RGB (red, green and blue) three primary colors which is close to 100 nm. However, under the illumination condition of the high-coherence light, the imaging process can generate relatively significant optical speckle effect, which results in the reduction of the imaging spatial resolution and quality. In addition, for high-coherence broad-spectrum ultrashort pulses, it is difficult to maintain the pulse width near the conversion limit for long distance transmission, because the dispersion of the transmission medium causes pulse broadening, and the nonlinear characteristics of the transmission medium cause the pulse to vary in characteristics such as time, space and spectrum, especially when the pulse is transmitted through an optical fiber (a large-mode-field hollow fiber or a photonic crystal fiber is beneficial to reducing these effects). Therefore, under such conditions, the optical path setup requirements of the ultrafast imaging system may increase significantly. In short, what kind of coherent wide-spectrum ultrashort pulse is selected to be used as the illumination light source of the multi-frame ultrafast imaging system, the requirements of actual detection on time and spatial resolution should be comprehensively considered, and two factors of space-time resolution and imaging quality should be balanced, which cannot be considered.

Although the ultrashort pulse width of the above embodiment is set at the time scale of 100fs to 10ps, the imaging time resolution of the obtained ultrafast imaging system is also in the time scale, and the range of the ultrafast imaging system time resolution involved in the multi-frame ultrafast imaging method is not limited to this range. As in the case of the 100nm spectral wide high coherence pulse discussed above, the limit time resolution of an ultrafast imaging system may be less than 10 fs. In fact, fs to ns, even μ s or ms time scale light pulses can be applied to the multi-frame ultrafast imaging system of the present invention, to obtain the ultrafast imaging system time resolution of the corresponding time scale. That is, the ultra-fast imaging method based on the color digital camera has wide applicability to various illumination pulses and is not limited by the pulse width characteristic.

In addition, the color digital camera in the present invention refers to a color camera that performs imaging recording based on the digital image sensor 22, that is, the "digital" in the color digital camera refers to that the image acquisition process of the camera is completed by the imaging of the digital image sensor 22, and does not refer to the image or video signal output mode of the camera — the output signal of the color digital camera in the present invention may be a digital signal or an analog signal.

It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An ultrafast imaging method based on a color digital camera is characterized in that: the method comprises the following steps:

s7: controlling the shutter of the color digital camera to expose for a certain time at a time to realize the shooting of the instantaneous scene of the ultrashort pulse train illumination imaged on the image sensor;

2. The ultrafast imaging method based on color digital camera of claim 1, wherein: step S1, a synchronous control system is adopted to send out a trigger signal to control the supercontinuum light source to output a single supercontinuum ultrashort pulse with a wavelength range covering all color channels of the color digital camera, and a trigger signal with a certain time lead is sent to the color digital camera to control the opening and the subsequent closing of a camera shutter.

3. The ultrafast imaging method based on color digital camera of claim 2, wherein: step S2, splitting the super-continuous spectrum ultra-short pulse by using a beam splitting dichroic mirror group to obtain each path of base color ultra-short pulse corresponding to each color channel wave band of the color digital camera; the said each way of base color ultrashort pulse carries on the central wavelength to choose and the control of the spectral width through the band-pass filter separately, make the wavelength range in the color digital camera each color channel wavelength range separately, realize the regulation and control to the ultrashort pulse time width of each way of base color while reducing the color crosstalk among each way of base colors; the energy of each path of primary color ultrashort pulse is controlled through a continuously adjustable neutral filter to obtain a certain energy proportion of each primary color pulse, so that an imaging target irradiated by a subsequent multi-primary color pulse string has an approximate white balance effect when the imaging target is imaged on a color digital camera.

4. The ultrafast imaging method based on color digital camera of claim 3, wherein: and step S3, controlling the delay time between pulses of each path of primary color ultrashort pulse by adopting a delay line group or a delay optical fiber group.

5. The ultrafast imaging method based on color digital camera of claim 4, wherein: and step S4, combining the primary color ultrashort pulses by using a beam-combining dichroic mirror group or an optical fiber beam combiner.

6. The ultrafast imaging method based on color digital camera of claim 5, wherein: and step S5, irradiating the imaging target by adopting a direct illumination or Kohler illumination mode, and realizing the active exposure time control of the multi-primary-color ultrashort pulse train of the imaging target.

7. The ultrafast imaging method based on color digital camera of claim 6, wherein: and step S6, imaging the imaging target irradiated by the multi-primary-color ultrashort pulse train on the color digital camera image sensor by adopting an optical microscope imaging optical path or a camera lens imaging optical path.

8. The ultrafast imaging method based on color digital camera of claim 7, wherein: and step S7, controlling the shutter of the color digital camera to expose for a period of time which can completely cover the whole course of the ultrashort pulse string illumination based on the trigger signal which is sent to the color digital camera by the synchronous control system and has a certain time lead, and realizing the shooting of the ultrashort pulse string illumination instant scene imaged on the image sensor.

9. The ultrafast imaging method based on color digital camera of claim 8, wherein: and step S8, acquiring gray level images of each color channel of images shot by the color digital camera through direct color channel separation output in the color digital camera or external subsequent color channel separation processing.

10. The ultrafast imaging method based on color digital camera of claim 9, wherein: the color digital camera types comprise a multispectral camera and a hyperspectral camera, namely the color channel number range of the color digital camera is the case of three primary colors or higher than the case of three primary colors; the color channel wavelength range of the color digital camera is from a visible light wave band, or from a visible light wave band to an infrared wave band, or from an ultraviolet wave band to a visible light wave band, or from an ultraviolet wave band to an infrared wave band.