CN115308174A - Plankton in-situ imaging system and method applied to open scene - Google Patents
Plankton in-situ imaging system and method applied to open scene Download PDFInfo
- Publication number
- CN115308174A CN115308174A CN202210852676.8A CN202210852676A CN115308174A CN 115308174 A CN115308174 A CN 115308174A CN 202210852676 A CN202210852676 A CN 202210852676A CN 115308174 A CN115308174 A CN 115308174A
- Authority
- CN
- China
- Prior art keywords
- fluorescence
- image
- plankton
- imaging
- telecentric
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
Landscapes
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
The invention discloses a plankton in-situ imaging system and a plankton in an open scene, wherein the system comprises: the system comprises a telecentric light source, a dichroic spectroscope, a back-shadow imaging module, a fluorescence imaging module, a control module and an image processing module; the telecentric light source is incident to a plankton plane in the water body, a plankton target containing chlorophyll emits a longer-wavelength fluorescent component under the excitation of light emitted by the telecentric light source, and the longer-wavelength fluorescent component and the illumination light are mixed together, and the two are separated by the dichroic spectroscope; the back imaging module performs back imaging on the separated illuminating light, and the fluorescence imaging module performs fluorescence imaging on the separated fluorescence; the control module controls the back-shadow imaging module and the fluorescence imaging module to trigger simultaneously, and the same frequency sampling of the back-shadow image and the fluorescence image in the same visual field is realized. The invention can realize effective observation of the micro plankton.
Description
Technical Field
The invention relates to the technical field of plankton monitoring, in particular to a plankton in-situ imaging system and method applied to an open scene.
Background
The plankton is used as an important component of the marine ecosystem, and accurate monitoring can provide evaluation basis for the health degree of the marine ecosystem. The imaging-based plankton observation mode can provide the morphological characteristics of the target in a non-invasive mode in a high-resolution mode, and with the development of imaging devices and deep learning, high-frequency acquisition of high-resolution images and automatic classification and identification lay the foundation for real-time monitoring of plankton. But the existing open type imaging platform can not realize effective observation.
Disclosure of Invention
The invention aims to solve the technical problem of effectively observing micro plankton and provides a plankton in-situ imaging system and method applied to an open scene.
The technical problem of the invention is solved by the following technical scheme:
the invention discloses a plankton in-situ imaging system applied to an open scene, which comprises: the system comprises a telecentric light source, a dichroic spectroscope, a back-shadow imaging module, a fluorescence imaging module, a control module and an image processing module; the telecentric light source is incident to a plankton plane in the water body, the plankton target containing chlorophyll emits a fluorescent component with longer wavelength under the excitation of light emitted by the telecentric light source, and is mixed with illumination light, and the two are separated by the dichroic spectroscope; the back imaging module performs back imaging on the separated illuminating light, and the fluorescence imaging module performs fluorescence imaging on the separated fluorescence; the control module controls the back-shadow imaging module and the fluorescence imaging module to trigger simultaneously, and the same-frequency sampling of the back-light image and the fluorescence image in the same visual field is realized.
In some embodiments, the image processing module is further configured to perform motion blur discrimination and processing on the acquired fluorescence image, and then fuse the processed fluorescence image with the background image to obtain a fused image.
In some embodiments, the back imaging module and the fluorescence imaging module comprise telecentric lenses, and form a double telecentric system in cooperation with telecentric illumination.
In some embodiments, the back imaging module includes a first telecentric lens and a CMOS camera, and the back imaging module uses the first telecentric lens to receive the illumination light reflected by the dichroic beamsplitter for back imaging on the CMOS camera.
In some embodiments, the fluorescence imaging module includes an optical filter, a second telecentric lens and an sCMOS camera, and the fluorescence imaging module filters wavelength components other than fluorescence by using the optical filter, and performs fluorescence imaging on the sCMOS camera through the second telecentric lens.
In some embodiments, a scene classification data set is established through an image processing module, a scene classification model is obtained by training the scene classification data set based on a lightweight network, whether motion blur exists in a fluorescence image is judged in an end-to-end mode, a motion blur kernel is solved for deconvolution to eliminate the motion blur for the fluorescence image with the motion blur, a weighted average method is used for carrying out pixel-level fusion on the fluorescence image and a background image in a spatial domain, and a final fusion image containing chlorophyll fluorescence information and sharp edges is obtained.
In some embodiments, the size of the illumination of the telecentric light source, the working distance of the telecentric light source and the working distance of the telecentric lens are determined by establishing the fluorescence illumination model on the premise that the illumination of the image plane of the fluorescence signal received by the fluorescence imaging module can be ensured under the condition of a certain chlorophyll concentration.
In some embodiments, the process of modeling fluorescence illumination comprises: the propagation of the excitation light to the target, the fluorescence of chlorophyll and the reception of the fluorescence emission by the fluorescence imaging module.
In some embodiments, further comprising a capsule, the shadowgraph imaging module and the fluorescence imaging module being located within the capsule; the dichroic spectroscope is arranged at the inlet of the sealed cabin and forms an angle of 45 degrees with the incident illumination light and the fluorescence.
In some embodiments, the telecentric light source is a 460nm telecentric light source which is incident on the plankton planes in the water body with uniform illumination through the sealed chamber.
The invention also discloses a plankton in-situ imaging method applied to open scenes, which comprises the following steps:
s1, a telecentric light source is transmitted into a plankton plane in a water body through a sealed cabin at uniform illumination, and a plankton target containing chlorophyll emits a longer-wavelength fluorescent component under the excitation of light of the telecentric light source; the illuminating light is transmitted and scattered by plankton and is mixed with the excited fluorescence;
s2, separating the illumination light and the fluorescence which are mixed together;
s3, carrying out back-shadow imaging on the separated illuminating light, and simultaneously carrying out fluorescence imaging on the separated fluorescence;
and S4, carrying out same-frequency sampling on the backlight image and the fluorescence image of the same visual field.
In some embodiments, further comprising the step of: and S5, carrying out motion blur discrimination and processing on the fluorescence image, and then fusing the fluorescence image with the shadow image to obtain a fused image.
In some embodiments, step S5 includes:
s51, establishing a scene classification data set, comprising: clear images, out-of-focus blurred images, motion blurred images;
s52, training by using the scene classification data set based on a lightweight network to obtain a scene classification model, and judging whether the fluorescent image has motion blur in an end-to-end mode;
s53, if the fluorescence image has no motion blur, directly entering the step S44; if the fluorescence image has motion blur, solving a motion blur kernel to perform deconvolution to eliminate the motion blur;
and S54, performing pixel-level fusion on the fluorescence image and the back image in a spatial domain by using a weighted average method to obtain a final fusion image containing chlorophyll fluorescence information and sharp edges.
In some embodiments, step S54 specifically includes: carrying out weighted average on each pixel of the fluorescence image M and the background image N, and carrying out weighted average on the values of the pixel points of the fused image G at the corresponding positions of M and N to obtain the value of the pixel point of the fused image G, namely:
G(i,j)＝w 1 M(i,j)+w 2 N(i,j)
wherein G (i, j) is the pixel value at the position of the fused image (i, j), M (i, j) is the pixel value at the position of the fluorescent image (i, j), and N (i, j) is the pixel value at the position of the background image (i, j); w is a 1 And w 2 Is a weighting coefficient, and satisfies w 1 +w 2 ＝1；
And converting the obtained original back gray image into an RGB three-channel color image, and performing weighted fusion on the fluorescence image pixel value and the back image in an R (red) channel to enable the chlorophyll-containing part to be red.
Compared with the prior art, the invention has the advantages that:
the plankton in-situ imaging system applied in the open scene is based on the microscopic back imaging light path, and chlorophyll fluorescence imaging is carried out on the target in the visual field at the same time, so that back imaging and fluorescence imaging are carried out at the same time. The back shadow imaging mode can provide clear morphological characteristics and high-contrast edges; chlorophyll fluorescence imaging can supplement classification information for objects with insufficient morphological information, and biological and non-biological components in the particles can be distinguished.
In some embodiments, the fluorescence image is fused with the back image after motion blur discrimination and processing, the obtained fused image increases information dimension, chlorophyll information can be supplemented on the basis of morphological information, and therefore target detection performance and extraction of biological components of background particles are improved.
Drawings
FIG. 1 is a schematic structural diagram of a plankton in-situ imaging system according to an embodiment of the present invention.
FIG. 2 is a flow chart of the plankton in-situ imaging system according to the embodiment of the present invention.
FIG. 3a is a fluorescence excitation spectrum of Chlorella in the example of the present invention.
FIG. 3b is a fluorescence excitation spectrum of chlorella in the example of the present invention.
FIG. 4 is a diagram of an image processing module according to an embodiment of the present invention.
FIG. 5 is a fluorescent image acquired in an embodiment of the present invention with motion blur.
Fig. 6 is a back image acquired in an embodiment of the present invention.
FIG. 7 is a fluorescence image after motion blur is removed in an embodiment of the present invention.
Fig. 8 is a fused image obtained in the embodiment of the present invention.
FIG. 9 is a flow chart of the plankton in-situ imaging method according to the embodiment of the invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and preferred embodiments. It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict.
It should be noted that the terms of orientation such as left, right, up, down, top and bottom in the present embodiment are only relative concepts to each other or are referred to the normal use state of the product, and should not be considered as limiting.
The inventor finds that the detection method in the prior art is lack of information of micro plankton (5-50 micrometers) with smaller size and biological components of background particles, so that effective observation cannot be realized, and therefore, the embodiment of the invention provides a plankton in-situ imaging system applied to an open scene, and chlorophyll fluorescence imaging is performed on plankton targets in a visual field simultaneously based on a double telecentric microscopic back imaging light path, so that back imaging and fluorescence imaging are performed simultaneously. The mode of back shadow imaging can provide clear morphological characteristics and high-contrast edges; chlorophyll fluorescence imaging can be used as supplement of classification information for a target with insufficient morphological information, and biological and non-biological components in the particles are distinguished. The fluorescence image is fused with the shadow image after being subjected to motion blur discrimination and processing, the obtained fusion image increases information dimensionality, and chlorophyll information can be supplemented on the basis of morphological information, so that the target detection performance is improved, and the extraction of biological components of background particles is facilitated.
As shown in fig. 1, the plankton in-situ imaging system applied in an open scene according to the embodiment of the present invention includes: the system comprises a power supply, a telecentric light source 2, a dichroic spectroscope 5, a back imaging module 6, a fluorescence imaging module 7, a control module 8 and an image processing module 9, wherein the back imaging module 6 and the fluorescence imaging module 7 are positioned in a sealed cabin 4. The back imaging module 6 comprises a first telecentric lens 10 and a CMOS camera 11, the first telecentric lens 10 is used by the back imaging module 6 to receive the irradiation light reflected by the dichroic spectroscope 5, and back imaging is carried out on the CMOS camera 11; the fluorescence imaging module 7 comprises an optical filter 12, a second telecentric lens 13 and an sCMOS camera 14, and the fluorescence imaging module 7 filters out wavelength components except fluorescence by adopting the optical filter 12 and performs fluorescence imaging on the sCMOS camera 14 through the second telecentric lens 13. The power supply 1 is used for supplying power to the telecentric light source 1, the CMOS camera 11, the sCMOS camera 14 and the control module 8, and the control module 8 is respectively connected with the CMOS camera 11, the sCMOS camera 14 and the image processing module 9.
As shown in fig. 2, a telecentric light source 2 is incident to a plankton plane in a water body, a plankton target 3 containing chlorophyll emits a longer-wavelength fluorescent component under the excitation of light (blue light) emitted by the telecentric light source 2, and is mixed with illumination light, and a dichroic spectroscope separates the two components; the back imaging module 6 performs back imaging on the separated illuminating light, and the fluorescence imaging module 7 performs fluorescence imaging on the separated fluorescence; the control module 8 controls the back imaging module 6 and the fluorescence imaging module 7 to trigger simultaneously, so that the backlight image and the fluorescence image in the same field of view are sampled at the same frequency.
The imaging principle of the present embodiment is as follows:
the back imaging light path uses an illumination light source to emit light from the back of the target to uniform the field of view, and the light is received by a camera on the other side, so that clear edges and high-contrast contours can be obtained. As most species in plankton have higher transparency, the pattern of the shadow imaging is matched with a double telecentric system of telecentric illumination and telecentric lens, so that the size, the edge information and other morphological characteristics of the plankton can be accurately obtained.
Fluorescence is a phenomenon of photoluminescence, in which when a fluorescent molecule is excited, extra-nuclear electrons absorb energy and transit from a ground state to an excited state, electrons in the excited state, due to high and unstable energy, transit back to the ground state by radiation and release energy in the form of photons, and emit outgoing light (generally, wavelength in a visible light band) longer than the wavelength of incident light, that is, fluorescence. Chlorophyll is commonly present in plankton, particularly in phytoplankton such as algae, and chlorophyll fluorescence imaging can acquire distribution information of chlorophyll contained in the plankton.
Chlorella common in coastal waters was selected as a representative microalgae for experiments. As shown in fig. 3a to 3b, according to the fluorescence excitation spectrum of chlorella measured by the microplate reader, it can be seen that chlorella has higher light energy absorption efficiency in the blue light band, so that chlorophyll molecules can be in a higher excited state, and a higher fluorescence intensity value can be obtained; and under the condition of fixing the wavelength of the excitation light, measuring the emission spectrum of the chlorella by using an enzyme-labeling instrument, wherein the wavelength corresponding to the peak of an emission spectrum curve is about 680 nm. Therefore, in the fluorescence imaging module, a 460nm LED telecentric light source is selected as an illumination light source and is simultaneously used as a high-efficiency fluorescence excitation light source, a long-wave pass filter is placed in front of a fluorescence receiving end to filter illumination light with a short wavelength of 460nm, and fluorescence with a long-wave band of about 680nm is reserved.
Specifically, in the imaging system of the present embodiment, the 460nm telecentric light source 2 is incident on the plankton plane in the water body through the sealed cabin 4 with uniform illumination, and the plankton target 3 containing chlorophyll emits fluorescence components with longer wavelength under excitation of excitation light (blue light). Illumination light (shown by a solid arrow in fig. 1) emitted by the telecentric light source 2 is transmitted and scattered by plankton and mixed with excited longer wavelength fluorescence (shown by a dashed arrow in fig. 1), and in order to separate the two parts, a dichroic beam splitter 5 is used at the entrance of the sealed cabin 4, which forms a 45-degree angle with the incident illumination light and fluorescence, and the fluorescence is directly transmitted and filtered by a filter 12 (high cut-off depth filter) to remove stray light, while the illumination light is reflected and received by a CMOS (Complementary Metal Oxide Semiconductor) camera 11 for back imaging. The fluorescence imaging module 7 filters out wavelength components except fluorescence by adopting an optical filter 12 (a high cut-off depth optical filter), and performs fluorescence imaging on a scientific research grade CMOS (sCMOS) camera with high sensitivity through a second telecentric lens 13 with 4 times of magnification; and the back imaging module 6 uses a first telecentric lens 10 with 4 times magnification of uniform specification to receive the illumination light reflected by the dichroic spectroscope 5, and performs back imaging on the CMOS camera 11. The control module 8 controls the CMOS camera 11 and the sCMOS camera 14 to trigger simultaneously, so as to realize the same frequency sampling of the backlight image and the fluorescence image in the same field of view. And transmits the collected back image and fluorescence image to the image processing module 9 for processing.
Further, in this embodiment, the fluorescence illumination model is established to determine the illumination intensity of the selected telecentric light source, the telecentric light source and the working distance of the telecentric lens on the premise that the illumination intensity of the image plane of the fluorescence signal received by the sCMOS camera 14 can be ensured under the condition of a certain chlorophyll concentration. The process of establishing the fluorescence illumination model comprises the following steps: the propagation of the excitation light to the target, the fluorescence of chlorophyll and the reception of the fluorescence emission by the camera. The method comprises the following specific steps:
seawater has strong absorption and scattering effects on light propagation, and for monochromatic parallel rays, the illumination propagated in water can be described by exponential decay:
E(λ，r)＝＝E 0 e -α(λ)r
e (lambda, r) is the illumination intensity of the monochromatic light source with the wavelength lambda transmitted to the position with the distance r from the reference point; e 0 Is the illuminance of the initial reference point, which is a constant; e is a natural constant, α (λ) is an attenuation coefficient, and is related to the wavelength of light, and can be divided into two parts, namely an absorption coefficient and a scattering coefficient:
α(λ)＝a(λ)+b(λ)
a (λ) is an absorption coefficient, and b (λ) is a scattering coefficient.
Thus, for an excitation light process, a wavelength λ can be approximated 0 Initial illuminance of E light source To a distance r from the light source 1 At the position of the target object, the illumination intensity E 1 Comprises the following steps:
a(λ 0 )、b(λ 0 ) Respectively, under water to a wavelength of lambda 0 The absorption coefficient and the scattering coefficient of light.
For chlorophyll fluorescence process, an efficiency factor k is used to describe the efficiency of absorption of excitation light and emission of fluorescence, which is related to the wavelength of the excitation light; in addition, the level of illumination E of the fluorescence emitted by the object plane emission (λ 0 ,r 1 ) Directly correlated with chlorophyll concentration c:
E emission (λ 0 ，r 1 )＝κ(λ 0 )cE 1 (λ 0 ，r 1 )
for the process of receiving the fluorescence emission light by the camera, the distance r from the target object 2 The camera plane received illumination is:
wherein λ e The wavelength of the fluorescence emission.
The collected fluorescence image and the acquired back image are transmitted to the image processing module 9 for image processing, and the image processing flow is shown in fig. 4. Due to the fact that the illumination of chlorophyll fluorescence is low, the exposure time of a fluorescence receiving end is long, and when the moving speed of a moving target in a water body is high, the image can generate a motion blurring phenomenon. In order to realize the determination of the target motion blur, a scene classification data set is established in the embodiment, and comprises three types of images, namely a clear image, a defocused blurred image and a motion blurred image, the scene classification data set is used for training based on a lightweight network (ShuffleNet) to obtain a scene classification model, whether the motion blur exists in the fluorescent image can be determined in an end-to-end mode, and the image determined as the motion blur is subjected to image processing to eliminate the blur. In the exposure time, the camera end does not move, but the movement of plankton can be approximately seen as uniform-speed linear movement, so that a method for eliminating linear movement blur is adopted, and a movement blur kernel is solved to perform deconvolution to eliminate the movement blur. And then, carrying out image fusion operation on the fluorescence image and the background image, and carrying out pixel-level fusion on the fluorescence image and the background image in a spatial domain by using a weighted average method to obtain a fused image finally containing chlorophyll fluorescence information and sharp edges, thereby laying a foundation for image-based target detection of plankton.
The weighted average fusion is to perform weighted average on each pixel of the original image, taking two images, i.e. the fluorescence image M and the background image N as an example, the value of the pixel point of the fused image G is obtained by performing weighted average on the values of the pixel points at the corresponding positions of the fluorescence image M and the background image N, that is:
G(i,j)＝w i M(i,j)+w 2 N(i,j)
wherein G (i, j) is the pixel value at the position of the fused image (i, j), M (i, j) is the pixel value at the position of the fluorescence image (i, j), and N (i, j) is the pixel value at the position of the background image (i, j); w is a 1 And w 2 Is a weighting coefficient, and satisfies w 1 +w 2 ＝1。
In this embodiment, the obtained original back gray image is converted into an RGB three-channel color image, and in an R, i.e., red channel, the fluorescence image pixel value and the back image are subjected to weighted fusion, so that the chlorophyll-containing portion 15 appears red. As shown in fig. 5 to 8, the fluorescence image with motion blur, the background image, the fluorescence image without motion blur, and the obtained fusion image are collected in this embodiment.
As shown in fig. 9, an embodiment of the present invention further provides a plankton in-situ imaging method applied in an open scene, including the following steps:
s1, a telecentric light source is transmitted into a plankton plane in a water body through a sealed cabin at uniform illumination, and a plankton target containing chlorophyll emits a longer-wavelength fluorescent component under the excitation of light of the telecentric light source; the illuminating light is transmitted and scattered by plankton and is mixed with the excited fluorescence.
And S2, separating the illumination light and the fluorescence which are mixed together.
And S3, carrying out back shadow imaging on the separated illuminating light, and simultaneously carrying out fluorescence imaging on the separated fluorescence.
And S4, carrying out same-frequency sampling on the backlight image and the fluorescence image of the same visual field.
In some embodiments, further comprising the step of: and S5, carrying out motion blur discrimination and processing on the fluorescence image, and then fusing the fluorescence image with the back image to obtain a fused image.
Specifically, step S5 includes:
s51, establishing a scene classification data set, comprising: sharp images, out-of-focus blurred images, motion blurred images.
And S52, training by using the scene classification data set based on a lightweight network to obtain a scene classification model, and judging whether the fluorescence image has motion blur in an end-to-end mode.
S53, if the fluorescence image has no motion blur, directly entering the step S44; and if the fluorescence image has motion blur, solving a motion blur kernel to perform deconvolution to eliminate the motion blur.
And S54, performing pixel-level fusion on the fluorescence image and the back image in a spatial domain by using a weighted average method to obtain a final fusion image containing chlorophyll fluorescence information and sharp edges.
Further, step S54 specifically includes: carrying out weighted average on each pixel of the fluorescence image M and the background image N, and carrying out weighted average on the values of the pixel points of the fused image G at the corresponding positions of M and N to obtain the value of the pixel point of the fused image G, namely:
G(i,j)＝w 1 M(i,j)+w 2 N(i,j)
wherein G (i, j) is the pixel value at the position of the fused image (i, j), M (i, j) is the pixel value at the position of the fluorescence image (i, j), and N (i, j) is the pixel value at the position of the background image (i, j); w is a 1 And w 2 Is a weighting coefficient, and satisfies w 1 +w 2 ＝1；
And converting the obtained original back gray image into an RGB three-channel color image, and performing weighted fusion on the fluorescence image pixel value and the back image in an R (red) channel to enable the chlorophyll-containing part to be red.
The plankton in-situ imaging system and the plankton in-situ imaging method applied to the open scene, provided by the embodiment of the invention, can be used for in-situ sampling in the offshore area, and the obtained back shadow and fluorescence fused image can solve the problems of biological information discrimination and accurate plankton classification and identification in micro plankton particles, so that the effective observation of the micro plankton is realized.
The foregoing is a further detailed description of the invention in connection with specific preferred embodiments and it is not intended to limit the invention to the specific embodiments described. For those skilled in the art to which the invention pertains, several equivalent substitutions or obvious modifications can be made without departing from the spirit of the invention, and all the properties or uses are considered to be within the scope of the invention.
Claims (14)
1. A plankton in-situ imaging system applied to an open scene is characterized by comprising: the system comprises a telecentric light source, a dichroic spectroscope, a back-shadow imaging module, a fluorescence imaging module, a control module and an image processing module; the telecentric light source is incident to a plankton plane in the water body, the plankton target containing chlorophyll emits a fluorescent component with longer wavelength under the excitation of light emitted by the telecentric light source, and is mixed with illumination light, and the two are separated by the dichroic spectroscope; the back imaging module performs back imaging on the separated illuminating light, and the fluorescence imaging module performs fluorescence imaging on the separated fluorescence; the control module controls the back-shadow imaging module and the fluorescence imaging module to trigger simultaneously, and the same-frequency sampling of the back-light image and the fluorescence image in the same visual field is realized.
2. The plankton in-situ imaging system applied to the open scene as claimed in claim 1, wherein the image processing module is further configured to perform motion blur discrimination and processing on the collected fluorescence image, and then fuse the processed fluorescence image with the background image to obtain a fused image.
3. The plankton in-situ imaging system applied to the open scene as claimed in claim 1, wherein the back imaging module and the fluorescence imaging module comprise telecentric lenses to form a double telecentric system in cooperation with telecentric illumination.
4. The plankton in-situ imaging system applied to the open scene as claimed in claim 3, wherein the back imaging module comprises a first telecentric lens and a CMOS camera, and the back imaging module uses the first telecentric lens to receive the irradiation light reflected by the dichroic beam splitter and performs back imaging on the CMOS camera.
5. The plankton in-situ imaging system applied to the open scene as claimed in claim 3, wherein the fluorescence imaging module comprises an optical filter, a second telecentric lens and an sCMOS camera, the fluorescence imaging module filters out wavelength components except for fluorescence by using the optical filter, and fluorescence imaging is performed on the sCMOS camera through the second telecentric lens.
6. The plankton in-situ imaging system applied to an open scene as claimed in claim 1, wherein a scene classification data set is established through an image processing module, a scene classification model is obtained through training by using the scene classification data set based on a lightweight network, whether motion blur exists in a fluorescence image is judged in an end-to-end mode, for the fluorescence image with the motion blur, a motion blur kernel is solved to perform deconvolution to eliminate the motion blur, and a weighted average method is used to perform pixel-level fusion on the fluorescence image and a background image in a spatial domain, so that a fused image which finally contains chlorophyll fluorescence information and sharp edges is obtained.
7. The plankton in-situ imaging system applied to the open scene as claimed in claim 1, wherein the size of the illumination of the telecentric light source, the working distance of the telecentric light source and the working distance of the telecentric lens are determined by establishing a fluorescence illumination model under the condition that the illumination of the image plane of the fluorescence signal received by the fluorescence imaging module can be ensured under a certain chlorophyll concentration condition.
8. The plankton in-situ imaging system applied in open scenes as claimed in claim 6, wherein the process of establishing the fluorescence illumination model comprises: the propagation of the excitation light to the target, the fluorescence of chlorophyll and the reception of the fluorescence emission by the fluorescence imaging module.
9. The plankton in-situ imaging system applied to the open scene as claimed in claim 1, further comprising a sealed cabin, wherein the back imaging module and the fluorescence imaging module are located in the sealed cabin; the dichroic beam splitter is placed at the entrance of the capsule at an angle of 45 degrees to the incident illumination light and fluorescence.
10. The plankton in-situ imaging system applied to the open scene as claimed in claim 1, wherein the telecentric light source is a 460nm telecentric light source, and is incident on plankton planes in the water body with uniform illumination through the sealed cabin.
11. A plankton in-situ imaging method applied to an open scene is characterized by comprising the following steps:
s1, a telecentric light source is transmitted into a plankton plane in a water body through a sealed cabin at uniform illumination, and a plankton target containing chlorophyll emits a longer-wavelength fluorescent component under the excitation of light of the telecentric light source; the illuminating light is transmitted and scattered by plankton and is mixed with the excited fluorescence;
s2, separating the illumination light and the fluorescence which are mixed together;
s3, carrying out back-shadow imaging on the separated illuminating light, and simultaneously carrying out fluorescence imaging on the separated fluorescence;
and S4, performing same-frequency sampling on the backlight image and the fluorescence image of the same visual field.
12. The plankton in-situ imaging method applied to open scenes as claimed in claim 11, further comprising the steps of: and S5, carrying out motion blur discrimination and processing on the fluorescence image, and then fusing the fluorescence image with the shadow image to obtain a fused image.
13. The plankton in-situ imaging method applied to the open scene as claimed in claim 12, wherein the step S5 comprises:
s51, establishing a scene classification data set, comprising: clear images, out-of-focus blurred images, motion blurred images;
s52, training by using the scene classification data set based on a lightweight network to obtain a scene classification model, and judging whether the fluorescent image has motion blur in an end-to-end mode;
s53, if the fluorescence image has no motion blur, directly entering the step S44; if the fluorescence image has motion blur, solving a motion blur kernel to perform deconvolution to eliminate the motion blur;
and S54, performing pixel-level fusion on the fluorescence image and the back image in a spatial domain by using a weighted average method to obtain a final fusion image containing chlorophyll fluorescence information and sharp edges.
14. The plankton in-situ imaging method applied to open scenes as claimed in claim 13, wherein the step S54 specifically comprises: carrying out weighted average on each pixel of the fluorescence image M and the background image N, and carrying out weighted average on the values of the pixel points of the fused image G at the corresponding positions of M and N to obtain the value of the pixel point of the fused image G, namely:
G(i,j)＝w 1 M(i,j)+w 2 N(i,j)
wherein G (i, j) is the pixel value at the position of the fused image (i, j), M (i, j) is the pixel value at the position of the fluorescence image (i, j), and N (i, j) is the pixel value at the position of the background image (i, j); w is a 1 And w 2 Is a weighting coefficient, and satisfies w 1 +w 2 ＝1；
And converting the obtained original back gray image into an RGB three-channel color image, and performing weighted fusion on the fluorescence image pixel value and the back image in an R (red) channel to enable the chlorophyll-containing part to be red.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210852676.8A CN115308174A (en) | 2022-07-20 | 2022-07-20 | Plankton in-situ imaging system and method applied to open scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210852676.8A CN115308174A (en) | 2022-07-20 | 2022-07-20 | Plankton in-situ imaging system and method applied to open scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115308174A true CN115308174A (en) | 2022-11-08 |
Family
ID=83856180
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210852676.8A Pending CN115308174A (en) | 2022-07-20 | 2022-07-20 | Plankton in-situ imaging system and method applied to open scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115308174A (en) |
-
2022
- 2022-07-20 CN CN202210852676.8A patent/CN115308174A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10768402B2 (en) | Microscopy of a tissue sample using structured illumination | |
| US11501544B2 (en) | Deep learning-enabled portable imaging flow cytometer for label-free analysis of water samples | |
| US11125675B2 (en) | Fluid suspended particle classifier | |
| JP2022095825A (en) | Cell sorting using high throughput fluorescence flow cytometer | |
| JP2009537021A (en) | Laser irradiation system in fluorescence microscopy | |
| JP2008539724A (en) | Method and imaging system for determining object properties | |
| CN102156976A (en) | Arithmetically operating device, arithmetically operating method, arithmetically operating program, and microscope | |
| CN104535481B (en) | imaging flow cytometer | |
| US20230028360A1 (en) | Artificial generation of color blood smear image | |
| US20230384223A1 (en) | Method and fluorescence microscope for determining the location of individual fluorescent dye molecules by means of adaptive scanning | |
| EP1839030B1 (en) | Method and device for measuring dynamic parameters of particles | |
| WO2022137243A1 (en) | Optical technique for analyzing insects, shrimp and fish | |
| CN107103604A (en) | A kind of particulate colourity auto-clustering analysis system | |
| CN115032196A (en) | Full-scribing high-flux color pathological imaging analysis instrument and method | |
| US20220254182A1 (en) | Optical technique for analyzing insects, shrimp and fish | |
| CN102590053A (en) | Ocean plankton size-spectra in-situ detection instrument | |
| TW202210794A (en) | Measurement device, and imaging control method | |
| CN115308174A (en) | Plankton in-situ imaging system and method applied to open scene | |
| JP7160047B2 (en) | Biological substance quantification method, image processing device and program | |
| JP2013109205A (en) | Image detection device | |
| US20230085045A1 (en) | Microscope for high-resolution and specific analysis of biological substances, and method of analysis | |
| WO2022270015A1 (en) | Biological specimen observation device and biological specimen observation system | |
| Weilguni et al. | Bladder cancer cell imaging system | |
| JP2022040160A (en) | Method and device for quantitating live organic body, and use of device | |
| Scrimgeour et al. | A computational method for two-dimensional quantitative analysis of standing wave images of red blood cells |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |