WO2019127090A1 - Dispositif et procédé d'imagerie optique de plancton sous-marin - Google Patents

Dispositif et procédé d'imagerie optique de plancton sous-marin Download PDF

Info

Publication number
WO2019127090A1
WO2019127090A1 PCT/CN2017/118958 CN2017118958W WO2019127090A1 WO 2019127090 A1 WO2019127090 A1 WO 2019127090A1 CN 2017118958 W CN2017118958 W CN 2017118958W WO 2019127090 A1 WO2019127090 A1 WO 2019127090A1
Authority
WO
WIPO (PCT)
Prior art keywords
underwater
optical imaging
plankton
optical
illumination
Prior art date
Application number
PCT/CN2017/118958
Other languages
English (en)
Chinese (zh)
Inventor
李剑平
陈涛
章逸舟
刘鹏
陈良培
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2017/118958 priority Critical patent/WO2019127090A1/fr
Publication of WO2019127090A1 publication Critical patent/WO2019127090A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications

Definitions

  • the invention relates to the field of underwater optical instruments, and more particularly to an underwater planktonic optical imaging device and method.
  • Plankton is a fundamental component of marine ecosystems and plays an important role in the material circulation and energy transfer of marine food chains.
  • the physiology, ecology, diversity and process research of plankton are essential to understanding marine resources, biodiversity levels, and climate change impacts on ecosystems.
  • in situ observation technology enables in situ measurement of marine plankton under water, avoiding time-consuming and laborious sample collection and manual analysis steps. Integrating in-situ observation technology and sensors into various ocean observation platforms is of great significance for expanding the information acquisition capability of the observation platform, enhancing the real-time observation, and understanding the geochemical coupling process of marine organisms.
  • underwater optical microscopy imaging devices for direct grain-level digital image recording of plankton is the most intuitive method for marine plankton observation.
  • VPR Video Plankton Recorder
  • IIS In Situ Ichthyoplankton Imaging System
  • UVP Underwater Video Profiler
  • SPC Scripps Plankton Camera
  • LOKI typical types of layered light illumination
  • LOKI Lightframe On-sight Keyspecies Investigation System
  • CPICS Continuous Particle Imaging and Classification System
  • These underwater imaging devices can record digital images of plankton from different sizes from a few ten micrometers to several centimeters, and can identify and quantify plankton species through image recognition processing.
  • the above imaging devices can be roughly classified into two types: silhouette imaging and dark field imaging.
  • VPR and ISIIS systems belong to silhouette imaging, and the acquired image contrast comes from the absorption of light by plankton; with linear array CCD device, silhouette imaging is more suitable for towing organisms with lower abundance of plankton in the way of dragging.
  • Systems such as UVP, SPC, LOKI, and CPICS are all dark field imaging, and the acquired image contrast depends on the scattering and diffraction of light by plankton.
  • dark field imaging can achieve higher resolution, richer plankton morphology, texture and color information, so it can be better classified and identified by plankton.
  • the dark field imaging device adopts area array CCD and grazing illumination, and its working mode and imaging characteristics are more suitable for fixed point observation of water bodies with higher abundance of plankton.
  • the dark field imaging device When the dark field imaging device is working, if the target is outside the depth of field and the depth of field is illuminated, it will produce defocusing halo and form background noise, which will cause the contrast of the dark field image of the plankton to decrease, which is not conducive to subsequent image processing. analysis.
  • the azimuth angle covered by the grazing illumination if the azimuth angle covered by the grazing illumination is limited, it may cause shadow effects in the image of the plankton, resulting in artificial artifacts of the image texture, and will also generate subsequent image processing analysis. Negative Effects.
  • the glancing illumination beam is generally limited in space, so that the axial depth of the illumination region is matched with the depth of field of the imaging device as much as possible to suppress background noise and ensure imaging contrast.
  • the circular illuminator of the LOKI system utilizes the one-dimensional concentrating action of a linear Fresnel lens to concentrate the light from the LED source as much as possible into the central region of the imaging field of view.
  • the dark field imaging device also increases the coverage of the glancing illumination azimuth as much as possible, so that the illumination is uniformized, and the shadow effect of the target in the captured image is avoided.
  • the UVP system uses a two-sided illumination
  • SPC uses the traditional Korla dark field illumination
  • both LOKI and CPICS systems use a circular illumination.
  • Both Korla and ring illumination have an azimuth of 360 degrees and are truly “shadowless” lighting.
  • dark field imaging In addition to spatially modulating illumination, dark field imaging also requires modulation of the illumination in the time dimension to improve imaging quality. Due to the influence of swimming ability and underwater water flow, plankton movement speed is often faster than imaging shutter speed. When this happens, the imaging produces a smearing effect, causing the motion blur of the target, which makes subsequent image analysis processing difficult. In order to avoid trailing blur, it is necessary to shorten the equivalent exposure time of the imaging camera to achieve "capture”. At present, digital cameras generally use electronic shutters, which can achieve extremely short exposure times directly through electronic settings. In contrast, the underwater dark field imaging device generally adopts a flash illumination operation mode, the illumination light illuminates the target in a pulse form, triggers the digital camera exposure at the moment of the illumination target, and records the target image.
  • the flash illumination can reduce the energy consumption of the light source on the one hand, and avoid the accumulation of a variety of plankton in the illumination area due to the phototaxis, which causes quantitative inaccuracy. Loss of the meaning of in situ measurement.
  • LED white light emitting diode
  • LED light sources as an incoherent light source, still have their shortcomings.
  • the divergence angle is still large.
  • the power density of LED illumination is still low, when using it to achieve flash illumination, in order to ensure that the brightness of the illumination instant is sufficiently strong, it is necessary to drive it with a large pulse current. This not only requires a special drive circuit, but also leads to an increase in total power consumption, which is detrimental to the reliability and extended working time of the underwater in-situ observation system.
  • LDs laser diodes
  • the LD is small in size and low in power consumption, contributing to miniaturization, reliability, and long-term operation of the in-situ imaging apparatus.
  • the inventors of the present invention have proposed a planktonic optical imaging apparatus and method based on LD laser illumination for the above technical problems, and are suitable for achieving high quality in situ imaging of plankton above 100 micrometers.
  • the system can be used for fixed-point observations, horizontal tow observations, vertical section observations or onboard underwater vehicles.
  • an underwater planktonic optical imaging apparatus comprising: an optical imaging unit for collecting underwater scattered light or fluorescent light emitted by the laser illumination unit by underwater illumination Plankton imaging; a laser illumination unit comprising a laser illumination module for illuminating an underwater imaging area; wherein the laser illumination module performs imaging illumination by emitting a fan-shaped optical layer to the illumination center.
  • the imaging device consists of a laser illumination unit, an optical imaging unit.
  • the optical imaging unit includes an optical module, an electronic control driving module, an image processing storage module, and a data transmission module.
  • the module is inside the sealed compartment, and the laser illumination unit is located outside the capsule and is connected to the capsule by a plurality of tubular support structures.
  • the laser illumination unit is composed of a plurality of laser fan-shaped illumination modules and deep ultraviolet band LEDs.
  • the laser fan-shaped illumination module is circularly distributed at equal angular intervals on the same plane, the illumination center direction is directed to the center of the circle, and the fan-shaped light layer having the same thickness is emitted to form overlapping polygonal layered illumination regions; the polygonal layered illumination region and the imaging
  • the optical axis is perpendicular to the depth of field, the thickness of the light layer is less than or equal to the depth of field, and the center of symmetry intersects the imaging optical axis.
  • the deep ultraviolet LED is adjacent to the laser fan-shaped illumination module, and has the same space and is equally angularly spaced apart, and the illumination direction also points to the center of the circle.
  • the laser sector lighting module comprises an LD laser, a set of beam diameter adjusting lens groups, a right angle prism, a Powell prism, a linear diaphragm, a set of deep ultraviolet LEDs and a watertight structure.
  • the wavelength at which the LD laser emits laser light may be a single visible wavelength band, or a combination of three color wavelengths of red, green, and blue.
  • the laser fan-shaped lighting module is packaged into a watertight tubular structure, and the entire structure is in an "L" shape.
  • an underwater plankton optical imaging apparatus auxiliary water sample manipulation unit characterized in that the water sample manipulation unit comprises a stepping motor, a transmission device, a ball screw piston, and a A stepper motor controller, a tubular sleeve and a watertight tubular structure.
  • the electromechanical module of the water sample manipulation unit is encapsulated in the watertight tubular structure to integrally form an open plunger structure, the opening of which is directed to the optical window of the optical imaging unit, the plane of the opening port being in close proximity to the imaging focal plane.
  • an underwater planktonic optical imaging apparatus characterized in that the optical module comprises an optical window, a telecentric lens, an optical beam splitter, a band pass filter, a converging lens, and a single point.
  • the photodetector and the area array digital camera are composed.
  • the optical window is made of fused silica or sapphire material and is coated with a titanium dioxide superhydrophobic film.
  • the object light collected by the telecentric lens is divided into two paths of transmission and reflection by a beam splitter, and the transmitted light is projected to a digital camera area array sensor through a band pass filter; the reflected light passes through another band pass filter and then passes through
  • the condenser lens is focused to a single point photodetector for generating a photo trigger signal.
  • the digital camera can be a monochrome or color CCD or CMOS camera.
  • the field of view of the imaging unit is slightly smaller than the area of the polygonal layered illumination area generated by the laser illumination unit.
  • the electronically controlled driving module is composed of a single chip microcomputer, an LD laser driver, a deep ultraviolet LED driver, an external camera trigger circuit and a camera control circuit.
  • an underwater planktonic optical imaging apparatus characterized in that the image processing storage module is composed of a discrete FPGA, a DSP, a GPU and a solid state memory based on an embedded platform, or is integrated by Heterogeneous embedded processor chip, GPU and solid state memory such as FPGA+DSP+ARM.
  • the FPGA is used for pre-processing of an original image
  • the DSP is used for advanced image processing algorithm implementation
  • the GPU is used for image parallel processing and a plankton recognition algorithm based on a deep neural network model
  • the solid state memory is used for floating Storage of statistical information obtained from biological raw images, compressed images, and image processing.
  • the data transmission module transmits data to the outside by sealing a data transmission interface on the end face of the cabin.
  • the data transmission module transmits data by wire between Ethernet or optical fiber and an external data center, or wirelessly transmits data between the mobile phone communication network or the satellite communication channel and the data center.
  • the capsule is an inner hollow cylinder made of organic or metallic material
  • the sealed compartment has a transparent window at one end and a gas nozzle, data transmission and external cable watertight interface at the other end.
  • deep-UV LEDs for preventing bio-adhesion
  • support and fixing structures for telecentric lenses, digital cameras and various electronic modules
  • thermal conductors in close contact with the digital camera heat sink structure and the sealed cabin. Cooling and stabilizing system operating temperature.
  • the sealed capsule has a plurality of deep ultraviolet LEDs distributed at equal angular intervals, which can form an illumination area of the optical window by direct, refraction, reflection, diffraction, scattering, etc.
  • the overlapping area of the window prevents the attachment of the organism by timed illumination.
  • an inert gas such as nitrogen
  • a two-position three-way valve and an air pump the gas is used by a bolt cover.
  • the mouth is sealed.
  • the laser illumination unit may be composed of a plurality of monochromatic laser fan-shaped illumination modules of the same wavelength to generate a monochromatic light polygonal layered illumination region through a monochrome digital camera in the optical imaging unit. Realize grayscale imaging of plankton.
  • the laser illumination unit may be composed of a plurality of laser-shaped illumination modules of the same wavelength including red, green and blue, to generate a white light polygonal layered illumination region through the optical imaging unit.
  • the color digital camera in the implementation of plankton color imaging.
  • the laser illumination unit may be composed of a plurality of different laser fan-shaped illumination modules each having a wavelength of red, green and blue, respectively, to generate a white light polygonal layered illumination region through the optical imaging unit.
  • the color digital camera in the implementation of plankton color imaging.
  • the rising edge of the exposure signal of the digital camera triggers the laser illumination unit to perform short pulse flash illumination, and the pulse width of the single light pulse is less than or equal to the exposure time of the camera, achieving free floating or swimming plankton Discrete single frame capture of the target.
  • the laser illumination unit is continuously illuminated, utilizing a short-time exposure within the free-running software of the digital camera to achieve discrete single-frame capture for free-floating or swimming plankton targets.
  • the laser illumination unit is continuously illuminated, and the water sample manipulation unit is used to drive the plankton to flow vertically through the focal plane into the field of view of the optical imaging device, and the overlap operation of the readout by the digital camera is performed. Mode to achieve multi-frame continuous shooting of plankton targets.
  • the underwater optical imaging apparatus may photograph a plankton target with a preset fixed operating parameter including a camera frame rate, an image size, an equivalent exposure time, and a laser illumination intensity.
  • the underwater optical imaging apparatus can automatically and automatically adjust the system operating parameters of the continuous frame after the real-time analysis result of the plankton target obtained by the pre-frame, and capture the plankton in an adaptive manner.
  • the operating parameters include camera frame rate, equivalent exposure time, and laser illumination intensity.
  • the laser illumination unit is continuously illuminated, and the digital camera is externally triggered by the single point photodetector to achieve discrete single frame capture for free floating or swimming plankton targets.
  • the laser illumination unit is continuously illuminated, and the water sample manipulation unit is used to drive the plankton to flow vertically through the focal plane into the field of view of the optical imaging device, and the single point photodetector external triggering device is used.
  • the digital camera is capable of realizing a three-dimensional tomographic image capture of a larger plankton target with a higher transparency and a two-dimensional image capture of an individual smaller plankton target.
  • the laser light emitted by the laser illumination unit is subjected to high frequency modulation in the operation mode of the continuous illumination or the pulse illumination, and the high frequency flash is generated during the actual exposure time of the digital camera. Multiple times to eliminate laser speckle noise in plankton images.
  • the underwater optical imaging device operates at a fixed position in the underwater position, by water flow or by the zooplankton itself swimming into the field of view of the imaging device for imaging.
  • an underwater plankton optical imaging method wherein the underwater optical imaging device operates at a fixed position in an underwater position, and the water sample manipulation unit of the underwater optical imaging device drives a water mass The wrapped plankton enters the field of view of the imaging device for imaging.
  • the optical imaging device can be vertically raised and lowered in a certain water body space, and the relative movement of the water body and the optical imaging device in the vertical direction causes the plankton to enter the field of view of the imaging device with the water flow. Imaging.
  • the optical imaging device may be fixed in a frame and horizontally towed by a surface vehicle at a certain water depth, and the relative movement of the water body and the optical imaging device in a horizontal direction causes the plankton to follow the water flow. Entering the field of view of the optical imaging device for imaging.
  • the optical imaging device can be mounted and fixed on the underwater vehicle, and the underwater vehicle moves in a three-dimensional space below the water surface, and the relative movement of the water body and the underwater vehicle in the forward direction makes the plankton Imaging with the water flow into the field of view of the optical imaging device.
  • an underwater plankton image processing method characterized in that an original image acquired by the underwater optical imaging device is preprocessed by using an FPGA in the image processing storage module.
  • an FPGA in the image processing storage module.
  • binarization and target contour expansion operations of the original image each plankton target subgraph is cut out in real time in a highly parallel manner, the image background data redundancy is removed, the plankton abundance is counted, and the calculation is performed. The total value of a single frame of pixels.
  • an underwater plankton image recognition method is provided, characterized in that the DSP in the image processing storage module is further processed by applying an advanced image processing algorithm to the original image preprocessed by the FPGA.
  • an underwater plankton image recognition and classification method characterized in that an embedded GPU in the image processing storage module processes and analyzes an image preprocessed by an FPGA and a DSP, based on the depth.
  • the neural network recognition model realizes intelligent identification and classification of plankton.
  • the invention can improve the contrast and signal-to-noise ratio of underwater plankton imaging, and is beneficial to subsequent identification and quantification of plankton based on image analysis; in addition, the invention can also expand underwater plankton optics The application mode and application scenarios of imaging, while reducing system power consumption, achieve system compactness, miniaturization and intelligence.
  • FIG. 1 is a schematic structural view of a planktonic optical imaging apparatus based on LD laser illumination according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram showing illumination effects of a planktonic optical imaging apparatus based on LD laser illumination according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram showing the structure and principle of a fan-shaped layered lighting module based on a monochrome or three-color LD laser according to an embodiment of the invention
  • FIG. 4 is a schematic diagram showing the relationship between the spectral response of a red, green, and blue color filter of a color digital camera and the center wavelength of a red, green, and blue LD;
  • FIG. 5 is a schematic diagram of an optical imaging unit of a phytoplankton optical imaging device based on LD laser illumination, in accordance with an embodiment of the present invention
  • FIG. 6 is a schematic diagram of a deep ultraviolet LED distribution of a phytoplankton optical imaging device based on LD laser illumination, in accordance with an embodiment of the present invention
  • FIG. 7 is a schematic diagram of a deep ultraviolet LED mounting position of an LD laser illumination-based planktonic optical imaging apparatus according to an embodiment of the present invention, wherein (a) is a side view and (b) is a rear view;
  • FIG. 8 is a schematic diagram of a deep ultraviolet LED illumination mode of a planktonic optical imaging device based on LD laser illumination according to an embodiment of the present invention, wherein (a) is direct illumination, (b) is a convergent illumination through a lens, and (c) is Reflected by a mirror;
  • FIG. 9 is a schematic diagram of an assisted water sample manipulation unit of a planktonic optical imaging apparatus based on LD laser illumination, in accordance with an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of an LD laser illuminated planktonic optical imaging apparatus according to an embodiment of the present invention integrated with an auxiliary water sample manipulation unit;
  • FIG. 11 is a timing diagram of illumination, triggering, camera exposure, and readout timing of different modes of operation of a phytoplankton optical imaging method based on LD laser illumination, wherein (a) is continuous illumination combined with a short exposure time of the camera.
  • Frame capture mode (b) triggers the camera discrete single-frame capture mode for short-pulse illumination, (c) continuous illumination combined with camera overlap exposure multi-frame continuous shooting mode, (d) triggers the camera discrete single-frame capture for continuous illumination combined with external signals Mode, (e) triggers the camera B-door exposure shooting mode for continuous illumination combined with external signals, and (f) triggers camera high-speed tomography 3D imaging shooting mode for continuous illumination combined with external signals;
  • FIG. 12 is a schematic diagram of a mounting platform of an underwater plankton imaging device based on LD laser illumination according to an embodiment of the present invention, wherein (a) is a shore-based/buoy underwater fixed-point observation, and (b) is a surface-level aircraft horizontal towing water. Under the observation, (c) is the underwater vertical section observation, and (d) is the underwater three-dimensional motion observation of the underwater vehicle.
  • FIG. 1 is a schematic structural view of a planktonic optical imaging apparatus based on LD laser illumination, in accordance with an embodiment of the present invention.
  • the underwater plankton imaging apparatus of the present invention uses the layered illumination light generated by the LD laser to illuminate the target with the vertical system optical axis laterally within the imaging depth of field, by collecting scattered light or fluorescence emitted by the object to be tested. Realize dark field imaging of plankton.
  • the imaging device is mainly composed of an optical imaging unit 1 and a laser illumination unit 2.
  • the optical imaging unit 1 includes an optical module, an electronic control driving module 16, an image processing storage module 17, and a data transmission module 22.
  • the optical imaging unit 1 is located inside the capsule 21; the laser illumination unit 2 is located outside the capsule 21 and is connected to the capsule 21 by a plurality of tubular support structures 3.
  • FIG. 2 is a schematic diagram of illumination effects of an LD laser illumination-based planktonic optical imaging apparatus according to an embodiment of the present invention
  • FIG. 3 is a sector diagram of a laser illumination unit 2 based on a monochromatic or trichromatic LD laser according to an embodiment of the present invention. Schematic diagram of the structure and principle of the layered lighting module.
  • the laser illumination unit 2 is composed of a plurality of (for example, three) fan-shaped laser illumination modules, as shown in FIG.
  • Each laser fan-shaped illumination module comprises an LD laser tube 4 arranged in sequence on the optical path, a set of beam diameter adjusting lens groups 5, a right-angle prism 6, a Powell prism 7, a set of deep ultraviolet LEDs 8, and a laser illumination module optical window 9 And the linear diaphragm 10, as shown in Figure 3.
  • the laser light emitted from the LD laser tube 4 is reflected by the right angle prism 6 through the beam diameter adjusting lens group 5, and the reflected beam is refracted by the Powell prism 7 to form a divergent fan-shaped optical layer; the incident beam is adjusted by the beam diameter adjusting lens group 5.
  • the diameter can control the thickness of the exiting fan-shaped light layer; the fan-shaped outgoing beam passes through the optical window 9 of the laser illumination module and is further suppressed by the linear diaphragm 10 to emit stray light; the optical component 9 and the tubular structure are used to form the above component by the laser illumination module (In addition to the linear diaphragm 10) water-sealed, the outgoing fan beam and the tubular structure are both "L" shaped.
  • the laser illumination modules are circumferentially distributed at equal angular intervals on the same plane.
  • the fan-shaped illumination center direction points to the center of symmetry, and emits a fan-shaped optical layer of uniform thickness.
  • the fan-shaped light layers overlap each other to form a polygonal layered illumination region that is perpendicular to the imaging optical axis and lies within the depth of field, the thickness being less than or equal to the depth of field, and the center of symmetry intersecting or immediately adjacent to the imaging optical axis.
  • the target to be measured is located near the center of the circular symmetry of the polygonal region, so illumination from multiple angles can be accepted, which makes the illumination more uniform and avoids shadowing by imaging.
  • all laser illumination modules (LD lasers) in the laser illumination unit 2 can adopt the same wavelength monochromatic illumination, for example, a common one in the visible light band or the near infrared band.
  • the wavelength is shown in Figure 2(a).
  • a plurality of fan-shaped laser illumination modules respectively emitting blue, green, and red monochromatic lasers respectively may be used to constitute the laser illumination unit 2 (as shown in FIG. 2(b)). It is also possible to constitute the laser illumination unit 2 by using a plurality of fan-shaped laser illumination modules capable of simultaneously emitting "white light” lasers composed of blue, green and red wavelengths (as shown in Fig. 2(c)).
  • a fan-shaped laser illumination module composed of a near-infrared LD laser may be additionally added to the above-mentioned monochromatic or tri-color laser illumination unit 2, through the optical imaging unit 1
  • the single-point detection channel generates a trigger signal that triggers laser illumination in the visible range and camera exposure to image the plankton (as shown in Figure 2(d)).
  • the laser illumination module optical window 9 can be irradiated from the interior of the laser illumination module through a set of deep ultraviolet LEDs 8 (as shown in Fig. 2(e)), deep ultraviolet
  • the manner and location of the LEDs 8 can be determined as shown in Figures 6 and 7.
  • FIG. 5 is a schematic structural diagram of an optical imaging unit 1 of an LD laser illumination-based planktonic optical imaging apparatus according to an embodiment of the present invention.
  • the optical module of the optical imaging unit 1 of the underwater plankton imaging apparatus includes an imaging unit optical window 11, a telecentric lens 12, a dichroic mirror/beam splitter 13, a first band pass filter 14 and a A two-pass filter 18, a condenser lens 19, a single-point photodetector 20, and an area array digital camera 15 are formed.
  • the imaging unit optical window 11 is a circular transparent plate having a certain thickness, which can be made of fused silica or sapphire material, and the surface of the contact with the external water body is plated with a titanium dioxide superhydrophobic film.
  • the light emitted by the plankton target illuminated by the laser illumination unit 2 passes through the imaging unit optical window 11 and is collected by the telecentric lens 12; the object light passing through the telecentric lens 12 is divided into two transmissions and reflections by the dichroic mirror/beam splitter 13 Road light.
  • the near-infrared LD sector laser module provided in the laser illumination unit 2 is always bright (as shown in FIG. 2(d))
  • the plankton entering its illumination area scatters near-infrared light, which is reflected by the dichroic mirror 13.
  • a second passband filter 18 having a narrow passband prevents the near-infrared light (such as sunlight) from other sources in the underwater environment from being detected and causing false triggering. If there is only one wavelength of visible light, then a passband bandpass filter is selected. If there are three wavelengths of blue, green and red, the corresponding bandpass filter with three passbands is selected. The visible light that has passed through the first band pass filter 14 is projected onto the digital camera area array sensor 15 for imaging.
  • the use of the band pass filter 14 in the transmitted light path can filter out stray light having complex spectral components other than the wavelength of the illumination light, which contributes to the environmental adaptability of the underwater imaging device and effectively improves the signal-to-noise ratio of the final captured image.
  • the imaging chip of the digital camera 15 may be a monochrome or color CCD or CMOS, and the area of the imaging chip should match the magnification and optical interface of the telecentric lens 12 such that the size of the imaging field of view is slightly smaller than the polygon generated by the laser illumination unit.
  • the area of the layered lighting area may be a monochrome or color CCD or CMOS, and the area of the imaging chip should match the magnification and optical interface of the telecentric lens 12 such that the size of the imaging field of view is slightly smaller than the polygon generated by the laser illumination unit.
  • the optical imaging unit 1 is enclosed in an inner hollow cylindrical capsule 21 made of organic matter (PVC or POM) or a metal material (which may be an aluminum alloy, a stainless steel, a titanium alloy or a white copper alloy).
  • the transparent imaging unit optical window 11 is located at one end of the capsule 21 and is fixedly sealed by a sealing ring and a mechanical snap ring or a flange.
  • the other end of the capsule 21 has a data transmission and an external cable watertight interface.
  • the sealing device may be subjected to an inert gas charging process before the imaging device is launched.
  • a nozzle joint 23 is provided at the rear end of the capsule 21, and when all the components inside the capsule 21 are installed, the capsule is closed.
  • the two-way three-way valve is used to connect the air pump, the capsule 21 and the inert gas bottle. First, the two-way three-way valve is adjusted to the state in which the air pump and the airtight chamber 21 are connected, and then the original air in the airtight chamber 21 is extracted by an air pump.
  • the two-position three-way valve is adjusted to the state in which the sealing chamber 21 is connected to the inert gas bottle, and the inert gas/protective gas (such as nitrogen) can be slowly filled into the sealing chamber 21. Finally, the gas nozzle 23 is sealed by a bolt cap. At this time, since there is almost no water vapor inside the sealed compartment 21, fogging does not occur.
  • the inert gas/protective gas such as nitrogen
  • the inside of the sealed chamber 21 adopts a UVC band (250-280 nm) deep ultraviolet LED 8 to illuminate the imaging unit optical window 11 from multiple angles, and the DNA replication damage effect of the microorganisms in the water by UVC ultraviolet light and the photocatalytic action with the titanium dioxide superhydrophobic film. Biofilm formation on the outer surface of the imaging unit optical window 11 is suppressed, thereby avoiding bio-adhesion.
  • the deep ultraviolet LED 8 is placed inside the sealed chamber 21, and the plurality of LEDs are distributed in a circular equiangular manner inside the sealed chamber, and the area of the illumination overlapping area is larger than or equal to the area of the optical window 11 of the imaging unit, as shown in FIG.
  • FIG. 7 is a schematic diagram of a mounting position of a deep ultraviolet LED 8 of an LD laser-illuminated planktonic optical imaging apparatus according to an embodiment of the present invention, wherein (a) is a side view and (b) is a rear view.
  • Figure 7 illustrates the geometric optical trace of the deep ultraviolet LED 8.
  • Point A is the position of the deep ultraviolet LED 8
  • OA represents the axial distance d of the LED 8 and the optical window 11 of the imaging unit
  • OC represents the longitudinal distance l of the deep ultraviolet LED 8 to the optical window 11 of the imaging unit
  • BE is the diameter D of the optical window 11 of the imaging unit.
  • ⁇ is the angle between the line OA and AC
  • ⁇ is the angle between the line CA and AE, that is, the illumination half angle of the LED 8
  • l' is the length of AC
  • AF and AG represent the deep ultraviolet LED 8 to the optical window 11 of the imaging unit. The distance from the lateral edge. Deep UV LED 8 illumination should follow the following distribution formula:
  • the ultraviolet illumination should be smaller than the working distance of the telecentric lens 12.
  • the imaging unit optical window 11 aperture 70 mm and the deep ultraviolet LED 8 divergence half angle 7.5 ° as an example, when the direct illumination scheme as shown in FIG. 8( a ) is adopted, the axial distance and diameter of the LED 8 from the imaging unit optical window 11 are taken. The distances are 230 mm and 132 mm, respectively, and the center line of the LED 8 is at an angle of 60° to the optical window 11 of the imaging unit.
  • the installation position of the deep ultraviolet LED 8 follows the purpose of the above formula in order to make its illumination in the optical window completely cover the light-passing area on the one hand, and the axial distance from the illumination/imaging optical axis as short as possible to keep the device on the other hand.
  • the deep ultraviolet LED 8 does not have to directly project the optical window 11 of the imaging unit, and the spatial volume can be compressed by adding a refractive, reflective or diffractive optical element to change the size of the illumination angle or the direction of light propagation, for example,
  • the distance between the telecentric lens 12 and the digital camera 15 to the optical window 11 of the imaging unit can be reduced, and the radial distance of the deep ultraviolet LED 8 can be reduced, thereby reducing the axial and radial dimensions of the capsule 21.
  • Fig. 8(c) is a reflective illumination scheme using a planar mirror 25, which realizes the deep ultraviolet LED 8 by changing the direction of propagation of the illumination light by using a mirror 25 at an angle to the optical axis of the deep ultraviolet LED 8. Compression of the volume of space occupied in the axial and radial directions.
  • the above-described arrangement principles and specific solutions are also applicable to the deep ultraviolet LED 8 of the laser illumination unit 2. That is, the above-described arrangement principle and specific scheme regarding the deep ultraviolet LED 8 are applicable to both the optical imaging unit 1 and the laser illumination unit 2, and may have any different combinations.
  • materials such as magnesium, aluminum, and zinc may be used and protected by a sacrificial anode method.
  • the capsule 21 is also in close contact with the heat sink structure of the digital camera 15 and the capsule 21
  • the thermal conductor is used for heat dissipation.
  • the electronically controlled driving module 16 in the capsule 21 is composed of a single chip microcomputer, an LD laser driver, an ultraviolet LED driver, an external camera trigger circuit, and a camera control circuit.
  • the image processing storage module 17 is composed of a discrete FPGA, DSP, GPU, and solid-state memory based on an embedded platform, or a heterogeneous embedded processor, a GPU, and a solid-state memory integrated with FPGA+DSP+ARM.
  • the FPGA performs pre-processing on the original image acquired by the underwater optical imaging device, and the DSP further processes the original image processed by the FPGA by using an advanced image processing algorithm, and the embedded GPU is processed by the FPGA and the DSP.
  • the latter image is processed and analyzed, and the plankton is intelligently identified and classified based on the deep neural network recognition model.
  • the heterogeneous embedded processor is a new processing chip (or system on a chip) that integrates FPGA, DSP and ARM into one chip.
  • the data transfer module 22 transmits various images or processes the resulting data to the outside by sealing the data transfer interface on the end face of the pod.
  • data can be transmitted by wire between Ethernet or optical fiber and an external data center, or wirelessly between a mobile communication network (such as a 4G network) or a satellite communication channel and a data center. Way to transfer data.
  • plankton In a natural underwater environment, plankton can swim freely or be surrounded by local water masses, and the direction of motion is random. Since the underwater imaging device is small in field of view and depth of field, the actual effective sampling water volume is small when imaging a slow or stationary water body. When the concentration of plankton is low, the number of plankton in this volume is scarce and the imaging efficiency is low. Therefore, the water flow can be manipulated to cover the fast flow of the plankton through the focal plane, and it is imaged in the process to enhance the detection flux.
  • FIG. 9 is a schematic diagram of a water sample manipulation unit 33 equipped with a LD laser illumination based planktonic optical imaging device in accordance with an embodiment of the present invention.
  • Figure 9 illustrates a piston-type water sample control device driven by a stepper motor that can be used as an auxiliary unit of an underwater plankton imaging device to achieve this.
  • the water sample control unit is connected by the stepping motor 28 via the transmission 29 to the ball screw 31 to drive the piston 32, and is reciprocated in the open cylindrical structure (glass or quartz tube) 30.
  • the electronically controlled drive module 16 controls its rotational speed and direction by the drive 27 of the stepper motor 28 so that the piston 32 can be driven to push the water within the tube for laminar motion.
  • the stepping motor 28 and its driver 27 and the ball screw 31 are all enclosed in the watertight structure 26, and only the outer end surface of the piston 32 is in contact with the external water body.
  • the water sample manipulation unit When the water sample manipulation unit is used, it can be connected to the capsule 21 of the underwater plankton imaging device through the tubular support structure 3 in a structure as shown in FIG. 10 such that the direction of movement of the piston 32 is collinear with the optical axis of the imaging device.
  • the nozzle is adjacent to the outside of the illumination layer.
  • the plankton in the tube when the plankton in the tube is pushed out of the nozzle by the piston 32, it is located in the focal plane of the imaging device, and is illuminated by the exiting light layer of the laser illumination unit 2, so that not only the image can be clearly imaged, but also the imaging throughput can be greatly improved.
  • the underwater plankton imaging device can realize a plurality of different imaging modes, and the images obtained in these different imaging modes also have different images. Image features. The timing relationships for illumination, triggering, camera exposure, and readout in different imaging modes are shown in Figure 11.
  • Figure 11 (a) illustrates the free snap mode under continuous illumination.
  • the laser illumination unit 2 is continuously illuminated, and the short-time exposure (several microseconds to tens of microseconds) is triggered by the free running software of the digital camera, which can effectively avoid motion blur and achieve free floating or swimming float.
  • Figure 11 (b) illustrates the free snap mode under triggered illumination.
  • the rising edge of the exposure signal of the digital camera 15 triggers the laser illumination unit to perform short pulse flash illumination, and the pulse width of a single light pulse (several microseconds to tens of microseconds) is less than or equal to the exposure time of the camera (ten to several Tens of milliseconds) to avoid motion blur, enabling discrete single-frame capture imaging for free-floating or swimming plankton targets.
  • Figure 11 (c) illustrates the overlap exposure continuous shooting mode under continuous illumination.
  • the laser illumination unit 2 is continuously illuminated, and the water sample manipulation unit 33 is used to drive the plankton to flow vertically through the focal plane into the field of view of the imaging device, and is exposed by the digital camera 15 (exposure time of ten to several tens of milliseconds).
  • the overlapping working mode can realize multi-frame continuous shooting imaging of plankton targets.
  • Figure 11 (d) illustrates the external trigger capture mode under continuous illumination. In this mode, the near-infrared light in the laser illumination unit 2 is continuously illuminated, and the visible light laser flash illumination in the laser illumination unit 2 is externally triggered by the single-point photodetector 20 (the pulse width is several microseconds to several tens of microseconds).
  • Exposure to the digital camera 15 (exposure time of a few tenths to tens of milliseconds) enables discrete single-frame captures for free-floating or swimming plankton targets.
  • Figure 11 (e) illustrates an externally triggered focal plane superimposed two-dimensional projection imaging mode. In this mode, the laser illumination unit 2 or the near-infrared light is continuously illuminated, and the water sample manipulation unit 33 drives the plankton to flow vertically through the focal plane into the field of view of the imaging device, and the near-infrared detected by the single-point photodetector 20 is used.
  • the scatter signal or the visible light-excited fluorescent signal triggers the digital camera 15 to use B-gate continuous exposure integral imaging to achieve two-dimensional projection imaging of the focal plane superposition of the plankton target.
  • Figure 11 (f) illustrates an externally triggered three-dimensional tomography imaging mode. In this mode, the laser illumination unit 2 or the near-infrared light is continuously illuminated, and the water sample manipulation unit 33 drives the plankton to flow vertically through the focal plane into the field of view of the imaging device, and the near-infrared detected by the single-point photodetector 20 is used.
  • the scatter signal or the visible light-excited fluorescent signal triggers the high-speed continuous shooting of the digital camera 15 to achieve a three-dimensional tomographic imaging of the plankton with a larger transparency (greater than the thickness of the light layer) and a smaller individual (less than the thickness of the light layer).
  • Two-dimensional imaging of biological targets In the above six imaging modes, the temporal coherence can be reduced by the high frequency modulation laser pulse repetition rate, that is, multiple flashes in the actual exposure time of each frame of the digital camera 15 to illuminate the target to eliminate the laser high coherence.
  • Sexually induced imaging speckle noise improves image quality.
  • the operating parameters including the camera frame rate, the equivalent exposure time, the laser illumination intensity, and the water sample manipulation flow rate may remain unchanged during the image acquisition process, or may be obtained by the plankton obtained from the pre-order frame.
  • the target real-time analysis results dynamically adjust the system operating parameters when the frame is continuously adjusted, and capture the plankton target in an adaptive manner. For example, when the signal-to-noise ratio of the current sequence image is relatively poor, the laser illumination intensity can be gradually increased during subsequent image acquisition to improve the signal-to-noise ratio.
  • the water flow speed and the laser illumination intensity can be simultaneously increased to ensure that the image signal-to-noise ratio is unchanged.
  • the duty cycle of the plankton in subsequent images can be simultaneously increased.
  • Figure 12 illustrates four main scene platforms for underwater plankton optical imaging devices based on LD laser illumination that can be deployed:
  • the system can be fixed on the shore or under the instrument derrick under the anchor buoy, pushed by the water stream, the zooplankton itself swims, or the water sample manipulation unit 33 drives the water mass to wrap the plankton into the field of view of the imaging device.
  • the underwater vertical profile work.
  • the system can be vertically lifted by a ship-mounted winch through a cable in a certain body of water, and the relative movement of the water body and the optical imaging device in a vertical direction causes the plankton to image with the water flow into the field of view of the imaging device;
  • the system can be fixed in the frame, horizontally towed by the surface vehicle through the cable at a certain water depth, and the relative movement of the water body and the optical imaging device in the horizontal direction causes the plankton to enter the field of view of the optical imaging device with the water flow. .
  • the system can be mounted and fixed on the underwater vehicle, and the underwater vehicle can move freely in the underwater three-dimensional space.
  • the relative movement of the water body and the underwater vehicle forward direction causes the plankton to enter the visual field of the optical imaging device with the water flow. Imaging.
  • the acquired original image of the plankton can be pre-processed in real time by the FPGA in the image processing storage module 17.
  • the DSP in the image processing storage module further processes the original image preprocessed by the FPGA by applying an advanced image processing algorithm.
  • the embedded GPU in the image processing storage module 17 processes and analyzes the images preprocessed by the FPGA and the DSP, and intelligently identifies and classifies the plankton images, thereby achieving the objective of the identification and quantitative analysis of the plankton.

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Studio Devices (AREA)

Abstract

L'invention concerne un dispositif et un procédé d'imagerie optique de plancton sous-marin, comprenant: une unité d'imagerie optique (1) pour imager le plancton sous-marin en collectant la lumière diffusée ou la fluorescence émise par le plancton sous-marin éclairé par une unité d'éclairage laser; et une unité d'éclairage laser (2) comprenant un module d'éclairage laser pour éclairer une zone d'imagerie sous-marine; le module d'éclairage laser réalisant un éclairage d'imagerie par émission d'une couche de lumière en forme d''éventail vers un centre d'éclairage. Par rapport aux dispositifs et procédés existants, le dispositif et le procédé d'imagerie optique de plancton sous-marin peuvent améliorer le contraste et le rapport signal-bruit de l'imagerie de plancton sous-marin, ce qui est bénéfique pour l'identification et la quantification ultérieures du plancton sur la base d'une analyse d'image; en outre, le dispositif et le procédé d'imagerie optique de plancton sous-marin peuvent également étendre le mode d'application et des scénarios d'application d'imagerie optique de plancton sous-marin, et réduire en même temps la consommation d'énergie du système et obtenir une compacité, une miniaturisation et une intelligence du système.
PCT/CN2017/118958 2017-12-27 2017-12-27 Dispositif et procédé d'imagerie optique de plancton sous-marin WO2019127090A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/118958 WO2019127090A1 (fr) 2017-12-27 2017-12-27 Dispositif et procédé d'imagerie optique de plancton sous-marin

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/118958 WO2019127090A1 (fr) 2017-12-27 2017-12-27 Dispositif et procédé d'imagerie optique de plancton sous-marin

Publications (1)

Publication Number Publication Date
WO2019127090A1 true WO2019127090A1 (fr) 2019-07-04

Family

ID=67064214

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/118958 WO2019127090A1 (fr) 2017-12-27 2017-12-27 Dispositif et procédé d'imagerie optique de plancton sous-marin

Country Status (1)

Country Link
WO (1) WO2019127090A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110501689A (zh) * 2019-09-24 2019-11-26 中国工程物理研究院电子工程研究所 一种水下激光周向扫描光束发射系统
CN114509396A (zh) * 2022-04-20 2022-05-17 中国海洋大学 一种海洋浮游生物发光测量和识别装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN200968926Y (zh) * 2006-11-06 2007-10-31 国家海洋技术中心 拖曳式浮游生物监测设备
US20090109432A1 (en) * 2007-10-26 2009-04-30 Olson Robert J Systems and methods for submersible imaging flow apparatus
CN106066172A (zh) * 2016-06-13 2016-11-02 长春德信光电技术有限公司 一种用于水下距离选通探测成像的激光照明器
CN106248642A (zh) * 2016-07-15 2016-12-21 陈玲玲 一种智能分辨激光光学层析成像方法的系统
CN106872408A (zh) * 2017-04-26 2017-06-20 赣南师范大学 一种浮游生物成像检测装置
CN107478651A (zh) * 2017-07-04 2017-12-15 深圳市蓝海绿洲科技有限公司 一种拖曳式浮游生物成像仪及拖曳式浮游生物成像系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN200968926Y (zh) * 2006-11-06 2007-10-31 国家海洋技术中心 拖曳式浮游生物监测设备
US20090109432A1 (en) * 2007-10-26 2009-04-30 Olson Robert J Systems and methods for submersible imaging flow apparatus
CN106066172A (zh) * 2016-06-13 2016-11-02 长春德信光电技术有限公司 一种用于水下距离选通探测成像的激光照明器
CN106248642A (zh) * 2016-07-15 2016-12-21 陈玲玲 一种智能分辨激光光学层析成像方法的系统
CN106872408A (zh) * 2017-04-26 2017-06-20 赣南师范大学 一种浮游生物成像检测装置
CN107478651A (zh) * 2017-07-04 2017-12-15 深圳市蓝海绿洲科技有限公司 一种拖曳式浮游生物成像仪及拖曳式浮游生物成像系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110501689A (zh) * 2019-09-24 2019-11-26 中国工程物理研究院电子工程研究所 一种水下激光周向扫描光束发射系统
CN114509396A (zh) * 2022-04-20 2022-05-17 中国海洋大学 一种海洋浮游生物发光测量和识别装置
CN114509396B (zh) * 2022-04-20 2022-07-29 中国海洋大学 一种海洋浮游生物发光测量和识别装置

Similar Documents

Publication Publication Date Title
CN109981929B (zh) 水下浮游生物光学成像装置及方法
US10222688B2 (en) Continuous particle imaging and classification system
CN112352158A (zh) 用于水样的无标记分析的能够深度学习的便携式成像流式细胞仪
CN107158579B (zh) 一种可显化皮肤病区的自动定位光学治疗仪器及控制方法
CN201251551Y (zh) 水下浮游动物在线可视化监测装置
CN105765558A (zh) 低功率眼睛跟踪系统和方法
CN107728136A (zh) 一种机场跑道异物监测和清除引导系统及方法
CN109444056B (zh) 一种双目成像式水下光谱反射率原位测量装置及测量方法
US10482347B2 (en) Inspection of the contoured surface of the undercarriage of a motor vehicle
WO2019127090A1 (fr) Dispositif et procédé d'imagerie optique de plancton sous-marin
CN106769882B (zh) 溢油监测仪及其监测方法
CN107084681A (zh) 能主动适应水环境和水下目标的主动光照视觉成像系统和方法
CN106291737B (zh) 一种水下光谱复合成像探测系统及方法
CN207571761U (zh) 虹膜成像系统
CN110312079A (zh) 图像采集装置及其应用系统
CN107018388B (zh) 一种超高速实时彩色计算鬼成像的装置及方法
JP2022515993A (ja) 近接照明モジュール
WO2022137243A1 (fr) Technique optique pour l'analyse d'insectes, de crevettes et de poissons
CN112284294A (zh) 一种水下多波段交叉线阵激光三维扫描系统
CN101566577A (zh) 中小型海洋浮游动物景深成像监测装置
CN114302080A (zh) 一种高性能水下电视
CN104010165B (zh) 降水粒子阴影图像自动采集装置
CN209460409U (zh) 一种水下激光成像侦察设备
CN216083100U (zh) 一种水下鱼类信息采集装置
CN107991685B (zh) 基于线激发光源的面探测激光荧光雷达及探测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17935885

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17935885

Country of ref document: EP

Kind code of ref document: A1