CN118311030A - Marine plankton imager based on digital holography and data processing method - Google Patents

Abstract

The present invention discloses a marine plankton imager based on digital holography and a data processing method, comprising: a light source cabin and a data cabin; an imaging area is arranged between the light source cabin and the data cabin; the light source cabin comprises: a laser, a beam splitter prism arranged at the output end of the laser, and a first light path and a second light path arranged at the rear side of the beam splitter prism; the first light path comprises: a first beam expander, a first collimating lens and a first optical window arranged coaxially in sequence; the second light path comprises: a reflector, a second beam expander, a second collimating lens and a second optical window arranged coaxially in sequence. The present invention divides the light emitted by the light source into the first light path and the second light path through the beam splitting light path, respectively realizing holographic microscopic imaging of microscale plankton and large-field holographic imaging of medium and large plankton, and the data cabin has a built-in neural network, which is based on an automatic classification and counting algorithm of a deep learning algorithm, and the output result is directly the type and corresponding number of plankton.

Description

A marine plankton imager based on digital holography and data processing method

Technical Field

The invention relates to the technical field of underwater in-situ micro-scale biological monitoring instruments, and in particular to a marine plankton imager based on digital holography and a data processing method.

Background technique

Plankton, as a vital component of the marine ecosystem, plays a vital role in maintaining the marine food chain, material circulation and energy transfer. Although these tiny organisms are difficult to detect with the naked eye, they play a vital role in the health and stability of the entire marine ecosystem. Therefore, in-depth research on plankton not only helps us understand the distribution and utilization of marine resources, but also reveals the level of biodiversity and the impact of climate change on the marine ecosystem.

However, when collecting data on marine plankton, the plankton obtained based on optical imaging methods is limited by image resolution, field of view, and imaging depth, and it is impossible to simultaneously obtain micro-plankton and small plankton at the micrometer level, medium-sized plankton at the millimeter level, and large plankton up to the centimeter level.

Therefore, it is necessary to provide a marine plankton imager and data processing method based on digital holography to solve the above technical problems.

Summary of the invention

The present invention overcomes the shortcomings of the prior art and provides an ocean plankton imager based on digital holography and a data processing method.

To achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a marine plankton imager based on digital holography, comprising: a light source cabin and a data cabin; an imaging area is arranged between the light source cabin and the data cabin;

The light source cabin comprises: a laser, a beam splitter prism arranged at the output end of the laser, and a first optical path and a second optical path arranged at the rear side of the beam splitter prism; the first optical path comprises: a first beam expander, a first collimating lens and a first optical window which are coaxially arranged in sequence; the second optical path comprises: a reflector, a second beam expander, a second collimating lens and a second optical window which are coaxially arranged in sequence;

The data cabin comprises: a plurality of industrial cameras, microscope objective lenses and telecentric lenses respectively arranged at the front ends of the plurality of industrial cameras, a digital holographic system and an embedded processor; the microscope objective lenses are arranged toward the first optical window; the telecentric lenses are arranged toward the second optical window;

The embedded processor includes: a data transmission module, a database and a target detection module; a plurality of industrial cameras transmit imaging images to the database for storage through the data transmission module;

The target detection module is equipped with a neural network to analyze the images in the database, identify and count the types and quantities of plankton, and output statistical results.

In a preferred embodiment of the present invention, a plurality of submarine cable connectors are provided at the rear end of the data cabin, one of which is used to connect to an underwater docking platform or a user host computer to transmit the statistical results to the underwater docking platform or the host computer.

In a preferred embodiment of the present invention, another submarine cable connector is connected to the light source cabin to transmit the electrical energy after voltage reduction in the data cabin to the laser in the light source cabin.

In a preferred embodiment of the present invention, the digital holographic system is a coaxial holographic system, and the submarine cable connector is used to power the digital holographic system.

In a preferred embodiment of the present invention, the laser emitted by the laser is split into transmitted light and reflected light by the beam splitter prism, the transmitted light is directed toward the first beam expander, and the reflected light is directed toward the second beam expander; the ratio of the transmitted light energy to the reflected light energy is 1:1 to 50.

In a preferred embodiment of the present invention, the statistical results are automatically sent out in character form based on the RS232 communication standard.

In a preferred embodiment of the present invention, the microscope objective lens and its corresponding industrial camera parameters have pixel resolution of 0.5-2um and field of view of 2-36mm ² ; the telecentric lens and its corresponding industrial camera parameters have pixel resolution of 5-20um and field of view of 200-3600mm ² .

A data processing method for a digital holographic marine plankton imager based on any one of the above comprises the following steps:

S1, the laser emitted by the laser is modulated by the first optical path and the second optical path, and then emitted from two optical windows respectively, and illuminates the plankton in the imaging area;

S2. In the imaging area, the light scattered or diffracted by the plankton in the seawater interferes with the light beam that has not passed through the plankton. The hologram generated after the interference is imaged by the microscope objective lens or telecentric lens in the data cabin, and the image is formed in the industrial camera behind it, and the image is transmitted to the database;

S3-1, working mode of saving imaging images: the imaging images are saved in the database, and the saved imaging images are transmitted to the target detection module for processing, and the statistical results of plankton are output and saved as local documents in the database;

S3-2, working mode without saving the image: the image is temporarily stored in the database, and the image is transmitted to the target detection module for processing, the statistical results of plankton are output, and saved as local documents in the database, and the statistical results are sent to the underwater docking platform or host computer through the serial port of the instrument.

In a preferred embodiment of the present invention, in S3-1, the location and category of the plankton detected by the target detection module will be marked on the image and will replace the original image for storage.

In a preferred embodiment of the present invention, the statistical results output by S3-2 are uploaded to a server via a wireless network.

The present invention solves the defects existing in the background technology and has the following beneficial effects:

(1) The present invention provides an ocean plankton imager based on digital holography. Based on the hardware structure design, the light emitted by the light source is divided into a first light path and a second light path through a split light path, thereby realizing holographic microscopic imaging of microscale plankton and large-scale plankton with a large field of view. The data cabin has a built-in neural network. The neural network is based on an automatic classification and counting algorithm of a deep learning algorithm. The output result is directly the type and corresponding number of plankton, which solves the practical problems of low transmission efficiency, poor data security, limited storage capacity, data delay, etc. in data transmission between in-situ instruments in the ocean.

(2) The present invention is based on a dual optical path design, which corresponds to a microscope objective lens and an industrial camera with a pixel resolution of 0.5 to ² um and a field of view of 2 to 36 mm2, and a telecentric lens and an industrial camera with a pixel resolution of 5 to 20 um and a field of view of 200 to 3600 ^mm2 , respectively, so as to obtain the species information of plankton at all scales at the same time. The optical path based on the holographic microscope part can obtain the species information of micro-scale plankton and the high-resolution detail information of the plankton appearance. The holographic imaging optical path based on the large-field telecentric imaging part can obtain the species and quantity of large-scale plankton. The two imaging units are combined to realize full-scale observation of marine plankton from micron level to centimeter level, so as to further analyze the species and quantity information of plankton in the marine environment.

(3) The present invention sets an embedded processor in the data cabin to complete the processing of plankton image data in the instrument, and uses a deep learning-based target detection module to directly convert the plankton image data into string data of the type and corresponding quantity of plankton, thereby greatly compressing the amount of data uploaded from the sensor to the server or docking platform.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG1 is a three-dimensional structural diagram of a marine plankton imager based on digital holography according to a preferred embodiment of the present invention;

FIG2 is a schematic diagram of the internal structure of a marine plankton imager based on digital holography according to a preferred embodiment of the present invention;

FIG3 is a flow chart of a digital holographic-based marine plankton imager according to a preferred embodiment of the present invention;

In the figure: 100, light source cabin; 110, laser; 120, beam splitter; 130, first beam expander; 140, first collimating lens; 150, first optical window; 160, reflector; 170, second beam expander; 180, second collimating lens; 190, second optical window; 200, data cabin; 210, industrial camera; 220, microscope objective; 230, telecentric lens; 240, embedded processor; 250, submarine cable connector; 300, imaging area.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the scope of protection of the present application. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the invention, unless otherwise specified, "multiple" means two or more.

In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood by specific circumstances.

As shown in FIG1 , the present invention provides a marine plankton imager based on digital holography, comprising: a light source cabin 100 and a data cabin 200; an imaging area 300 is arranged between the light source cabin 100 and the data cabin 200; the light source cabin 100 and the data cabin 200 are arranged separately, and are arranged on both sides of the imaging area 300.

The light source cabin 100 in this embodiment includes: a laser 110, a beam splitter prism 120 arranged at the output end of the laser 110, and a first optical path and a second optical path arranged on the rear side of the beam splitter prism 120; the first optical path includes: a first beam expander 130, a first collimating lens 140 and a first optical window 150 arranged coaxially in sequence; the second optical path includes: a reflector 160, a second beam expander 170, a second collimating lens 180 and a second optical window 190 arranged coaxially in sequence.

It is worth mentioning that the light source emitted by the laser 110 is divided into two parts by the beam splitter prism 120, one part is emitted horizontally directly into the first light path, and the other part is emitted vertically downward into the second light path. The reflector 160 is set at an angle of 45° to reflect the vertical light from the beam splitter prism 120 as horizontal direct light entering the second beam expander 170, realizing the structural design of the same light source and two light paths.

The laser light emitted by the laser 110 in this embodiment is split into transmitted light and reflected light by the beam splitter prism 120 , the transmitted light is directed toward the first beam expander 130 , and the reflected light is directed toward the second beam expander 170 ; the energy ratio of the transmitted light to the reflected light is 1:1-50.

It is worth noting that different proportions of transmitted light energy and reflected light energy are distributed according to the parameters of the selected beam splitter prism 120 or beam splitter, preferably 1:4, to achieve different light intensities of the first light path and the second light path. The splitting ratio is changed by changing the coating material, mainly the reflectivity and transmittance of the coating.

The transmitted light passes through the first beam expander 130 and the first collimating lens 140 to form a parallel light beam, and the parallel light beam passes through the first optical window 150. The reflected light passes through the second beam expander 170 and the second collimating lens 180 to form a parallel light beam, and the parallel light beam passes through the second optical window 190. The light beams of the first light path and the second light path illuminate the imaging area 300. In the imaging area 300, the light scattered or diffracted by the plankton in the seawater interferes with the light beam that has not passed through the plankton to produce a holographic image.

The data cabin 200 in this embodiment includes: a number of industrial cameras 210, a microscope objective lens 220 and a telecentric lens 230 respectively arranged at the front end of the several industrial cameras 210, a digital holographic system and an embedded processor 240; the microscope objective lens 220 is set toward the first optical window 150; the telecentric lens 230 is set toward the second optical window 190.

The number of the industrial cameras 210 is preferably two, which respectively image the microscope objective lens 220 and the telecentric lens 230 .

The parameters of the microscope objective lens in this embodiment and its corresponding industrial camera are: pixel resolution of 0.5-2um, field of view of 2-36mm ² ; the parameters of the telecentric lens and its corresponding industrial camera are: pixel resolution of 5-20um, field of view of 200-3600mm ² .

It is worth mentioning that the first optical path based on the holographic microscopy part can obtain the species information of micro-scale plankton and the high-resolution detail information of the plankton appearance, and the second optical path based on the holographic imaging of the large-field telecentric imaging part can obtain the species and quantity of large-scale plankton.

The embedded processor 240 in this embodiment includes: a data transmission module, a database and a target detection module; a number of industrial cameras 210 transmit the imaging images to the database for storage through the data transmission module; the target detection module is equipped with a neural network to analyze the imaging images in the database, identify and count the types and quantities of plankton, and output the statistical results.

The construction of a neural network includes the following steps:

Step 1: Collect a large amount of plankton image data and ensure the diversity and representativeness of the data. Perform data enhancement and preprocessing on the collected image data to ensure the accuracy and robustness of the model training.

Data enhancement includes rotation, scaling, cropping, flipping, etc. to increase the generalization ability of the model; preprocessing includes image normalization, denoising, labeling and other preprocessing operations to ensure the quality and consistency of input data. The image data is divided into training set, validation set and test set for model training, validation and testing.

Step 2: Select the YOLOv9 neural network model and use the training set and validation set for end-to-end training. Evaluate the performance of the model on the test set, including accuracy, recall, mAP and other indicators. Further optimize the model based on the evaluation results, such as adjusting the model structure and adding data.

Step 3: Use the retrained YOLOv9 neural network model to detect the image data, identify different types of plankton, and obtain the quantity information of plankton in the picture.

Step 4: Perform statistical analysis on the detection results of the retrained YOLOv9 neural network model to generate quantitative statistics of each category of plankton.

Realize automatic identification and quantity counting of plankton.

The data cabin 200 of the present invention has a built-in embedded processor 240. The embedded processor 240 is provided with a target detection module equipped with a neural network and a built-in automatic classification and counting algorithm based on a deep learning algorithm. During the plankton collection process, the plankton hologram data is processed underwater in real time, and the plankton in the hologram is statistically classified. The instrument outputs directly the type and quantity information of the plankton.

In this embodiment, the rear end of the data cabin 200 is provided with a plurality of submarine cable connectors 250, one of which is used to connect to an underwater docking platform or a user host computer, and is used to transmit statistical results to the underwater docking platform or the host computer.

Another submarine cable connector 250 in this embodiment is connected to the light source cabin 100 to transmit the electrical energy after voltage reduction in the data cabin to the laser 110 in the light source cabin 100 .

The statistical results are automatically sent out in character form based on the RS232 communication standard to facilitate connection with other marine instruments or in-situ docking platforms, and facilitate subsequent adaptation with other instruments or platforms.

The digital holographic system in this embodiment is a coaxial holographic system. The light from the light source chamber 100 is irradiated on the water body, and then irradiated on the object in the water body. The diffracted or scattered light interferes with the light not irradiated on the object, and a hologram can be generated. At the same time, the submarine cable connector 250 is used to power the digital holographic system.

When the present invention is used, the laser 110 emits a laser, which is split by the beam splitter 120. Different proportional distributions of the transmitted light energy and the reflected light energy can be achieved according to the parameters of the selected beam splitter prism 120. The transmitted light/reflected light passes through the first beam expander 130/the second beam expander 170 and the first collimating lens 140/the second collimating lens 180 to form a parallel light beam. The parallel light beam passes through the first optical window 150/the second optical window 190 to illuminate the imaging area 300. In the imaging area 300, the light scattered or diffracted by the plankton in the seawater interferes with the light beam that has not passed through the plankton. The hologram generated after the interference is imaged by the microscope objective 220 or the telecentric lens 230 in the data cabin 200, and the image is formed in the industrial camera 210 behind it, and the data is transmitted and stored in the embedded processor 240.

As shown in FIG3 , the present invention also provides a data processing method for a marine plankton imager based on digital holography, comprising the following steps:

S1. The laser emitted by the laser 110 is modulated by the first optical path and the second optical path, and then emitted from two optical windows respectively to illuminate the plankton in the imaging area 300.

S2. In the imaging area 300, the light scattered or diffracted by the plankton in the seawater interferes with the light beam that has not passed through the plankton. The hologram generated after the interference is imaged by the microscope objective lens 220 or the telecentric lens 230 in the data cabin 200, and the image is formed in the industrial camera 210 behind it, and the image is transmitted to the database.

S3-1. Working mode of saving imaging images: the imaging images are saved in the database. The saved imaging images will be transmitted to the target detection module for processing, and the statistical results of plankton will be output and saved as local documents in the database.

S3-2, working mode without saving the image: the image is temporarily stored in the database, and the image is transmitted to the target detection module for processing, the statistical results of plankton are output, and saved as local documents in the database, and the statistical results are sent to the underwater docking platform or host computer through the serial port of the instrument.

In step S3-1, the location and category of the plankton detected by the target detection module will be marked on the image and will replace the original image for storage.

In this embodiment, the statistical results output by S3-2 can also be uploaded to the server via the wireless network.

The present invention sets an embedded processor 240 in the data cabin 200 to complete the processing of plankton image data in the instrument, and uses a deep learning-based target detection module to directly convert the plankton image data into character string data of the type and corresponding quantity of plankton, thereby greatly compressing the amount of data uploaded from the sensor to the server or docking platform.

The above is based on the ideal embodiment of the present invention. Through the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.

Claims (10)

1. A marine plankton imager based on digital holography, comprising: a light source cabin and a data cabin, characterized in that: an imaging area is provided between the light source cabin and the data cabin; The light source cabin comprises: a laser, a beam splitter prism arranged at the output end of the laser, and a first optical path and a second optical path arranged at the rear side of the beam splitter prism; the first optical path comprises: a first beam expander, a first collimating lens and a first optical window which are coaxially arranged in sequence; the second optical path comprises: a reflector, a second beam expander, a second collimating lens and a second optical window which are coaxially arranged in sequence; The data cabin comprises: a plurality of industrial cameras, microscope objective lenses and telecentric lenses respectively arranged at the front ends of the plurality of industrial cameras, a digital holographic system and an embedded processor; the microscope objective lenses are arranged toward the first optical window; the telecentric lenses are arranged toward the second optical window; The embedded processor includes: a data transmission module, a database and a target detection module; a plurality of industrial cameras transmit imaging images to the database for storage through the data transmission module; The target detection module is equipped with a neural network to analyze the images in the database, identify and count the types and quantities of plankton, and output statistical results. 2. According to claim 1, a marine plankton imager based on digital holography is characterized in that: a plurality of submarine cable connectors are provided at the rear end of the data cabin, one of which is used to connect to an underwater docking platform or a user host computer, and is used to transmit the statistical results to the underwater docking platform or the host computer. 3. According to claim 2, a marine plankton imager based on digital holography is characterized in that: another of the submarine cable connectors is connected to the light source cabin to transmit the electrical energy after the voltage reduction in the data cabin to the laser in the light source cabin. 4. The marine plankton imager based on digital holography according to claim 1 is characterized in that the digital holographic system is a coaxial holographic system, and the submarine cable connector is used to power the digital holographic system. 5. According to claim 1, a marine plankton imager and data processing method based on digital holography is characterized in that: the laser emitted by the laser is split into transmitted light and reflected light by the beam splitter prism, the transmitted light is directed toward the first beam expander, and the reflected light is directed toward the second beam expander; the ratio of the transmitted light energy to the reflected light energy is 1:1~50. 6. The marine plankton imager and data processing method based on digital holography according to claim 1 is characterized in that the statistical results are in character form and automatically sent out based on the RS232 communication standard. 7. A marine plankton imager and data processing method based on digital holography according to claim 1, characterized in that: the microscope objective lens and its corresponding industrial camera parameters: pixel resolution is 0.5-2um, field of view is ^2-36mm2 ; the telecentric lens and its corresponding industrial camera parameters: pixel resolution is 5-20um, field of view is ^200-3600mm2 . 8. A data processing method for a marine plankton imager based on digital holography according to any one of claims 1 to 7, characterized in that it comprises the following steps: S1, the laser emitted by the laser is modulated by the first optical path and the second optical path, and then emitted from two optical windows respectively, and illuminates the plankton in the imaging area; S2. In the imaging area, the light scattered or diffracted by the plankton in the seawater interferes with the light beam that has not passed through the plankton. The hologram generated after the interference is imaged by the microscope objective lens or telecentric lens in the data cabin, and the image is formed in the industrial camera behind it, and the image is transmitted to the database; S3-1, working mode of saving imaging images: the imaging images are saved in the database, and the saved imaging images are transmitted to the target detection module for processing, and the statistical results of plankton are output and saved as local documents in the database; S3-2, working mode without saving the image: the image is temporarily stored in the database, and the image is transmitted to the target detection module for processing, the statistical results of plankton are output, and saved as local documents in the database, and the statistical results are sent to the underwater docking platform or host computer through the serial port of the instrument. 9. The method for using a digital holographic-based marine plankton imager and data processing method according to claim 8 is characterized in that: in S3-1, the position and category of the plankton detected by the target detection module will be marked on the image and the original image will be replaced for storage. 10. The marine plankton imager and data processing method based on digital holography according to claim 8, characterized in that the statistical results output by S3-2 are uploaded to the server via a wireless network.