CN116343205A - Automatic labeling method for fluorescence-bright field microscopic image of planktonic algae cells - Google Patents

Abstract

The invention provides an automatic labeling method for fluorescence-bright field microscopic images of floating algae cells, which comprises the steps of firstly synchronously measuring fluorescence of the floating algae cells and the bright field microscopic images, then carrying out digital image morphological processing on the fluorescence images, automatically drawing outlines of the bright field microscopic images of the floating algae cells through an image processing technology, converting the fluorescence images into mask images required by segmentation of mask NN networks of training examples, and finally training the mask NN networks by using the automatically generated labeling mask images, thereby providing an effective means for identifying the floating algae cells. The automatic labeling method of the invention greatly reduces or even replaces the workload of manual labeling.

Description

An automatic labeling method for fluorescence-bright-field microscopic images of planktonic algae cells

technical field

The invention belongs to the field of resources and the environment, in particular to the field of image recognition of phytoplankton, and specifically relates to an automatic labeling method for fluorescence-bright-field microscopic images of phytoplankton cells.

Background technique

The monitoring of phytoplankton diversity is an important part of biological evaluation of water quality and is of great significance to the protection of water ecological environment. The traditional method of microscopic algae community identification requires professional operation and is time-consuming and labor-intensive. There are very obvious morphological differences between different phytoplankton cell images. The phytoplankton monitoring method based on image recognition completes the classification of algae by extracting the shape, color, and texture features of algae cell images, which plays a pivotal role in the monitoring of phytoplankton cells. .

Image segmentation is an important step in identifying planktonic algae cell images. Traditional image segmentation methods such as threshold method and edge detection method have unstable segmentation results, and it is easy to segment complete algal cells into several parts and segment out background impurities. In recent years, image recognition methods based on convolutional neural networks such as Mask RCNN have been well applied in the segmentation of planktonic algae cell images. However, training convolutional neural networks requires a large amount of labeled data, and currently labeling images of phytoplankton cells still relies on manually drawing the outlines of algae cells with the mouse in LabelMe and other labeling tools, and the workload of labeling microscopic images of phytoplankton is huge. It has become a bottleneck problem that restricts the development of monitoring methods for planktonic algae cells based on image recognition technology.

Contents of the invention

The invention discloses a method for automatically labeling bright-field microscopic images by using fluorescent microscopic images of planktonic algae, which replaces the traditional workload of manually drawing algae cell outlines in labeling tools such as LabelMe.

The technical solution of the present invention is: a method for automatically labeling planktonic algae cell fluorescence-bright field microscopic images, comprising the following steps:

Step (1) synchronously collecting microscopic fluorescence images and microscopic bright field images of planktonic algae cells in a fluorescence-bright field dual-channel microscopic imaging instrument;

Step (2) converts the fluorescence image into a grayscale image, and converts the grayscale fluorescence image into a binary image by using Dalu method;

Step (3) Use the cross-shaped structural element M to perform morphological opening operation on the binary image to eliminate isolated points caused by noise factors:

Step (4) using the cross-shaped structural element M to perform a morphological expansion operation on the binarized image;

Step (5) determines the connected regions in the binary image, thereby automatically drawing the outline of the planktonic algal cells, and generating the example mask map required for training the Mask RCNN network;

Step (6) mark the algae species corresponding to each example mask image, and construct the planktonic algae cell image data set;

Step (7) First use the COCO dataset to pre-train the Mask RCNN network, and then use the planktonic algae cell image dataset to train the Mask RCNN network on the basis of the pre-trained model;

Step (8) Input the collected phytoplankton cell image into the trained Mask RCNN model, and output the frame, mask image and phytoplankton cell type of the phytoplankton cell image.

Beneficial effect:

Aiming at the problem that the data labeling of planktonic algae cell segmentation network is heavily dependent on manual labeling and the cost of manual labeling is high, the invention proposes a method for automatically labeling bright-field microscopic images using fluorescent microscopic images of planktonic algae cells. In this method, the bright-field images and fluorescence images of planktonic algae cells are measured synchronously in a fluorescence-bright-field dual-channel imaging system. Coded images, so as to realize the effective and rapid automatic generation of mask images, replacing the workload of manual labeling, and finally use the automatically generated label mask images to train the Mask RCNN network, which provides an effective means for the identification of planktonic algae cells.

Description of drawings

Fig. 1 method flowchart of the present invention;

Figure 2 Schematic diagram of Mask RCNN network structure;

Fig. 3 Effect diagram of bright field image and fluorescence image of Nostoc, a. bright field image, b. fluorescence image, c. additive fusion of bright field image and fluorescence image, d. automatically drawn mask image;

Fig. 4 Effect diagram of bright field image and fluorescence image of Peridinosa, a. bright field image, b. fluorescence image, c. additive fusion of bright field image and fluorescence image, d. automatically drawn mask image;

Fig. 5 Effect diagram of bright field image and fluorescence image of oocystis lacustrine; a. Bright field image, b. Fluorescence image, c. Additive fusion of bright field image and fluorescence image, d. Automatically drawn mask image;

Fig. 6 is an effect diagram after the automatic labeling method of the present invention is integrated into the LabelMe labeling tool, a. Nostoc, b. Peridinosa c. Lacustrine oocysts;

Figure 7 Mask RCN segmentation effect diagram; a. Nostoc, b. Peridinosa, c. Lacustrine oocysts.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The invention discloses a method for automatically labeling the outline of a bright-field microscopic image by using a fluorescence image of planktonic algae cells. As shown in FIG. 1 , the method includes the following steps:

Step (1) synchronously collecting microscopic fluorescence images and microscopic bright field images of planktonic algae cells in a fluorescence-bright field dual-channel microscopic imaging instrument;

Step (2) Convert the fluorescence image into a grayscale image, and convert the grayscale fluorescence image into a binary image by using Dalu method. In the binarized image B processed in this step, the bright area is the area where the planktonic algae cells are located, and the black area is the background area;

Step (3) Use the cross-shaped structural element M to perform morphological opening operation on the binarized image B to eliminate isolated points caused by noise and other factors:

Step (4) In order to ensure that the automatically drawn outline can contain the entire algal cell, use the cross-shaped structural element M to perform morphological expansion on the binary image _B1 :

Step (5) Determine the connected regions in the binary image _B2 through the cv2.findContours method in the Python-Opencv image processing library, so as to automatically draw the outline of planktonic cells and generate the instance masks required for training the Mask RCNN network picture. Its network structure is shown in Figure 2. Mask RCNN is an instance segmentation model developed on the basis of Faster RCNN. By using the fully connected network FCN at the output end, the bounding box of planktonic algae cells and the mask of segmentation are simultaneously output. Code map and classification results;

Step (6) uses LabelMe software to mark the algal species of each mask instance, and constructs a phytoplankton cell image data set;

Step (7) The COCO dataset is a large-scale image dataset for object detection and segmentation. In order to accelerate the convergence speed of the model, the COCO dataset is used to pre-train the Mask RCNN network, and then on the basis of the pre-trained model, the network is trained with the planktonic algae cell image dataset;

Step (8) Input the collected phytoplankton cell image into the trained Mask RCNN model, and output the frame, mask image and phytoplankton cell type of the phytoplankton cell image.

See Figure 3, which is an automatically labeled effect diagram, in which the bright field image and fluorescence image effect diagram of Nostoc are shown. In Figure 3, a is the bright field image, b is the fluorescence image, and c is the bright field image and fluorescence image. Additive fusion of images, d is the automatically drawn mask map;

The present invention carries out digital image processing on the fluorescent image to automatically obtain the mask image of the algae cells. In order to facilitate the labeling of the bright field image type, the source code of the labeling tool LabelMe is modified, as shown in a in Figure 6, when the bright field is turned on When imaging, automatically load the fluorescent image according to the path and draw a mask, then right-click the area where the algal cells are located and click the "Edit Label" button to label the type of algal cells.

See Figure 4, which is the effect diagram of the bright field image and the fluorescence image of Peridinosa, where a is the bright field image, b is the fluorescence image, c is the additive fusion of the bright field image and the fluorescence image, and d is the automatically drawn mask picture.

See Figure 5, which is the effect diagram of the bright field image and the fluorescence image of Oocyst algae, where a is the bright field image, b is the fluorescence image, c is the additive fusion of the bright field image and the fluorescence image, and d is the automatic drawing mask map.

Referring to Figure 6, the method for automatically drawing the outline of planktonic algae cells of the present invention is integrated into the LabelMe annotation. When the bright field image is opened, the fluorescent image is automatically loaded according to the path and the edge outline is drawn as shown in Figure 6. Users can modify the type of planktonic algae by right-clicking in the pop-up box. Fig. 6 is the effect diagram after the automatic labeling method of the present invention is integrated into the LabelMe labeling tool, wherein a is Nostoc, b is Peridinosa, and c is Lake Oocysts;

See Figure 7, which is the effect diagram of Mask RCN segmentation. After the Mask RCNN model training is completed, input the bright-field image of planktonic algae cells into the Mask RCNN model, and the instance segmentation results of planktonic algae can be obtained, including three types of algae types, mask images, and borders. An example image of the segmentation result is shown in Figure 7: where a is Nostoc, b is Peridinosa, and c is Lake Oocyst.

Although the illustrative specific embodiments of the present invention have been described above, so that those skilled in the art can understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, As long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are included in the protection list.

Claims (3)

1. A method for automatic labeling of planktonic algae cell fluorescence-bright-field microscopic image, is characterized in that, comprises the steps: Step (1) synchronously collecting microscopic fluorescence images and microscopic bright field images of planktonic algae cells in a fluorescence-bright field dual-channel microscopic imaging instrument; Step (2) converts the fluorescence image into a grayscale image, and converts the grayscale fluorescence image into a binary image by using Dalu method; Step (3) Use the cross-shaped structural element M to perform morphological opening operation on the binary image to eliminate isolated points caused by noise factors: Step (4) using the cross-shaped structural element M to perform a morphological expansion operation on the binarized image; Step (5) determines the connected regions in the binary image, thereby automatically drawing the outline of the planktonic algal cells, and generating the required example mask map for training the MaskRCNN network; Step (6), labeling the algae species corresponding to each example mask image, constructing a data set of planktonic algae cell images; Step (7), first use the COCO data set to pre-train the MaskRCNN network, and then use the planktonic algae cell image data set to train the MaskRCNN network on the basis of the pre-trained model; Step (8) Input the collected phytoplankton cell image into the trained MaskRCNN model, and output the frame, mask image and phytoplankton cell type of the phytoplankton cell image. 2. a kind of phytoplankton cell fluorescence-bright field microscope image automatic labeling method according to claim 1, is characterized in that, described step (2) comprises: In the binarized image B processed in this step, the bright area is the area where the phytoplankton cells are located, and the dark area is the background area. 3. a kind of phytoplankton cell fluorescence-bright field microscope image automatic labeling method according to claim 1, it is characterized in that, described step (4) uses cross-shaped structural element M to carry out morphological expansion to binary image operation, ensuring that the automatically drawn contours encompass the entire algal cell.

Images