WO2023284340A1 - 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法 - Google Patents

一种自底向上的寄生虫虫种发育阶段及图像像素分类方法 Download PDF

Info

Publication number
WO2023284340A1
WO2023284340A1 PCT/CN2022/086750 CN2022086750W WO2023284340A1 WO 2023284340 A1 WO2023284340 A1 WO 2023284340A1 CN 2022086750 W CN2022086750 W CN 2022086750W WO 2023284340 A1 WO2023284340 A1 WO 2023284340A1
Authority
WO
WIPO (PCT)
Prior art keywords
parasite
detection
image
classification
category
Prior art date
Application number
PCT/CN2022/086750
Other languages
English (en)
French (fr)
Inventor
李柏蕤
连荷清
Original Assignee
北京小蝇科技有限责任公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小蝇科技有限责任公司 filed Critical 北京小蝇科技有限责任公司
Priority to EP22840984.3A priority Critical patent/EP4361945A1/en
Publication of WO2023284340A1 publication Critical patent/WO2023284340A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/698—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A50/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE in human health protection, e.g. against extreme weather
    • Y02A50/30—Against vector-borne diseases, e.g. mosquito-borne, fly-borne, tick-borne or waterborne diseases whose impact is exacerbated by climate change

Definitions

  • This application relates to the technical fields of calculation, calculation and counting, and specifically relates to a bottom-up method for classifying the developmental stages of parasite species and image pixels.
  • Parasite detection is the main means of public prevention and control in hospitals and CDCs.
  • Common parasites include Plasmodium, Amoeba, Leishmania donovani, Toxoplasma gondii, Babesia canis, Trypanosoma evanii, Carl's living leukocytozoa and so on. Flow cytometry screens for possible presence, not detection of parasites.
  • the detection methods of parasites mainly include:
  • Antigen or antibody detection by injecting a specific antibody (or antigen) into the sample and observing whether a specific reaction occurs, it is estimated whether there is a parasite in the microorganism;
  • the microscopic inspection method is closely related to the professional level of the inspectors, and has problems such as low efficiency, slow speed, high work intensity, easy fatigue, and easy missed inspections; training methods generally require The parasites grow on the culture medium for 18-24 hours, and the cycle is long; antigen or antibody detection requires a clear antigen-antibody reaction, or all methods need to be enumerated; genetic detection methods are expensive and difficult to perform under certain conditions. Poor areas (such as Africa, remote mountainous areas) are popularized.
  • the loss function cannot distinguish the contribution weights of the samples of the tail category and the samples of the head category to the update of the entire model, and the simple samples and the difficult samples have the same loss function. Therefore, the biggest problem in parasite detection and classification is the long-tail problem, that is, it is easy to miss the detection of parasites, the detection effect is not good, and it is not practical.
  • This application is to solve the problem of detection and classification of blood parasites, and provides a bottom-up method for classifying the developmental stages of parasite species and image pixels, using a microscope or automatic slide scanning equipment to digitize the slides, and constructing a Transformer-based
  • the deep learning algorithm detects parasites visually by field of view.
  • the focal_loss function, CB_focal_loss function and Jaccard function are used to suppress the long-tailed distribution.
  • the method is simple. Only a thin blood film can be used for parasite identification, and the time and labor cost of detection can also be reduced. Improve the level of parasite detection in local areas, and identify the species and developmental stages of parasites, with high efficiency and speed, reducing labor costs and regional medical differences.
  • This application provides a bottom-up parasite species developmental stage and image pixel classification method, including the following steps:
  • training set and test set under the microscope imaging equipment, the blood slides for labeling are seamlessly photographed and collected by field of view, and the border parts of the images are seamlessly connected by stitching adjacent images to obtain the image set for labeling , mark the position, category, developmental stage and pixel point category of the parasite on a single image in the image set for labeling, generate labeling data, and divide the labeling data into a training set and a test set;
  • Construction of Transformer-based blood parasite classification, detection and segmentation model build a Transformer-based blood parasite classification, detection and segmentation model, optimize the model with a specific loss function to suppress long-tail distribution, use training set and test set for model training, and obtain blood Segmentation model for parasite classification and detection;
  • Detection Input the image set to be detected into the classification, detection and segmentation model of blood parasites, detect and classify the species and developmental stages of blood parasites, obtain and output the detection results, and the detection is completed.
  • step S2 includes:
  • Model training use the training set and the test set for model training to obtain a blood parasite classification, detection and segmentation model.
  • the Transformer-based blood parasite classification, detection and segmentation model includes an encoder and a decoder set in sequence and multilayer neural network mapping layers;
  • the encoder consists of at least two multi-head self-attention modules connected in sequence, and the encoder is used to segment a single image into k image patches, through the query matrix Qi, value matrix Vi and keyword matrix Ki of the first multi-head self-attention module
  • the image block is linearly transformed to obtain the feature expression of the image block, and the feature expression is superimposed on the features of the multi-head self-attention module and then input to the next multi-head self-attention module until the last multi-head self-attention module obtains the final feature expression of the image block, and the image block
  • the final feature expression is superimposed with the position code to obtain the decoder input feature;
  • the decoder includes at least one multi-head self-attention module connected in sequence, and the decoder is used to decode the decoder input features through at least two multi-head self-attention modules to obtain the decoded feature vector;
  • the multi-layer neural network mapping layer is used to calculate the decoded feature vector to obtain the corresponding feature vector and obtain the detection frame coordinates (x n , y n , w n , h n ) and confidence through linear mapping.
  • a bottom-up parasite species development stage and image pixel classification method described in the present application as a preferred method, a single image data pixel is 608*608, k is 16, and the image block pixel is 38*38;
  • the encoder includes sequentially connected four multi-head self-attention modules
  • the decoder includes sequentially connected four multi-head self-attention modules
  • the multilayer neural network is a three-layer fully connected layer neural network.
  • a bottom-up parasite species development stage and image pixel classification method described in the present application, as a preferred method, in step S21, the algorithm for constructing a Transformer-based blood parasite classification, detection and segmentation model also includes any of the following or more:
  • the focal_loss function is:
  • FL( pt ) is the improved cross-entropy
  • ⁇ is the focusing parameter
  • ⁇ is the between-class weight parameter
  • ⁇ (1- pt ) is the modulation coefficient
  • the focal_loss function is used to suppress the long-tailed distribution with a small number of parasites and a large number of background categories, and the background category is blood cells;
  • the focal_loss function increases the weight of difficult samples to make the Transformer-based blood parasite classification detection segmentation model Focus more on hard samples during training.
  • step S22 the CB_focal function is:
  • En is the expected value of valid samples of a certain category, and N is the total number of samples of this category;
  • the CB_focal function is used to alleviate the long-tail problem in the parasite classification problem with inverse weighting using an effective sample size.
  • step S22 the Jaccard loss function is:
  • A is the predicted value and B is the actual value
  • the Jaccard loss function is used to alleviate the sample imbalance in which the number of pixels occupied by positive samples is small and the number of pixels occupied by negative samples is large.
  • step S4 when the parasite is Plasmodium, the parasite determined by each image in the picture set is fused Classes are used to determine the developmental stage of Plasmodium and finally to determine the species class.
  • the specific method of this application is as follows: First, input a 608*608 image, and divide it into 16 equal parts according to the pixel size of 38*38.
  • the 16 image blocks are sent to the encoder's 'M-SELF-ATTENTION' module as the image blocks that need to calculate the attention coefficient, and the correlation between the image block and other image blocks is calculated respectively .
  • For each image block there is a query matrix Qi, a value matrix Vi, and a keyword matrix Ki.
  • the operation rules of the ‘multi-head self-attention’ module are as follows:
  • the coordinates of each detection frame Category information, the category information of each pixel.
  • the category information is counted, and the percentage of the pixel category that accounts for the most in the detection frame is obtained, which is used as the confidence level of the pixel level of the cell category information in the detection frame,
  • the confidence degree and the category confidence degree obtained by the classification head are weighted and fused to obtain the final category probability, and the category with the highest probability is selected as the final output category.
  • pos is the position number of the image block in the entire image
  • i is a certain dimension of the position encoding
  • d model is the total dimension of the position encoding
  • the entire model is affected by the offset of the long-tail distribution, that is, the classification model is performing category recognition. Sometimes it tends to have a large number of head categories, resulting in recognition errors. So we applied focal_loss and CB_focal_loss respectively.
  • the parasite category is the tail category, and the parasite has a tendency to be recognized as the background category.
  • the modulation coefficient tends to 1 at this time, that is, the sample is a difficult sample, and its contribution to the entire loss is 100%.
  • the classification is correct and it is a simple sample, and the modulation coefficient tends to 0.
  • the contribution of the sample to the total loss is very small, so the loss function can effectively alleviate the long-tail distribution. coming detection problems.
  • the model faces a severe long-tail problem in classification and recognition, that is, the classification results tend to be more head parasites. Similar to the loss function of the detection stage, we have made further improvements on the classification problem.
  • En is the expected value of valid samples of a certain category
  • N is the total number of samples of this category
  • the classification of Plasmodium consists of two levels, the first category is the specific species, the second level is the developmental stage of the species, and the two levels together constitute the classification of Plasmodium.
  • Other parasites only have a first-level category, and generally speaking, there is only one species of parasite in a blood slide, so the identification of the parasite category needs to be determined in combination with each parasite category in the entire blood slide, collected under the microscope imaging equipment
  • the images of the blood slides constitute the image database of blood slides. For each image in the database, the algorithm shown in step 3 is used to detect individual parasites and determine the species and developmental stages of the parasites. category.
  • This application proposes full-slice collection and splicing of adjacent images at physical locations to avoid omissions, which is very meaningful for samples with a small number of parasites.
  • This application uses transformer technology to integrate parasite detection and classification, without additional modules for segmentation and classification, in one step, and uses a category fusion mechanism to make the final recognition result more accurate.
  • This application uses the focal_loss function, CB_focal_loss function and Jaccard function to effectively suppress the long-tail distribution of the number and species of parasites, avoid missed detection, and improve detection accuracy;
  • This application can carry out parasite identification only with a thin blood film, which can also reduce the time and labor cost of detection, improve the detection level of parasites in local areas, and can identify parasite species and developmental stages, and the efficiency is high. High, fast, reduce labor costs and regional medical differences.
  • Fig. 1 is a bottom-up parasite species development stage and image pixel classification method embodiment 1-2 flowchart;
  • Fig. 2 is a flow chart of step S2 of embodiment 1-2 of a bottom-up parasite species developmental stage and image pixel classification method
  • Fig. 3 is a flow chart of embodiment 3 of a bottom-up parasite species development stage and image pixel classification method
  • Fig. 4 is a bottom-up parasite species development stage and image pixel classification method embodiment 3 parasite detection and classification model architecture schematic diagram;
  • Fig. 5 is a bottom-up parasite species development stage and image pixel classification method embodiment 3 insect species and development stage classification algorithm flowchart;
  • FIG. 6 is a flow chart of category fusion algorithm in Example 3 of a bottom-up method for classifying developmental stages of parasite species and image pixels.
  • a bottom-up parasite species development stage and image pixel classification method includes the following steps:
  • training set and test set under the microscope imaging equipment, the blood slides for labeling are seamlessly photographed and collected by field of view, and the border parts of the images are seamlessly connected by stitching adjacent images to obtain the image set for labeling , mark the position, category, developmental stage and pixel point category of the parasite on a single image in the image set for labeling, generate labeling data, and divide the labeling data into a training set and a test set;
  • Construction of Transformer-based blood parasite classification, detection and segmentation model build a Transformer-based blood parasite classification, detection and segmentation model, use a loss function to optimize the model to suppress the long-tail distribution, use the training set and test set for model training, and obtain blood parasites Insect classification detection segmentation model;
  • Detection Input the image set to be detected into the classification, detection and segmentation model of blood parasites, detect and classify the species and developmental stages of blood parasites, obtain and output the detection results, and complete the detection.
  • a bottom-up parasite species development stage and image pixel classification method includes the following steps:
  • training set and test set under the microscope imaging equipment, the blood slides for labeling are seamlessly photographed and collected by field of view, and the border parts of the images are seamlessly connected by stitching adjacent images to obtain the image set for labeling , mark the position, category, developmental stage and pixel point category of the parasite on a single image in the image set for labeling, generate labeling data, and divide the labeling data into a training set and a test set;
  • Construction of Transformer-based blood parasite classification, detection and segmentation model build a Transformer-based blood parasite classification, detection and segmentation model, use a loss function to optimize the model to suppress the long-tail distribution, use the training set and test set for model training, and obtain blood parasites Insect classification detection segmentation model;
  • S21 build a model: build the topology of the blood parasite classification detection segmentation model based on Transformer, the blood parasite classification detection segmentation model is used to detect the parasite position, category, developmental stage and pixel in a single image point class;
  • the Transformer-based blood parasite classification detection segmentation model includes an encoder, a decoder, and a multi-layer neural network mapping layer set in sequence;
  • the encoder consists of at least two multi-head self-attention modules connected in sequence, and the encoder is used to segment a single image into k image patches, through the query matrix Qi, value matrix Vi and keyword matrix Ki of the first multi-head self-attention module
  • the image block is linearly transformed to obtain the feature expression of the image block, and the feature expression is superimposed on the features of the multi-head self-attention module and then input to the next multi-head self-attention module until the last multi-head self-attention module obtains the final feature expression of the image block, and the image block
  • the final feature expression is superimposed with the position code to obtain the decoder input feature;
  • the decoder includes at least one multi-head self-attention module connected in sequence, and the decoder is used to decode the decoder input features through at least two multi-head self-attention modules to obtain the decoded feature vector;
  • the multi-layer neural network mapping layer is used to calculate the decoded feature vector to obtain the corresponding feature vector and obtain the detection frame coordinates (x n , y n , w n , h n ) and confidence through linear mapping;
  • a single image data pixel is 608*608, k is 16, and an image block pixel is 38*38;
  • the encoder includes four multi-head self-attention modules connected in sequence, the decoder includes four multi-head self-attention modules connected in sequence, and the multi-layer neural network is a three-layer fully connected layer neural network;
  • the algorithm for constructing the Transformer-based blood parasite classification detection segmentation model also includes any one or more of the following:
  • the focal_loss function is:
  • FL( pt ) is the improved cross-entropy
  • ⁇ is the focusing parameter
  • ⁇ is the weight parameter between categories
  • ⁇ (1- pt ) is the modulation coefficient
  • the focal_loss function is used to suppress the long-tailed distribution with a small number of parasites and a large number of background categories, and the background category is blood cells;
  • the focal_loss function increases the weight of difficult samples to make the Transformer-based blood parasite classification detection segmentation model focus more on difficult samples during training;
  • the CB_focal function is:
  • En is the expected value of valid samples of a certain category, and N is the total number of samples of this category;
  • the CB_focal function is used to use an effective sample size for inverse weighting to alleviate the long-tail problem in the parasite classification problem;
  • the Jaccard loss function is:
  • A is the predicted value and B is the actual value
  • the Jaccard loss function is used to alleviate the sample imbalance in which the number of pixels occupied by positive samples is small and the number of pixels occupied by negative samples is large;
  • Model training use the training set and the test set for model training to obtain a blood parasite classification, detection and segmentation model
  • Detection input the image set to be detected into the blood parasite classification, detection and segmentation model, detect and classify the blood parasite species and developmental stages, obtain the detection results and output them, and when the parasite is Plasmodium, fuse the images
  • the parasite category determined by each image is collected to determine the developmental stage of Plasmodium and finally determine the parasite category, and the detection is completed.
  • a bottom-up parasite species development stage and image pixel classification method includes the following steps:
  • the application first seamlessly takes pictures and collects the blood slides field by field under the microscopic imaging equipment.
  • the border part is seamlessly connected by stitching adjacent images to avoid Leave out any possible parasites.
  • This application constructs a bottom-up parasite detection classification and pixel classification algorithm.
  • the encoder's 'M-SELF-ATTENTION' module As shown in Figure 4, first, input a 608*608 image, and divide it into 16 equal parts according to the pixel size of 38*38.
  • the 16 image blocks are sent to the encoder's 'M-SELF-ATTENTION' module as the image blocks that need to calculate the attention coefficient, and the correlation between the image block and other image blocks is calculated respectively .
  • For each image block there is a query matrix Qi, a value matrix Vi, and a keyword matrix Ki.
  • the operation rules of the ‘multi-head self-attention’ module are as follows:
  • the coordinates of each detection frame Category information, the category information of each pixel.
  • the category information is counted, and the percentage of the pixel category with the largest proportion in the detection frame is obtained, which is used as the confidence level of the pixel level of the cell category information in the detection frame,
  • the confidence degree and the category confidence degree obtained by the classification head are weighted and fused to obtain the final category probability, and the category with the highest probability is selected as the final output category.
  • the entire model is affected by the offset of the long-tail distribution, that is, the classification model is performing category recognition. Sometimes it tends to have a large number of head categories, resulting in recognition errors. So we applied focal_loss and CB_focal_loss respectively.
  • the parasite category is the tail category, and the parasite has a tendency to be recognized as the background category.
  • the modulation coefficient tends to 1 at this time, that is, the sample is a difficult sample, and its contribution to the entire loss is 100%.
  • the classification is correct and it is a simple sample, and the modulation coefficient tends to 0.
  • the contribution of the sample to the total loss is very small, so the loss function can effectively alleviate the long-tail distribution. coming detection problems.
  • the model faces a severe long-tail problem in classification and recognition, that is, the classification results tend to be more head parasites. Similar to the loss function of the detection stage, we have made further improvements on the classification problem.
  • En is the expected value of valid samples of a certain category
  • N is the total number of samples of this category
  • the classification of Plasmodium consists of two levels, the first category is the specific species, the second level is the developmental stage of the species, and the two levels together constitute the classification of Plasmodium.
  • Other parasites only have the first-level category, and generally speaking, a blood slide has only one species, so the identification of the parasite category needs to be determined by combining each parasite category in the entire blood slide, as shown in Figure 5-6
  • the images collected under the microscopic imaging equipment form the blood slide image database, and the algorithm shown in step 3 is used for each image in the database to detect a single parasite and determine the species and developmental stage category, and then the whole slide parasite Insect category fusion to determine the final insect category.
  • the embodiment of the present application provides a bottom-up method for classifying the developmental stages of parasite species and image pixels, using a microscope or an automatic slide scanning device to digitize the slides, and constructing a Transformer-based deep learning algorithm to detect parasites field by field , using the focal_loss function, CB_focal_loss function and Jaccard function to suppress the long-tailed distribution.
  • This application can carry out parasite identification through thin blood film, which can reduce the detection time and labor cost, improve the detection level of parasites in local areas, and can identify the parasite species and developmental stages, with high efficiency, fast speed, and low cost. Labor costs and regional medical differences.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

本申请提供一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，利用显微镜或者玻片自动扫描设备将玻片进行数字化，构建基于Transformer的深度学习算法逐视野进行寄生虫检测，采用focal_loss函数、CB_focal_loss函数和Jaccard函数抑制长尾分布。本申请方法简单，仅用薄血膜即可进行寄生虫鉴定，还可降低检测的时间和人工成本，提高局部区域的寄生虫检测水平，并且可对寄生虫虫种及发育阶段进行鉴定，效率高、速度快、降低人工成本和地区医疗差异。

Description

一种自底向上的寄生虫虫种发育阶段及图像像素分类方法
交叉引用
本申请要求于2021年07月15日提交的、申请号为202110802685.1、申请名称为“一种自底向上的寄生虫虫种发育阶段及图像像素分类方法”的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算、推算、计数技术领域，具体涉及一种自底向上的寄生虫虫种发育阶段及图像像素分类方法。
背景技术
寄生虫检测是医院、疾控中心等实现公共防控的主要手段，常见的寄生虫包括疟原虫、阿米巴原虫、杜氏利士曼原虫、弓形虫、犬巴贝斯虫、伊氏锥虫、卡氏住白细胞虫等。流式细胞术可筛选并提示可能存在，并不能检测寄生虫。
目前，寄生虫的检测方法主要包括：
(1)镜检：医生利用显微镜观察外周血涂片检查疟原虫是诊断疟疾的常规方法，通常需要在显微镜100x物镜下观察整个血玻片，一般需要查看上千个视野，耗时一天，如果发现疟原虫则为阳性，就可作为确诊的可靠依据。一次阴性结果不能否定诊断，需多次复查；
(2)培养：在一定的营养条件下，培养样本中的微生物，观察繁殖扩增的生长特性来判断是否存在寄生虫；
(3)抗原或抗体检测：通过向样本中注入特定抗体(或抗原)，观察是否发生特异性反应，来推定微生物是否存在寄生虫；
(4)基因检测：利用核酸杂交、基因芯片、聚合酶反应等技术，检测寄生虫的核酸序列来判断是否存在寄生虫。
但是，这些检测方法存在一定的局限性，镜检方法，与检测人员的专业水平密切相关，存在效率低、速度慢、工作强度大、人员易疲劳、易漏检等问题；培养方法，一般需要寄生虫在培养基上生长18-24小时，周期较长；抗原或抗体检测，则需要有明确的抗原抗体反应，或者需要枚举所有方式；基因检测方法，成本较高，难以在一些条件较差的地区(如非洲、偏远山区)得到普及。
目前也有采用数学模型进行寄生虫检测的先例，但是现有的寄生虫检测模型 在建立时，采用的传统的损失函数：
Figure PCTCN2022086750-appb-000001
由于寄生虫分布的长尾现象，该损失函数并不能将尾部类别的样本同头部类被的样本对于整个模型的更新拥有的贡献权重进行区分，且简单样本和困难样本对损失函数拥有相同的贡献度，因此无法克服寄生虫检测分类中最大问题即的长尾问题，即其易对寄生虫漏检，检测效果不佳，不具实用性。
发明内容
本申请是为了解决血液寄生虫的检测和分类问题，提供一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，利用显微镜或者玻片自动扫描设备将玻片进行数字化，构建基于Transformer的深度学习算法逐视野进行寄生虫检测，采用focal_loss函数、CB_focal_loss函数和Jaccard函数抑制长尾分布，方法简单，仅用薄血膜即可进行寄生虫鉴定，还可降低检测的时间和人工成本，提高局部区域的寄生虫检测水平，并且可对寄生虫虫种及发育阶段进行鉴定，效率高、速度快、降低人工成本和地区医疗差异。
本申请提供一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，包括如下步骤：
S1、训练集和测试集准备：在显微成像设备下对标注用血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到标注用图像集，在标注用图像集中的单个图像上标注出寄生虫位置、类别、发育阶段和像素点类别，生成标注数据，将标注数据分为训练集和测试集；
S2、基于Transformer的血液寄生虫分类检测分割模型构建：构建基于Transformer的血液寄生虫分类检测分割模型，采用特定损失函数优化模型以抑制长尾分布，使用训练集和测试集进行模型训练，得到血液寄生虫分类检测分割模型；
S3、全片采集及拼接：在显微成像设备下对待检测血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到待检测图像集；
S4、检测：将待检测图像集输入血液寄生虫分类检测分割模型中，进行血液 寄生虫虫种及发育阶段的检测和分类，得到检测结果并输出，检测完成。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S2包括：
S21、构建模型：构建基于Transformer的血液寄生虫分类检测分割模型的拓扑结构，血液寄生虫分类检测分割模型用于检测单个图像中的寄生虫位置、类别、发育阶段和像素点类别；
S22、构建优化目标：采用focal_loss函数、CB_focal_loss函数和Jaccard函数优化基于Transformer的血液寄生虫分类检测分割模型，损失函数抑制长尾分布；
S23、模型训练：使用训练集和测试集进行模型训练，得到血液寄生虫分类检测分割模型。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S21中，基于Transformer的血液寄生虫分类检测分割模型包括依次设置的编码器、解码器和多层神经网络映射层；
编码器包括顺序连接的至少两个多头自注意力模块，编码器用于将单个图像分割成k个图像块，通过第一个多头自注意力模块的查询矩阵Qi、值矩阵Vi和关键字矩阵Ki将图像块线性变换得到图像块的特征表达，将特征表达叠加多头自注意力模块的特征后输入下一个多头自注意力模块，直至最后一个多头自注意力模块得到图像块最终特征表达，图像块最终特征表达与位置编码叠加后得到解码器输入特征；
解码器包括顺序连接的至少一个多头自注意力模块，解码器用于将解码器输入特征经至少两个多头自注意力模块解码得到解码后特征向量；
多层神经网络映射层用于将解码后特征向量进行计算得到对应的特征向量并经过线性映射得到检测框坐标(x n,y n,w n,h n)和置信度。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，单个图像数据像素为608*608，k为16，图像块像素为38*38；
编码器包括顺序连接的四个多头自注意力模块，解码器包括顺序连接的四个多头自注意力模块，多层神经网络为三层全连接层神经网络。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S21中，构建基于Transformer的血液寄生虫分类检测分割模 型的算法还包括以下任意一种或多种：
SSD、FPN、Fast R-CNN、faster R-CNN、mask R-CNN、efficentNet、YOLO/v2/v3/v4/v5、RetianNet,Deeplabv1/v2v/3和Unet,MaskRcnn等。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S22中，focal_loss函数为：
FL(p t)＝-α t(1-p t) γlog(p t)；
其中FL(p t)为改进的交叉熵，γ为聚焦参数，α为类别间权重参数，α(1-p t)为调制系数。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，
α∈(0,+∞)，p t∈[0,1]，γ∈{1,2,3}；
focal_loss函数用于抑制寄生虫数量稀少，背景类别数量较大的长尾分布，背景类别为血液细胞；
当含有寄生虫的样本被错误识别为背景类别时，调制系数趋于1，含有寄生虫的样本为困难样本，focal_loss函数通过调高困难样本的权重，使得基于Transformer的血液寄生虫分类检测分割模型在训练时更专注于困难样本。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S22中，CB_focal函数为：
Figure PCTCN2022086750-appb-000002
其中，E n＝(1-β n)/(1-β)；
β＝(N-1)/N；
En为某一类别有效的样本的期望值，N为该类别总的样本数量；
CB_focal函数用于使用有效的样本量进行逆向加权缓解寄生虫分类问题中的长尾问题。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S22中，Jaccard损失函数为：
Figure PCTCN2022086750-appb-000003
其中，A为预测值，B为真实值；
Jaccard损失函数用于缓解正样本所占像素点个数稀少、负样本所占像素点数量较大的样本不均衡。
本申请所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，作为优选方式，步骤S4中，寄生虫为疟原虫时，融合所述图片集中每一个图像确定的寄生虫类别来确定疟原虫发育阶段并最终确定虫种类别。
本申请的具体方法为：首先，输入608*608的图像，将其按照38*38的像素大小等分成16份。将该16个图像块作为需要计算注意力系数的图像块，送入编码器的‘多头自注意力(M-SELF-ATTENTION)’模块，分别计算该图像块同其他图像块之间的相关度。其中对于每个图像块，都有查询矩阵Qi,值矩阵Vi，关键字矩阵Ki。‘多头自注意力’模块运算规则如下：
1)对于当前图像块，设置其查询矩阵Qi,并将38*38图像块向量化后作为值矩阵Vi，将Qi和Vi进行矩阵运算，得到线性变换后的查询矩阵Qi*。
2)对于其他的图像块，分别设置关键字矩阵Ki，利用当前值矩阵Vi，将Ki和Vi进行矩阵运算，得到线性变换后的关键字矩阵Ki*。
3)基于(1)所计算得到的Qi*,基于(2)所计算的16个矩阵组合维一个大矩阵K*，将Q*和K*进行矩阵相乘运算即得到相关度矩阵W，对W进行归一化操作得到该图像块同其他图像块之间的相关度(0-1)。
4)将该相关度矩阵同其他图像块的值矩阵Vi做矩阵乘法，得到基于不同图像块的加权特征。将该特征经过全连接层进行映射，得到最终的该图像块的特征表达。其中全连接层用来控制最终表述的特征维度。
5)将最后得到的特征同输入‘多头自注意力’的特征(或图像)进行‘叠加’操作，得到新特征，作为下一级‘多头自注意力’的输入。将上述新特征送入第二级‘多头自注意力’模块。
重复上述操作，在编码器内得到最终的特征。整个编码流程共进行了四级‘多头自注意力’计算。然后进行解码，将编码器得到的特征的位置信息进行编码。将该位置编码同编码器得到的特征进行‘叠加’操作，得到最终解码器的输入特 征。将该特征送入解码器，该解码器共包含四级‘多头注意力’操作，同编码器‘多头注意力’操作计算流程相似，最终解码器输出解码后的特征向量。将该特征向量分别送入检测框，类别，mask所对应的三层全连接层，得到不同任务独有的特征向量，将该特征向量分别进行线性映射，得到检测框坐标，每一个检测框的类别信息，每一个像素点的类别信息。依据检测头的检测结果，对于检测框内的像素点，统计类别信息，得到检测框内占比最多的像素类别所占百分比，作为该检测框的内细胞类别信息的像素点层面的置信度，将该置信度和分类头得到的类别置信度进行加权融合，得到最终的类别概率，选出概率最大的类别作为最终的输出类别。
Figure PCTCN2022086750-appb-000004
Figure PCTCN2022086750-appb-000005
其中pos为图像块在整个图像中的位置编号,i是位置编码的某个维度,d model为位置编码的总的维度。
由于寄生虫在血液中相对于血细胞数量数目较少，且在寄生虫内不同类别的细胞所采集到的样本数量不一，整个模型面临长尾分布的偏移影响，即分类模型在进行类别识别时会倾向于数量较多的头部类别，造成识别错误。因此我们分别应用了focal_loss和CB_focal_loss。
4.1检测头focal_loss
在检测时，由于寄生虫数量较少，且检测的背景类别为大量的血液细胞，寄生虫类别为尾部类别，寄生虫有被识别为背景类别的倾向。此时1、当一个样本被分错的时候，pt很小，此时调制系数就趋于1，即该样本为困难样本，且其对于整个损失的贡献度为100％。当pt趋于1的时候，此时分类正确而且是简单样本，调制系数趋于0，此时，该样本对于总的loss的贡献很小，所以该损失函数可以有效的缓解长尾分布所带来的检测问题。
4.2分类头CB_focal
由于在寄虫分数据采集过程中，不同类别的寄生虫数量不同，导致模型在分类识别面临严峻的长尾问题，即分类结果倾向于数量较多的头部寄生虫类别。同检测阶头的损失函数，我们在分类问题上进行了进一步的改进。
我们采用了有效样本理论，即在深度学习模型中，不同类别的样本对于模型优化的贡献程度不同，且对于某个类别，存在一个饱和的数据量，使得在该类别上继续增加数据对模型优化无显著影响，进而可以利用有效样本容量对损失函数进行加权。
假设某类的样本总容量N，我们所收集到的数据均是由该总集中采样得到。我们有
E n＝(1-β n)/(1-β)    (1)
β＝(N-1)/N     (2)
其中En为某一类别有效的样本的期望值，N为该类别总的样本数量。
当n＝1时，有效样本的期望为1，假设n＝N-1时，上式成立，则有n＝N时，有p＝En-1/N的概率同前面已经采样得到的样本重合，则此时
Figure PCTCN2022086750-appb-000006
由(1)，(2)，(3)式可知
Figure PCTCN2022086750-appb-000007
又因为
Figure PCTCN2022086750-appb-000008
所以
Figure PCTCN2022086750-appb-000009
但是，因为我们无法知晓某一类别确切的样本总量，所以我们用某一类别采集到的样本数量代替有效的样本总量。所以，此时，我们有加权损失函数
Figure PCTCN2022086750-appb-000010
利用有效的样本量进行逆向加权，可以一定程度上缓解分类问题中的长尾问题。
4.3语义分割头Jaccard损失函数
同理，在像素点分类(语义分割)任务中，同样面临正样本所占像素点个数稀少，负样本所占像素点数量较大，造成较为严重的样本不均衡问题，我们采用Jaccard损失函数来缓解此种问题。
Figure PCTCN2022086750-appb-000011
4.4类别确定
疟原虫的类别由两级组成，一级类别为具体虫种，二级为虫种发育阶段，两级共同构成疟原虫类别。其他寄生虫只有一级类别，且一般来说一张血玻片仅有一个虫种，因此虫种类别鉴定需要结合整张血玻片中每一个寄生虫类别才能确定，显微成像设备下采集的图像组成血玻片图像数据库，对数据库中每一张图像运用步骤3所示算法进行单个寄生虫检测及虫种和发育阶段类别判定，然后全玻片寄生虫虫种类别融合，确定最终虫种类别。
本申请具有以下优点：
(1)本申请提出全片采集，并对物理位置相邻图像进行拼接，避免遗漏，对于寄生虫数量较少的样本非常有意义。
(2)本申请利用transformer技术，将寄生虫检测和分类一体化，无需额外模块进行分割分类，一步到位，且采用类别融合机制，使得最终的识别结果更加精确。
(3)本申请采用focal_loss函数、CB_focal_loss函数和Jaccard函数，有效抑制寄生虫数量、种类的长尾分布，避免漏检，提高检测准确率；
(4)本申请仅用薄血膜即可进行寄生虫鉴定，还可降低检测的时间和人工成本，提高局部区域的寄生虫检测水平，并且可对寄生虫虫种及发育阶段进行鉴定，效率高、速度快、降低人工成本和地区医疗差异。
附图说明
图1为一种自底向上的寄生虫虫种发育阶段及图像像素分类方法实施例1-2流程图；
图2为一种自底向上的寄生虫虫种发育阶段及图像像素分类方法实施例1-2步骤S2流程图；
图3为一种自底向上的寄生虫虫种发育阶段及图像像素分类方法实施例3流程图；
图4为一种自底向上的寄生虫虫种发育阶段及图像像素分类方法实施例3寄生虫检测分类模型架构示意图；
图5为一种自底向上的寄生虫虫种发育阶段及图像像素分类方法实施例3虫种及发育阶段分类算法流程图；
图6为一种自底向上的寄生虫虫种发育阶段及图像像素分类方法实施例3类别融合算法流程图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。
实施例1
如图1所示，一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，包括如下步骤：
S1、训练集和测试集准备：在显微成像设备下对标注用血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到标注用图像集，在标注用图像集中的单个图像上标注出寄生虫位置、类别、发育阶段和像素点类别，生成标注数据，将标注数据分为训练集和测试集；
S2、基于Transformer的血液寄生虫分类检测分割模型构建：构建基于Transformer的血液寄生虫分类检测分割模型，采用损失函数优化模型以抑制长尾分布，使用训练集和测试集进行模型训练，得到血液寄生虫分类检测分割模型；
S3、全片采集及拼接：在显微成像设备下对待检测血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到待检测图像集；
S4、检测：将待检测图像集输入血液寄生虫分类检测分割模型中，进行血液寄生虫虫种及发育阶段的检测和分类，得到检测结果并输出，检测完成。
实施例2
如图1所示，一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，包括如下步骤：
S1、训练集和测试集准备：在显微成像设备下对标注用血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到标注用图像集，在标注用图像集中的单个图像上标注出寄生虫位置、类别、发育阶段和像素点类别，生成标注数据，将标注数据分为训练集和测试集；
S2、基于Transformer的血液寄生虫分类检测分割模型构建：构建基于Transformer的血液寄生虫分类检测分割模型，采用损失函数优化模型以抑制长尾分布，使用训练集和测试集进行模型训练，得到血液寄生虫分类检测分割模型；
如图2所示，S21、构建模型：构建基于Transformer的血液寄生虫分类检测分割模型的拓扑结构，血液寄生虫分类检测分割模型用于检测单个图像中的寄生虫位置、类别、发育阶段和像素点类别；
基于Transformer的血液寄生虫分类检测分割模型包括依次设置的编码器、解码器和多层神经网络映射层；
编码器包括顺序连接的至少两个多头自注意力模块，编码器用于将单个图像分割成k个图像块，通过第一个多头自注意力模块的查询矩阵Qi、值矩阵Vi和关键字矩阵Ki将图像块线性变换得到图像块的特征表达，将特征表达叠加多头自注意力模块的特征后输入下一个多头自注意力模块，直至最后一个多头自注意力模块得到图像块最终特征表达，图像块最终特征表达与位置编码叠加后得到解码器输入特征；
解码器包括顺序连接的至少一个多头自注意力模块，解码器用于将解码器输入特征经至少两个多头自注意力模块解码得到解码后特征向量；
多层神经网络映射层用于将解码后特征向量进行计算得到对应的特征向量并经过线性映射得到检测框坐标(x n,y n,w n,h n)和置信度；
单个图像数据像素为608*608，k为16，图像块像素为38*38；
编码器包括顺序连接的四个多头自注意力模块，解码器包括顺序连接的四个多头自注意力模块，多层神经网络为三层全连接层神经网络；
构建基于Transformer的血液寄生虫分类检测分割模型的算法还包括以下任意一种或多种：
SSD、FPN、Fast R-CNN、faster R-CNN、mask R-CNN、efficentNet、YOLO/v2/v3/v4/v5、RetianNet,Deeplabv1/v2v/3和Unet,MaskRcnn；
S22、构建优化目标：采用focal_loss函数、CB_focal_loss函数和Jaccard函数优化基于Transformer的血液寄生虫分类检测分割模型，损失函数抑制长尾分布；
focal_loss函数为：
FL(p t)＝-α t(1-p t) γlog(p t)；
其中FL(p t)为改进的交叉熵，γ为聚焦参数，α为类别间权重参数，α(1-p t)为调制系数；
α∈(0,+∞)，p t∈[0,1]，γ∈{1,2,3}；
focal_loss函数用于抑制寄生虫数量稀少，背景类别数量较大的长尾分布，背景类别为血液细胞；
当含有寄生虫的样本被错误识别为背景类别时，调制系数趋于1，含有寄生虫的样本为困难样本，focal_loss函数通过调高困难样本的权重，使得基于Transformer的血液寄生虫分类检测分割模型在训练时更专注于困难样本；
CB_focal函数为：
Figure PCTCN2022086750-appb-000012
其中，E n＝(1-β n)/(1-β)；
β＝(N-1)/N；
En为某一类别有效的样本的期望值，N为该类别总的样本数量；
CB_focal函数用于使用有效的样本量进行逆向加权缓解寄生虫分类问题中的长尾问题；
Jaccard损失函数为：
Figure PCTCN2022086750-appb-000013
其中，A为预测值，B为真实值；
Jaccard损失函数用于缓解正样本所占像素点个数稀少、负样本所占像素点数量较大的样本不均衡；
S23、模型训练：使用训练集和测试集进行模型训练，得到血液寄生虫分类检测分割模型；
S3、全片采集及拼接：在显微成像设备下对待检测血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到待检测图像集；
S4、检测：将待检测图像集输入血液寄生虫分类检测分割模型中，进行血液 寄生虫虫种及发育阶段的检测和分类，得到检测结果并输出，寄生虫为疟原虫时，融合所述图片集中每一个图像确定的寄生虫类别来确定疟原虫发育阶段并最终确定虫种类别，检测完成。
实施例3
如图3所示，一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，包括如下步骤：
1、全片采集及拼接
为了避免寄生虫的漏检，本申请首先在显微成像设备下对血玻片下逐视野无缝进行拍照采集，对于图像边界，采取相邻图像拼接的方式将边界部分无缝连接，以避免漏掉任何一个可能的寄生虫。
2、寄生虫数据标注
由专业医生在单个视野图上用特定标注工具标注出寄生虫位置，类别信息及像素点类别信息，标注量达到一定规模后，将标注数据按照一定比例分为训练集和测试集，为构建深度学习网络模型准备数据。
3、基于Transformer的血液寄生虫分类检测分割模型构建
本申请构建了一个自底向上的寄生虫检测分类及像素分类算法。
如图4所示，首先，输入608*608的图像，将其按照38*38的像素大小等分成16份。将该16个图像块作为需要计算注意力系数的图像块，送入编码器的‘多头自注意力(M-SELF-ATTENTION)’模块，分别计算该图像块同其他图像块之间的相关度。其中对于每个图像块，都有查询矩阵Qi,值矩阵Vi，关键字矩阵Ki。‘多头自注意力’模块运算规则如下：
1)对于当前图像块，设置其查询矩阵Qi,并将38*38图像块向量化后作为值矩阵Vi，将Qi和Vi进行矩阵运算，得到线性变换后的查询矩阵Qi*。
2)对于其他的图像块，分别设置关键字矩阵Ki，利用当前值矩阵Vi，将Ki和Vi进行矩阵运算，得到线性变换后的关键字矩阵Ki*。
3)基于(1)所计算得到的Qi*,基于(2)所计算的16个矩阵组合维一个大矩阵K*，将Q*和K*进行矩阵相乘运算即得到相关度矩阵W，对W进行归一化操作得到该图像块同其他图像块之间的相关度(0-1)。
4)将该相关度矩阵同其他图像块的值矩阵Vi做矩阵乘法，得到基于不同图像块的加权特征。将该特征经过全连接层进行映射，得到最终的该图像块的特征 表达。其中全连接层用来控制最终表述的特征维度。
5)将最后得到的特征同输入‘多头自注意力’的特征(或图像)进行‘叠加’操作，得到新特征，作为下一级‘多头自注意力’的输入。将上述新特征送入第二级‘多头自注意力’模块。
重复上述操作，在编码器内得到最终的特征。整个编码流程共进行了四级‘多头自注意力’计算。然后进行解码，将编码器得到的特征的位置信息进行编码。将该位置编码同编码器得到的特征进行‘叠加’操作，得到最终解码器的输入特征。将该特征送入解码器，该解码器共包含四级‘多头注意力’操作，同编码器‘多头注意力’操作计算流程相似，最终解码器输出解码后的特征向量。将该特征向量分别送入检测框，类别，mask所对应的三层全连接层，得到不同任务独有的特征向量，将该特征向量分别进行线性映射，得到检测框坐标，每一个检测框的类别信息，每一个像素点的类别信息。依据检测头的检测结果，对于检测框内的像素点，统计类别信息，得到检测框内占比最多的像素类别所占百分比，作为该检测框的内细胞类别信息的像素点层面的置信度，将该置信度和分类头得到的类别置信度进行加权融合，得到最终的类别概率，选出概率最大的类别作为最终的输出类别。
Figure PCTCN2022086750-appb-000014
Figure PCTCN2022086750-appb-000015
除本申请提出的模型之外，还包括SSD、FPN、Fast R-CNN、faster R-CNN、mask R-CNN、efficentNet、YOLO/v2/v3/v4/v5、RetianNet,Deeplabv1/v2v/3,Unet,MaskRcnn等算法均可完成。
4、抑制长尾分布
由于寄生虫在血液中相对于血细胞数量数目较少，且在寄生虫内不同类别的细胞所采集到的样本数量不一，整个模型面临长尾分布的偏移影响，即分类模型在进行类别识别时会倾向于数量较多的头部类别，造成识别错误。因此我们分别应用了focal_loss和CB_focal_loss。
4.1检测头focal_loss
1)focal_loss
FL(p t)＝-α t(1-p t) γlog(p t)
其中α∈(0,+∞)，pt∈[0,1]，γ∈{1,2,3}，α(1-pt)为调制系数
在检测时，由于寄生虫数量较少，且检测的背景类别为大量的血液细胞，寄生虫类别为尾部类别，寄生虫有被识别为背景类别的倾向。此时1、当一个样本被分错的时候，pt很小，此时调制系数就趋于1，即该样本为困难样本，且其对于整个损失的贡献度为100％。当pt趋于1的时候，此时分类正确而且是简单样本，调制系数趋于0，此时，该样本对于总的loss的贡献很小，所以该损失函数可以有效的缓解长尾分布所带来的检测问题。
4.2分类头CB_focal
由于在寄虫分数据采集过程中，不同类别的寄生虫数量不同，导致模型在分类识别面临严峻的长尾问题，即分类结果倾向于数量较多的头部寄生虫类别。同检测阶头的损失函数，我们在分类问题上进行了进一步的改进。
我们采用了有效样本理论，即在深度学习模型中，不同类别的样本对于模型优化的贡献程度不同，且对于某个类别，存在一个饱和的数据量，使得在该类别上继续增加数据对模型优化无显著影响，进而可以利用有效样本容量对损失函数进行加权。
假设某类的样本总容量N，我们所收集到的数据均是由该总集中采样得到。我们有
E n＝(1-β n)/(1-β)   (1)
β＝(N-1)/N  (2)
其中En为某一类别有效的样本的期望值，N为该类别总的样本数量。
当n＝1时，有效样本的期望为1，假设n＝N-1时，上式成立，则有n＝N时，有p＝En-1/N的概率同前面已经采样得到的样本重合，则此时
Figure PCTCN2022086750-appb-000016
由(1)，(2)，(3)式可知
Figure PCTCN2022086750-appb-000017
又因为
Figure PCTCN2022086750-appb-000018
所以
Figure PCTCN2022086750-appb-000019
但是，因为我们无法知晓某一类别确切的样本总量，所以我们用某一类别采集到的样本数量代替有效的样本总量。所以，此时，我们有加权损失函数
Figure PCTCN2022086750-appb-000020
利用有效的样本量进行逆向加权，可以一定程度上缓解分类问题中的长尾问题。
4.3语义分割头Jaccard损失函数
同理，在像素点分类(语义分割)任务中，同样面临正样本所占像素点个数稀少，负样本所占像素点数量较大，造成较为严重的样本不均衡问题，我们采用Jaccard损失函数来缓解此种问题。
Figure PCTCN2022086750-appb-000021
4.4类别确定
疟原虫的类别由两级组成，一级类别为具体虫种，二级为虫种发育阶段，两级共同构成疟原虫类别。其他寄生虫只有一级类别，且一般来说一张血玻片仅有一个虫种，因此虫种类别鉴定需要结合整张血玻片中每一个寄生虫类别才能确定，如图5-6所示，显微成像设备下采集的图像组成血玻片图像数据库，对数据库中每一张图像运用步骤3所示算法进行单个寄生虫检测及虫种和发育阶段类别判定，然后全玻片寄生虫虫种类别融合，确定最终虫种类别。
以上所述，仅为本申请较佳的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，根据本申请的技术方案及其发明构思加以等同替换或改变，都应涵盖在本申请的保护范围之内。
工业实用性
本申请实施例提供一种自底向上的寄生虫虫种发育阶段及图像像素分类方 法，利用显微镜或者玻片自动扫描设备将玻片进行数字化，构建基于Transformer的深度学习算法逐视野进行寄生虫检测，采用focal_loss函数、CB_focal_loss函数和Jaccard函数抑制长尾分布。本申请通过薄血膜即可进行寄生虫鉴定，可降低检测的时间和人工成本，提高局部区域的寄生虫检测水平，可对寄生虫虫种及发育阶段进行鉴定，效率高、速度快、降低人工成本和地区医疗差异。

Claims (10)

  1. 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：包括如下步骤：
    S1、训练集和测试集准备：在显微成像设备下对标注用血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到标注用图像集，在所述标注用图像集中的单个图像上标注出寄生虫位置、类别、发育阶段和像素点类别，生成标注数据，将所述标注数据分为训练集和测试集；
    S2、基于Transformer的血液寄生虫分类检测分割模型构建：构建基于Transformer的血液寄生虫分类检测分割模型，采用损失函数优化模型以抑制长尾分布，使用所述训练集和所述测试集进行模型训练，得到血液寄生虫分类检测分割模型；
    S3、全片采集及拼接：在显微成像设备下对待检测血玻片逐视野无缝进行拍照采集图像，采取相邻图像拼接的方式将图像边界部分无缝连接，得到待检测图像集；
    S4、检测：将所述待检测图像集输入所述血液寄生虫分类检测分割模型中，进行血液寄生虫虫种及发育阶段的检测和分类，得到检测结果并输出，检测完成。
  2. 根据权利要求1所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：
    步骤S2包括：
    S21、构建模型：构建所述基于Transformer的血液寄生虫分类检测分割模型的拓扑结构，所述血液寄生虫分类检测分割模型用于检测所述单个图像中的寄生虫位置、类别、发育阶段和像素点类别；
    S22、构建优化目标：采用focal_loss函数、CB_focal_loss函数和Jaccard函数优化所述基于Transformer的血液寄生虫分类检测分割模型，损失函数抑制长尾分布；
    S23、模型训练：使用所述训练集和所述测试集进行模型训练，得到所述血液寄生虫分类检测分割模型。
  3. 根据权利要求2所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：步骤S21中，所述基于Transformer的血液寄生虫分类检测分割模型包括依次设置的编码器、解码器和多层神经网络映射层；
    所述编码器包括顺序连接的至少两个多头自注意力模块，所述编码器用于将所述单个图像分割成k个图像块，通过第一个多头自注意力模块的查询矩阵Qi、值矩阵Vi和关键字矩阵Ki将所述图像块线性变换得到图像块的特征表达，将所述特征表达叠加所述多头自注意力模块的特征后输入下一个所述多头自注意力模块，直至最后一个所述多头自注意力模块得到图像块最终特征表达，所述图像块最终特征表达与位置编码叠加后得到解码器输入特征；
    所述解码器包括顺序连接的至少一个多头自注意力模块，所述解码器用于将所述解码器输入特征经至少两个所述多头自注意力模块解码得到解码后特征向量；
    所述多层神经网络映射层用于将所述解码后特征向量进行计算得到对应的特征向量并经过线性映射得到检测框坐标(x n,y n,w n,h n)和置信度。
  4. 根据权利要求3所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：所述单个图像数据像素为608*608，k为16，所述图像块像素为38*38；
    所述编码器包括顺序连接的四个所述多头自注意力模块，所述解码器包括顺序连接的四个所述多头自注意力模块，所述多层神经网络为三层全连接层神经网络。
  5. 根据权利要求1所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：步骤S21中，构建所述基于Transformer的血液寄生虫分类检测分割模型的算法还包括以下任意一种或多种：
    SSD、FPN、Fast R-CNN、faster R-CNN、mask R-CNN、efficentNet、YOLO/v2/v3/v4/v5、RetianNet,Deeplabv1/v2v/3和Unet,MaskRcnn。
  6. 根据权利要求2所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：步骤S22中，所述focal_loss函数为：
    FL(p t)＝-α t(1-p t) γlog(p t)；
    其中FL(p t)为改进的交叉熵，γ为聚焦参数，α为类别间权重参数，α(1-p t)为调制系数。
  7. 根据权利要求6所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：
    α∈(0,+∞)，p t∈[0,1]，γ∈{1,2,3}；
    所述focal_loss函数用于抑制寄生虫数量稀少，背景类别数量较大的长尾分布，所述背景类别为血液细胞；
    当含有寄生虫的样本被错误识别为背景类别时，所述调制系数趋于1，所述含有寄生虫的样本为困难样本，所述focal_loss函数通过调高所述困难样本的权重，使得所述基于Transformer的血液寄生虫分类检测分割模型在训练时更专注于所述困难样本。
  8. 根据权利要求2所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：步骤S22中，所述CB_focal函数为：
    Figure PCTCN2022086750-appb-100001
    其中，E n＝(1-β n)/(1-β)；
    β＝(N-1)/N；
    En为某一类别有效的样本的期望值，N为该类别总的样本数量；
    所述CB_focal函数用于使用有效的样本量进行逆向加权缓解寄生虫分类问题中的长尾问题。
  9. 根据权利要求2所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：
    步骤S22中，所述Jaccard损失函数为：
    Figure PCTCN2022086750-appb-100002
    其中，A为预测值，B为真实值；
    在像素点分类任务中采用所述Jaccard损失函数，所述Jaccard损失函数用于缓解正样本所占像素点个数稀少、负样本所占像素点数量较大的样本不均衡。
  10. 根据权利要求2所述的一种自底向上的寄生虫虫种发育阶段及图像像素分类方法，其特征在于：步骤S4中，寄生虫为疟原虫时，融合所述图片集中每一个图像确定的寄生虫类别来确定疟原虫发育阶段并最终确定虫种类别。
PCT/CN2022/086750 2021-07-15 2022-04-14 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法 WO2023284340A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22840984.3A EP4361945A1 (en) 2021-07-15 2022-04-14 Method for classifying species and development stage of parasite and classifying image pixel from bottom to top

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110802685.1 2021-07-15
CN202110802685.1A CN113256636B (zh) 2021-07-15 2021-07-15 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法

Publications (1)

Publication Number Publication Date
WO2023284340A1 true WO2023284340A1 (zh) 2023-01-19

Family

ID=77180464

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/086750 WO2023284340A1 (zh) 2021-07-15 2022-04-14 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法

Country Status (3)

Country Link
EP (1) EP4361945A1 (zh)
CN (1) CN113256636B (zh)
WO (1) WO2023284340A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116091870A (zh) * 2023-03-01 2023-05-09 哈尔滨市科佳通用机电股份有限公司 从板座破损故障识别检测网络训练及检测方法、系统及介质
CN116543385A (zh) * 2023-07-05 2023-08-04 江西农业大学 一种水稻叶片细胞形态学智能检测方法及装置

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113256636B (zh) * 2021-07-15 2021-11-05 北京小蝇科技有限责任公司 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法
CN115712828A (zh) * 2021-08-18 2023-02-24 华为技术有限公司 一种图像分类方法及其相关设备
CN114628031B (zh) * 2022-02-10 2023-06-20 郑州大学 检测癌症个体病人动态网络生物标志物的多模态优化方法
CN114419619B (zh) * 2022-03-29 2022-06-10 北京小蝇科技有限责任公司 红细胞检测分类方法、装置、计算机存储介质及电子设备
CN114418863B (zh) * 2022-03-31 2022-06-07 北京小蝇科技有限责任公司 细胞图像修复方法、装置、计算机存储介质及电子设备
CN115019891B (zh) * 2022-06-08 2023-07-07 郑州大学 一种基于半监督图神经网络的个体驱动基因预测方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110807426A (zh) * 2019-11-05 2020-02-18 北京罗玛壹科技有限公司 一种基于深度学习的寄生虫检测系统及方法
CN112749667A (zh) * 2021-01-15 2021-05-04 中国科学院宁波材料技术与工程研究所 一种基于深度学习的线虫分类识别方法
CN113256636A (zh) * 2021-07-15 2021-08-13 北京小蝇科技有限责任公司 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2730057A1 (en) * 2008-07-07 2010-01-14 Mass Institute Of Technology Micropatterned co-culture systems as infectious disease analysis platforms
US9836839B2 (en) * 2015-05-28 2017-12-05 Tokitae Llc Image analysis systems and related methods
CN111382804A (zh) * 2020-03-18 2020-07-07 长沙理工大学 一种识别样本不平衡的架空线路异常的方法
CN112766379B (zh) * 2021-01-21 2023-06-20 中国科学技术大学 一种基于深度学习多权重损失函数的数据均衡方法
CN113011485B (zh) * 2021-03-12 2023-04-07 北京邮电大学 多模态多病种长尾分布眼科疾病分类模型训练方法和装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110807426A (zh) * 2019-11-05 2020-02-18 北京罗玛壹科技有限公司 一种基于深度学习的寄生虫检测系统及方法
CN112749667A (zh) * 2021-01-15 2021-05-04 中国科学院宁波材料技术与工程研究所 一种基于深度学习的线虫分类识别方法
CN113256636A (zh) * 2021-07-15 2021-08-13 北京小蝇科技有限责任公司 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DAVOOD KARIMI; SERGE VASYLECHKO; ALI GHOLIPOUR: "Convolution-Free Medical Image Segmentation using Transformers", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 26 February 2021 (2021-02-26), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081893128 *
FUHAD K. M. FAIZULLAH, TUBA JANNAT FERDOUSEY, SARKER MD. RABIUL ALI, MOMEN SIFAT, MOHAMMED NABEEL, RAHMAN TANZILUR: "Deep Learning Based Automatic Malaria Parasite Detection from Blood Smear and Its Smartphone Based Application", DIAGNOSTICS, vol. 10, no. 5, 20 May 2020 (2020-05-20), pages 329, XP093024727, DOI: 10.3390/diagnostics10050329 *
YIN CUI; MENGLIN JIA; TSUNG-YI LIN; YANG SONG; SERGE BELONGIE: "Class-Balanced Loss Based on Effective Number of Samples", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 17 January 2019 (2019-01-17), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081003388 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116091870A (zh) * 2023-03-01 2023-05-09 哈尔滨市科佳通用机电股份有限公司 从板座破损故障识别检测网络训练及检测方法、系统及介质
CN116091870B (zh) * 2023-03-01 2023-09-12 哈尔滨市科佳通用机电股份有限公司 从板座破损故障识别检测网络训练及检测方法、系统及介质
CN116543385A (zh) * 2023-07-05 2023-08-04 江西农业大学 一种水稻叶片细胞形态学智能检测方法及装置
CN116543385B (zh) * 2023-07-05 2023-09-05 江西农业大学 一种水稻叶片细胞形态学智能检测方法及装置

Also Published As

Publication number Publication date
EP4361945A1 (en) 2024-05-01
CN113256636A (zh) 2021-08-13
CN113256636B (zh) 2021-11-05

Similar Documents

Publication Publication Date Title
WO2023284340A1 (zh) 一种自底向上的寄生虫虫种发育阶段及图像像素分类方法
US20240153272A1 (en) Method for detecting urine formed elements based on deep learning and context relation
JP6453298B2 (ja) 細胞診標本を観察および解析するためのシステムおよび方法
Quinn et al. Automated blood smear analysis for mobile malaria diagnosis
CN103518224B (zh) 用于分析微生物生长的方法
CN101900737A (zh) 基于支持向量机的尿沉渣有形成分自动识别系统
Davidson et al. Automated detection and staging of malaria parasites from cytological smears using convolutional neural networks
CN110807426B (zh) 一种基于深度学习的寄生虫检测系统及方法
Simon et al. Shallow cnn with lstm layer for tuberculosis detection in microscopic images
CN114140437A (zh) 一种基于深度学习的眼底硬渗出物分割方法
Li et al. Gadet: A geometry-aware x-ray prohibited items detector
CN114022688A (zh) 一种实时的牲畜身份识别方法
CN112949723A (zh) 一种子宫内膜病理图像分类方法
Yu et al. An improved automated diatom detection method based on YOLOv5 framework and its preliminary study for taxonomy recognition in the forensic diatom test
CN114913523B (zh) 基于yolox的植物气孔多功能实时智能识别系统
CN114037868B (zh) 图像识别模型的生成方法及装置
CN112785559B (zh) 基于深度学习的多个异构模型相互组合的骨龄预测方法
Kelley et al. Honey sources: neural network approach to bee species classification
CN115482530A (zh) 程序产品、信息处理方法和信息处理装置
CN118072115B (zh) 一种医学细胞检测方法及系统
CN117496276B (zh) 肺癌细胞形态学分析、识别方法及计算机可读存储介质
CN115018760B (zh) 一种基于人机混合增强智能的血细胞形态学辅助检验系统及方法
CN118052814B (zh) 基于ai技术的全自动标本前处理系统及方法
Zhao et al. Weimin Yu†, Qingqing Xiang†, Yingchao Hu, Yukun Du, Xiaodong Kang, Dongyun Zheng, He Shi, Quyi Xu, Zhigang Li, Yong Niu*, Chao Liu* and Jian Zhao
Feng et al. AI-powered microscopy image analysis for parasitology: integrating human expertise

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22840984

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022840984

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022840984

Country of ref document: EP

Effective date: 20240125

NENP Non-entry into the national phase

Ref country code: DE