WO2020253773A1 - 一种医学图像的分类方法、模型训练方法、计算设备以及存储介质 - Google Patents

一种医学图像的分类方法、模型训练方法、计算设备以及存储介质 Download PDF

Info

Publication number
WO2020253773A1
WO2020253773A1 PCT/CN2020/096797 CN2020096797W WO2020253773A1 WO 2020253773 A1 WO2020253773 A1 WO 2020253773A1 CN 2020096797 W CN2020096797 W CN 2020096797W WO 2020253773 A1 WO2020253773 A1 WO 2020253773A1
Authority
WO
WIPO (PCT)
Prior art keywords
medical image
medical
image
data set
classification
Prior art date
Application number
PCT/CN2020/096797
Other languages
English (en)
French (fr)
Inventor
肖凯文
韩骁
叶虎
周昵昀
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2020253773A1 publication Critical patent/WO2020253773A1/zh
Priority to US17/375,177 priority Critical patent/US11954852B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/155—Segmentation; Edge detection involving morphological operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Definitions

  • This application relates to medical image classification methods, model training methods, and servers, and specifically to microscope image classification methods, model training methods, computing devices, and storage media.
  • Pathological examination and analysis is an important reference for the final judgment standard of clinical medicine.
  • the main process is to obtain the corresponding tissue cells from the diseased tissue, and confirm the condition of the disease by observing and analyzing the made pathological tissue sections under a microscope.
  • microscope pathological image data has shown a blowout, and the number of digital microscope pathological images has increased geometrically.
  • this application proposes a medical image classification method, model training method, computing device and storage medium.
  • the embodiment of the present application provides a method for classifying medical images, which is executed by a computing device.
  • the method includes: acquiring a data set of medical images; performing quality analysis on the data set of medical images to extract feature information of the medical images; based on the extracted feature information, using pre-trained deep learning for abnormal detection and classification of medical images
  • the network classifies medical images and obtains classification results.
  • the embodiment of the present application also provides a method for training a model for abnormal detection and classification of medical images, which is executed by a computing device, and the method includes: acquiring a data set of original medical images and a corresponding set of original image annotation information; Perform quality analysis on the image data set to extract the feature information of the original medical image; train a deep learning network for abnormal detection and classification of medical images based on the feature information in the extracted original medical image data set and the original image annotation information set, and obtain A trained deep learning network model for anomaly detection and classification of medical images.
  • the embodiment of the present application also provides a medical image classification device.
  • the device includes an acquisition module, a quality analysis module and a classification module.
  • the acquisition module is configured to acquire a data set of medical images.
  • the quality analysis module is configured to perform quality analysis on the data set of medical images and extract feature information of the medical images.
  • the classification module is configured to classify the medical image by using a pre-trained deep learning network for abnormal detection and classification of the medical image based on the extracted feature information, and obtain a classification result.
  • the embodiment of the present application also provides a training device for a model for abnormal detection and classification of medical images.
  • the training device includes an acquisition module, a quality analysis module and a training module.
  • the acquisition module is used to acquire the original medical image data collection and the corresponding original image annotation information collection.
  • the quality analysis module is used to perform quality analysis on the data set of the original medical image and extract the characteristic information of the original medical image.
  • the training module is used to train a deep learning network for abnormal detection and classification of medical images based on the extracted feature information in the original medical image data set and the original image annotation information set, and obtain a trained deep learning network for abnormal detection and classification of medical images model.
  • An embodiment of the present application also provides a computing device, including: a memory, a processor, and a computer program stored on the memory and capable of running on the processor.
  • the processor executes the program, the computer program described in the embodiment of the present application is implemented Methods.
  • the embodiment of the present application also provides a medical image classification system, wherein the medical image classification system includes a medical image acquisition device and a medical image processing device; the medical image acquisition device is used to scan medical images and send the medical images to the medical image processing device; The medical image processing equipment is used to perform any of the methods described in the embodiments of the present application.
  • the embodiment of the present application also provides a non-volatile computer-readable storage medium, which stores a computer program executable by a computer device.
  • the program runs on the computer device, the computer device executes the implementation of the present application. The method described in the example.
  • Figure 1 illustrates the method of screening microscope images according to related technologies
  • Figure 2 illustrates a smart microscope system to which the image classification and screening module according to an embodiment of the present application is applied;
  • Fig. 3 illustrates a schematic diagram of a method for training a model according to an embodiment of the present application
  • Figure 4a illustrates a schematic diagram of an original data set and a decoding and file verification module
  • Figure 4b illustrates a schematic diagram of the quality analysis module
  • Figure 4c illustrates a schematic diagram of an interpolation and normalization processing module
  • Figure 4d illustrates a schematic diagram of a deep learning network to be trained
  • Figure 4e illustrates a schematic diagram of loading a trained deep learning network model
  • FIG. 5a illustrates a system architecture diagram applicable to the method for classifying medical images according to an embodiment of the present application
  • Fig. 5b illustrates a flowchart of a method for classifying images according to an embodiment of the present application
  • Fig. 6 illustrates a flowchart of a method for training a deep learning network model according to an embodiment of the present application
  • Fig. 7 illustrates a schematic diagram of a server for classifying medical images according to an embodiment of the present application
  • FIG. 8 illustrates a schematic diagram of a server for training a deep learning network model according to an embodiment of the present application
  • Figure 9 illustrates an example system that includes example computing devices representing one or more systems and/or devices that can implement the various technologies described herein.
  • Artificial intelligence technology is a comprehensive discipline, covering a wide range of fields, including both hardware-level technology and software-level technology.
  • Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, and mechatronics.
  • Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.
  • AI Artificial Intelligence
  • DL deep learning
  • a large number of training images are used to train deep learning neural network models to make It can perform image processing, for example, it can classify medical images by performing quality analysis on medical images, such as filtering out invalid and unqualified pathological images.
  • microscope pathological images mainly relies on manual review, mainly through medical workers marking medical images to obtain abnormal conditions, and then feedback the abnormal conditions to medical institutions or medical experts for confirmation.
  • microscope image quality analysis is mainly performed by pathologists or other medical experts repeatedly and accurately marking to propose abnormalities, perform quality evaluation and eliminate noise information. In some cases, it is through trained technicians to obtain abnormal results and return these noise data to medical institutions or medical experts for confirmation. In this way, not only the efficiency is low, but also low-quality data or invalid data is easily mixed.
  • the existing microscopy image labeling technology has low labeling reliability, high cost, and a long period for obtaining valid data.
  • the embodiments of the present application provide a method for classifying medical images, which can be described by digital image morphology (for example, including color gamut, color gamut, etc.) in a huge number of pathological images mixed with a large number of invalid and unqualified microscope pathological images.
  • digital image morphology for example, including color gamut, color gamut, etc.
  • Saturation, brightness, sharpness, texture, entropy, etc. describe pathological images, filter out unqualified microscope pathological images, and reduce the noise and confusion caused by microscope pathological images for disease diagnosis.
  • Imaging imbalance and overexposure Due to the structural problems of the microscope imaging camera, the captured image has serious contrast imbalance and overexposure, and the original image information is lost.
  • Fig. 1 schematically shows a method 100 for screening microscope images.
  • doctors or other medical experts repeatedly and accurately label 102 the acquired data set 101, and finally eliminate 103 abnormal pictures and evaluate the quality of the pictures to eliminate noise information.
  • it takes a lot of time to filter medical images before using the data set 101.
  • the quality of the first screening is generally not high, it needs to be returned to the medical labeling staff for relabeling.
  • Even if the data does not find any obvious errors for the time being, it is likely to find abnormal results in the later model training process. This not only has a great impact on the accuracy and reliability of the model, it may also require more time and cost to check other data to confirm the validity, accuracy and completeness of the data.
  • Fig. 2 schematically shows an intelligent microscope system 200 to which the image classification and screening module of the present application is applied according to an embodiment of the present application.
  • the intelligent microscope system 200 may include an image acquisition module 201, an image classification screening module 202, and an image analysis module 203.
  • the image acquisition module 201 acquires a medical image data set.
  • the medical image data set is passed through the image screening module 202 according to the embodiment of the present application to perform abnormality detection on the medical images, so as to screen out abnormal medical images.
  • Abnormal medical images include irrelevant tissues, out-of-focus and blurred images, images with invalid white balance, etc. These abnormal medical images cannot be used for disease detection and discovery, so they need to be excluded from the medical image data set to obtain normal Medical image.
  • the normal medical images obtained after screening are sent to the image analysis module 203 to further analyze the microscope pathological images, so as to detect and discover diseases based on these normal medical images.
  • the intelligent microscope system 200 described above with reference to FIG. 2 is mainly used to obtain microscope pathological image data marked by medical annotators. It can be used not only for data screening of microscope TNM (tumor, node, metastasis) staging projects, but also for medicine
  • the pathology center archives confirms and organizes the microscope pathological images.
  • the quality of each image in the medical image data set needs to be analyzed to extract The feature information of each image, according to the extracted feature information, remove irrelevant images in the medical image data set, such as cat and dog images, natural images and other non-pathological images; then, the abnormal pathological image of the microscope is performed through the image classification and screening equipment Detection and classification, you can get irrelevant tissues, out-of-focus and blurred images, cell overlapping wrinkles, images with invalid white balance, normal medical images, and irrelevant images, and filter out abnormal images, that is, irrelevant tissues and lens out-of-focus Blurred images, images with invalid white balance, etc. cannot be used for disease detection and medical images; finally, the screened effective and credible normal medical images are sent to the microscope pathology image analysis system for analysis, according to These normal medical images detect and discover diseases.
  • Another application scenario is to organize and archive microscope images in the pathology center.
  • Using the image classification and screening device according to the application to help or replace doctors in classifying various types of pathological microscope images it is possible to obtain irrelevant tissues, images with out-of-focus blur, cell overlapping wrinkles, images with invalid white balance, and normal medical images.
  • archiving and sorting out the classified medical images in these major categories can not only help doctors reduce workload and improve efficiency, but also help detect the accuracy of labeling results.
  • FIG. 3 schematically shows a schematic diagram of a method 300 for training and testing a model for abnormal detection and classification of medical images according to an embodiment of the present application.
  • the solid line in Figure 3 represents training data, and the dashed line represents test data.
  • the training and testing method 300 establishes a data processing flow based on digital image processing and deep learning classification algorithms to distinguish images of irrelevant tissues, camera out of focus, cell overlapping wrinkles, and white balance failures mixed in the medical image data set , Eliminate, and archive and store these data.
  • the medical image data set is the aforementioned medical image data set, and includes at least one medical image. In some embodiments, images of different diseases can also be classified, and the image data can be archived for different diseases. It mainly includes the following steps:
  • the decoding and verification module 302 decodes and verifies the data set
  • the quality analysis module 303 performs one or more of color gamut analysis, saturation analysis, brightness analysis, definition analysis, texture analysis, and information entropy analysis on the digital image;
  • the interpolation and normalization processing module 305 performs interpolation and normalization on each medical image in the medical image data set
  • FIG. 4a shows a schematic diagram of the original data set 301 and the decoding and file verification module 302.
  • the decoding and file verification module 302 includes a file decoding module 3021 and a file verification module 3022. Regardless of whether it is training data or test data, it is necessary to verify the validity of the data itself. The reason is that the data set is mixed with the damage caused by the storage problem of the image itself. Therefore, after acquiring the data set of the academic image, the file decoding module 3021 decodes the image into a digital image matrix for subsequent image processing. The file verification module 3022 performs file verification on the image and confirms that the file can be parsed and is not damaged. The two verifications of file verification and decoding ensure the validity of input data in subsequent processing, avoid irrelevant data from being mixed into the overall system, and are very important to the overall scheme of anomaly detection and classification according to this application.
  • FIG. 4b shows a schematic diagram of the quality analysis module 303.
  • Quality analysis mainly includes color gamut analysis, saturation analysis, brightness analysis, sharpness analysis, texture analysis and entropy analysis of digital image matrix.
  • FIG. 4b shows a schematic diagram of the quality analysis module 303.
  • Quality analysis mainly includes color gamut analysis, saturation analysis, brightness analysis, sharpness analysis, texture analysis and entropy analysis of digital image matrix.
  • FIG. 4b shows a schematic diagram of the quality analysis module 303.
  • Quality analysis mainly includes color gamut analysis, saturation analysis, brightness analysis, sharpness analysis, texture analysis and entropy analysis of digital image matrix.
  • FIG. 4b shows a schematic diagram of the quality analysis module 303.
  • Quality analysis mainly includes color gamut analysis, saturation analysis, brightness analysis, sharpness analysis, texture analysis and entropy analysis of digital image matrix.
  • FIG. 4b shows a schematic diagram of the quality analysis module 303.
  • Quality analysis mainly includes color gamut analysis, saturation analysis, brightness analysis, sharpness analysis
  • the color gamut-saturation-brightness analysis refers to the conversion of the acquired image matrix of the RGB domain to the HSV space, and the conversion formula is as follows:
  • (r, g, b) is the red, green and blue coordinates of a point in the image, their values are real numbers between 0 and 1, max is the maximum value among r, g, and b, and min is r, g, and The minimum value in b.
  • the mean value and variance of the color gamut value h matrix, the saturation value s matrix and the brightness value matrix v are respectively calculated as the feature information expression of color gamut-saturation-brightness.
  • HSL space can also be used instead of HSV space, where H represents hue, S represents saturation, and L represents lightness.
  • the digital image matrix value is set to a, and the convolution value b of a and a 5 ⁇ 5 Gaussian convolution kernel is calculated. Then, calculate the minimum mean square error value MSE of a and b, and calculate the peak signal-to-noise ratio PSNR of the MSE.
  • n is the color level of the medical image, which may be a preset positive integer, for example, the preset value may be a positive integer 8, and PSNR is selected as the sharpness index of the image. Its calculation formula is as formula (4):
  • detection operators such as Sobel or convolution kernel are used to extract and calculate the edges of gray images.
  • operators such as Isotropic Sobel, Roberts, Prewit, and Laplacian can also be used for processing.
  • the entropy analysis according to the length of the medical image, the width of the medical image, and the gray value of the center in the sliding window and the gray average value of the gray value in the sliding window except the center pixel in the medical image The number of occurrences and the probability are used to calculate the entropy value.
  • FIG. 4c shows a schematic diagram of the interpolation and normalization processing module 305.
  • the system in this application uses a size of 299 ⁇ 299 (RGB). Then, the image matrix is linearly scaled to the range of [-1, 1].
  • methods such as neighbor interpolation and spline interpolation may also be used for interpolation calculation, and alternative solutions may use methods such as normal normalization for data normalization.
  • FIG. 4d shows a schematic diagram of the deep learning network 308 to be trained.
  • the deep learning network adopts the InceptionV3 structure in deep learning, and optimizes the model through a stochastic gradient descent algorithm (SGD). Finally, the model is evaluated, and the model results that meet the indicators are selected and saved.
  • SGD stochastic gradient descent algorithm
  • Figure 4e shows a schematic diagram of loading a trained deep learning network model. After the training is confirmed, the model is loaded, and the image data that needs to be detected for abnormal classification is inferred. Using the judgment accuracy of the trained model that meets the requirements of the loss function, the corresponding image is filtered and judged, so as to realize the abnormal detection and classification of the microscope case image. Specifically, (1) load the selected model, (2) input the processed image data set, (3) output the result and compare the result to generate the corresponding result label, and (4) save according to the label and archive the data.
  • model training and judgment in Figures 4d and 4e a variety of model structures based on convolutional neural networks can be used for classification.
  • the essence is that neural networks perform feature extraction, feature fusion and Feature judgment belongs to the same method.
  • traditional image processing algorithms such as SIFT and HOG can also be used for feature extraction, and classifiers such as svm, mlp, and adaboost are used for classification processing, which essentially belong to the same machine learning classification algorithm as this application .
  • FIG. 5a is a system architecture diagram to which the method for classifying medical images provided by an embodiment of the present application is applicable.
  • the server 51 establishes a connection with the user terminal 53 through the network 52.
  • the server 51 is a back-end server that classifies medical images.
  • the server 51 and the user terminal 53 provide services for the user together. For example, after the server 51 classifies the medical image, it sends the classification result to the user terminal 53 for use by the user.
  • the user may be a medical person, or for example
  • the server 51 can also train a deep learning network for abnormality detection and classification of medical images.
  • the server 51 may be a single server or a cluster server composed of multiple servers.
  • the network 52 may include a wired network and a wireless network. As shown in Figure 5a, on the side of the access network, the user terminal 53 can be connected to the network 52 in a wireless or wired manner; on the core network side, the server 51 is generally connected to the network 52 in a wired manner. . Of course, the above-mentioned server 11 may also be connected to the network 52 in a wireless manner.
  • the aforementioned user terminal 53 may refer to a smart device with data calculation and processing functions. For example, it can display and analyze information such as classification results provided by the server, including but not limited to smartphones (with communication modules installed), handheld computers, and tablet computers. Wait.
  • An operating system is installed on the user terminal 53, including but not limited to: Android operating system, Symbian operating system, Windows mobile operating system, Apple iPhone OS operating system, and so on.
  • Fig. 5b schematically shows a flowchart of a method 500 for classifying medical images according to an embodiment of the present application.
  • the method for classifying medical images is executed by a computing device, and the computing device may be the server 51 in FIG. 5a or the user terminal 53 in FIG. 5a, as shown in FIG. 5b, the method includes the following steps:
  • step 501 first obtain a data set of medical images.
  • the medical image data set includes at least one medical image
  • the acquired medical image data set is file checked and decoded. Through decoding, at least one medical image in the data set of medical images is converted into a digital image matrix.
  • step 502 quality analysis is performed on the data set of the medical image to extract feature information in the medical image.
  • the process of performing quality analysis on the data set of the medical image to extract feature information of the medical image performing color gamut-saturation-brightness analysis on the data set of the medical image , At least one of sharpness analysis, texture analysis, and entropy analysis to obtain feature information of the medical image.
  • the feature information of the medical image includes feature information of color gamut, feature information of saturation, feature information of brightness, At least one of sharpness index, gray edge, and entropy value.
  • the quality analysis specifically includes color gamut-saturation-brightness analysis, sharpness analysis, texture analysis, and entropy value analysis.
  • difference and normalization processing is performed on the medical image data set.
  • the size of images of different sizes can be unified to facilitate the training of the network.
  • all images are unified in dimension, which is convenient for image measurement and calculation.
  • the function of the interpolation and normalization module is mainly to dimension the image. Specifically, first, interpolation is performed on the R, G, and B channels of the image respectively. This application uses bilinear interpolation to unify the images to the same size.
  • the system in this application uses a size of 299 ⁇ 299 (RGB). Then, the image matrix is linearly scaled to the range of [-1, 1].
  • methods such as neighbor interpolation and spline interpolation may also be used for interpolation calculation, and alternative solutions may use methods such as normal normalization for data normalization.
  • a pre-trained deep learning network is used to classify the medical image to obtain a classification result.
  • irrelevant images in the data set of the medical image are removed, and the irrelevant images include non-medical images; using the method for abnormal detection and classification of the medical image
  • the deep learning network of ” classifies the data set of the medical image after removing the irrelevant image.
  • the classified deep learning network classifies the data set of the medical image after removing the irrelevant images, and the classification results obtained include normal tissue, irrelevant tissue, lens out of focus and white balance failure categories, which can improve the deep learning The accuracy of network classification.
  • the deep learning network used for abnormal detection and classification of the medical images classifies the medical image data set containing these small parts of irrelevant images, and the classification results obtained include normal tissues, irrelevant tissues, lens out of focus and whiteness. In addition to these categories of balance failure, a category that contains irrelevant images can also be obtained.
  • the medical image is determined to be the irrelevant image and removed from the medical image data set.
  • the medical image is determined to be an irrelevant image.
  • the medical image is determined to be an irrelevant image and removed from the data set of the medical image; for another example, If there is a medical image whose sharpness index is smaller than the threshold value corresponding to the sharpness index and the entropy value of the medical image is smaller than the threshold value corresponding to the entropy value, the medical image is determined to be an irrelevant image, and the medical image Removed from the data set.
  • the classification result includes one or more of normal tissue, irrelevant tissue, lens out of focus, white balance failure, and irrelevant image categories.
  • the medical image can be classified, filed and sorted according to the obtained classification result.
  • normal medical images in the classification result can be used for disease detection and discovery.
  • This application provides a complete set of microscope pathological image processing and feature extraction methods, so that invalid and unqualified pathological images can be filtered out. Therefore, this method purifies microscopic pathological images, making image-based disease diagnosis more accurate. On the other hand, by describing various morphological attributes of the image, it is convenient to archive and organize the microscope pathological images according to their quality.
  • Fig. 6 schematically shows a flowchart of a method 600 for training a deep learning network model according to an embodiment of the present application.
  • the method for training the model for abnormal detection and classification of medical images is executed by a computing device.
  • the computing device may be the server 51 in FIG. 5a or the user terminal 53 in FIG. 5a.
  • the method includes The following steps:
  • step 601 a data set of original medical images and a corresponding set of original image annotation information are obtained.
  • the annotation information in the original image annotation information set may be provided by a doctor or a medical annotator, including labels for whether the medical image is normal tissue, irrelevant tissue, lens out of focus, and white balance failure, for example, Irrelevant tissue, lens out of focus, blurry white balance, low-information images, overlapping cell folds, etc.
  • file check and decoding are performed on the acquired medical image data set. Through decoding, the medical image is converted into a digital image matrix.
  • step 602 quality analysis is performed on the data set of the original medical image, and feature information of the original medical image is extracted.
  • the quality analysis may specifically include color gamut-saturation-brightness analysis, sharpness analysis, texture analysis, entropy analysis, and the like. See the description of quality analysis in Figure 3 for details.
  • difference and normalization processing is performed on the medical image data set.
  • the size of images of different sizes can be unified to facilitate the training of the network.
  • all images are unified in dimension, which is convenient for image measurement and calculation.
  • the function of the interpolation and normalization module is mainly to dimension the image. Specifically, first, interpolation is performed on the R, G, and B channels of the image respectively.
  • This application uses bilinear interpolation to unify the images to the same size.
  • the size of this application system is 299 ⁇ 299 (RGB).
  • the image matrix is linearly scaled to the range of [-1, 1].
  • methods such as neighbor interpolation and spline interpolation may also be used for interpolation calculation, and alternative solutions may use methods such as normal normalization for data normalization.
  • step 603 the deep learning network is trained based on the feature information in the extracted original medical image data set and the original image annotation information set to obtain a deep learning network model.
  • Fig. 7 schematically shows a schematic diagram of an apparatus 700 for classifying medical images according to an embodiment of the present application.
  • the device 700 includes an acquisition module 701, a quality analysis module 702, and a classification module 703.
  • the obtaining module 701 is used to obtain the data set of the medical image.
  • the quality analysis module 702 is configured to perform quality analysis on the data set of the medical image, and extract feature information of the medical image.
  • the classification module 703 is configured to use a pre-trained deep learning network to classify the medical image based on the extracted feature information to obtain a classification result.
  • FIG. 8 schematically shows a schematic diagram of a training device 800 for a model for abnormal detection and classification of medical images according to an embodiment of the present application.
  • the device 800 includes an acquisition module 801, a quality analysis module 802, and a training module 803.
  • the obtaining module 801 is used to obtain a data set of original medical images and a corresponding set of original image annotation information.
  • the quality analysis module 802 is configured to perform quality analysis on the data set of the original medical image, and extract feature information of the original medical image.
  • the training module 803 is configured to train the deep learning network for abnormal detection and classification of the medical image based on the feature information and the original image annotation information set in the extracted original medical image data set, and obtain the trained medical image Deep learning network model for anomaly detection and classification.
  • Figure 9 illustrates an example system 900 that includes an example computing device 910 that represents one or more systems and/or devices that can implement the various technologies described herein.
  • the computing device 910 may be, for example, a server of a service provider, a device associated with a client (eg, a client device), a system on a chip, and/or any other suitable computing device or computing system.
  • the server 700 for classifying medical images in FIG. 7 or the server 800 for training a deep learning network model in FIG. 8 above may take the form of a computing device 910.
  • the server 700 for classifying medical images or the server 800 for training a model of FIG. 8 may be implemented as a computer program in the form of a medical image classification application 916.
  • the example computing device 910 as illustrated includes a processing system 911, one or more computer-readable media 912, and one or more I/O interfaces 913 that are communicatively coupled to each other.
  • the computing device 910 may also include a system bus or other data and command transfer system that couples various components to each other.
  • the system bus may include any one or a combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or a processor using any of various bus architectures. Local bus.
  • Various other examples are also contemplated, such as control and data lines.
  • the processing system 911 represents a function of performing one or more operations using hardware. Therefore, the processing system 911 is illustrated as including hardware elements 914 that can be configured as processors, functional blocks, and the like. This may include implementation in hardware as an application specific integrated circuit or other logic devices formed using one or more semiconductors.
  • the hardware element 914 is not limited by the material it is formed of or the processing mechanism adopted therein.
  • the processor may be composed of semiconductor(s) and/or transistors (e.g., electronic integrated circuits (ICs)).
  • processor-executable instructions may be electronically executable instructions.
  • the computer readable medium 912 is illustrated as including a memory/storage device 915.
  • the memory/storage device 915 represents memory/storage capacity associated with one or more computer-readable media.
  • the memory/storage device 915 may include volatile media (such as random access memory (RAM)) and/or nonvolatile media (such as read only memory (ROM), flash memory, optical disks, magnetic disks, etc.).
  • RAM random access memory
  • ROM read only memory
  • flash memory optical disks
  • magnetic disks etc.
  • the memory/storage device 915 may include fixed media (for example, RAM, ROM, fixed hard disk drive, etc.) and removable media (for example, flash memory, removable hard disk drive, optical disk, etc.).
  • the computer-readable medium 912 may be configured in various other ways as described further below.
  • the one or more I/O interfaces 913 represent functions that allow the user to input commands and information to the computing device 910 and optionally also allow various input/output devices to be used to present information to the user and/or other components or devices.
  • input devices include keyboards, cursor control devices (e.g., mice), microphones (e.g., for voice input), scanners, touch functions (e.g., capacitive or other sensors configured to detect physical touch), cameras ( For example, visible or invisible wavelengths (such as infrared frequencies) can be used to detect motions that do not involve touch as gestures) and so on.
  • Examples of output devices include display devices (for example, monitors or projectors), speakers, printers, network cards, haptic response devices, and so on. Therefore, the computing device 910 may be configured in various ways as described further below to support user interaction.
  • the computing device 910 also includes a medical image classification application 916.
  • the medical image classification application 916 may be, for example, a software instance of the server 700 for classifying medical images described in relation to FIGS. 7 and 8 or the server 800 for training a model in FIG. The other elements of the combined to implement the technology described herein.
  • modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
  • module means that these technologies can be implemented on various computing platforms with various processors.
  • the computer-readable media may include various media that can be accessed by the computing device 910.
  • the computer-readable medium may include “computer-readable storage medium” and “computer-readable signal medium”.
  • Computer-readable storage medium refers to a medium and/or device capable of permanently storing information, and/or a tangible storage device. Therefore, computer-readable storage media refers to non-signal bearing media.
  • Computer-readable storage media include such as volatile and nonvolatile, removable and non-removable media and/or suitable for storing information (such as computer-readable instructions, data structures, program modules, logic elements/circuits or other data) ) Hardware such as storage devices implemented by methods or technologies.
  • Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, hard disks, cassette tapes, magnetic tapes, disk storage Apparatus or other magnetic storage device, or other storage device, tangible medium, or article suitable for storing desired information and accessible by a computer.
  • Computer-readable signal medium refers to a signal-bearing medium configured to send instructions to the hardware of the computing device 910, such as via a network.
  • the signal medium can typically embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave, a data signal, or other transmission mechanism.
  • the signal medium also includes any information transmission medium.
  • modulated data signal refers to a signal that encodes information in the signal in such a way to set or change one or more of its characteristics.
  • communication media include wired media such as a wired network or direct connection, and wireless media such as acoustic, RF, infrared, and other wireless media.
  • the hardware element 914 and the computer readable medium 912 represent instructions, modules, programmable device logic and/or fixed device logic implemented in hardware, which in some embodiments can be used to implement the technology described herein At least some aspects.
  • the hardware elements may include integrated circuits or system-on-chips, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon or components of other hardware devices.
  • ASICs application-specific integrated circuits
  • FPGAs field programmable gate arrays
  • CPLDs complex programmable logic devices
  • a hardware element can serve as a processing device that executes program tasks defined by instructions, modules, and/or logic embodied by the hardware element, and a hardware device that stores instructions for execution, for example, as previously described Computer-readable storage medium.
  • software, hardware or program modules and other program modules may be implemented as one or more instructions and/or logic embodied by one or more hardware elements 914 on some form of computer-readable storage medium.
  • the computing device 910 may be configured to implement specific instructions and/or functions corresponding to software and/or hardware modules. Therefore, for example, by using the computer-readable storage medium and/or the hardware element 914 of the processing system, the module may be implemented at least partially in hardware as a module executable by the computing device 910 as software.
  • the instructions and/or functions may be executable/operable by one or more articles of manufacture (eg, one or more computing devices 910 and/or processing system 911) to implement the techniques, modules, and examples described herein.
  • the computing device 910 may adopt various different configurations.
  • the computing device 910 may be implemented as a computer-type device including a personal computer, a desktop computer, a multi-screen computer, a laptop computer, a netbook, and the like.
  • the computing device 910 may also be implemented as a mobile device type device including mobile devices such as a mobile phone, a portable music player, a portable game device, a tablet computer, a multi-screen computer, and the like.
  • the computing device 910 may also be implemented as a television-type device, which includes a device with or connected to a generally larger screen in a casual viewing environment. These devices include televisions, set-top boxes, game consoles, etc.
  • the techniques described herein can be supported by these various configurations of the computing device 910 and are not limited to specific examples of the techniques described herein. Functions can also be implemented in whole or in part on the "cloud" 920 using a distributed system, such as through the platform 922 described below.
  • Cloud 920 includes and/or represents a platform 922 for resources 924.
  • the platform 922 abstracts the underlying functions of the hardware (for example, server) and software resources of the cloud 920.
  • the resources 924 may include applications and/or data that can be used when performing computer processing on a server remote from the computing device 910.
  • Resources 924 may also include services provided through the Internet and/or through subscriber networks such as cellular or Wi-Fi networks.
  • the platform 922 can abstract resources and functions to connect the computing device 910 with other computing devices.
  • the platform 922 can also be used to abstract the classification of resources to provide a corresponding level of classification of the requirements encountered for the resources 924 implemented via the platform 922. Therefore, in the interconnected device embodiment, the implementation of the functions described herein may be distributed throughout the system 900. For example, the functions may be partially implemented on the computing device 910 and through a platform 922 that abstracts the functions of the cloud 920.
  • each functional module can be implemented in a single module, implemented in multiple modules, or implemented as a part of other functional modules without departing from this application.
  • functionality described as being performed by a single module may be performed by multiple different modules. Therefore, references to specific functional modules are only regarded as references to appropriate modules for providing the described functionality, rather than indicating a strict logical or physical structure or organization. Therefore, the present application may be implemented in a single module, or may be physically and functionally distributed between different modules and circuits.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Public Health (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

一种医学图像分类方法、模型训练方法和服务器。该医学图像的分类方法，包括：获取医学图像的数据集合（501）；对医学图像的数据集合进行质量分析，提取特征信息（502）；基于特征信息，利用经预先训练的神经网络对医学图像进行分类（503），得到分类结果。

Description

一种医学图像的分类方法、模型训练方法、计算设备以及存储介质
本申请要求于2019年6月21日提交中国专利局、申请号为201910543018.9、名称为“一种医学图像的分类方法、模型训练方法和服务器”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及医学图像的分类方法、模型训练方法和服务器，具体涉及显微镜图像的分类方法、模型训练方法、计算设备以及存储介质。
背景
病理检验分析是临床医学最终判断标准的重要参考，其主要过程是从病体组织中获得相应组织细胞，通过在显微镜下观察分析制作的病理组织切片来确认病情。随着医疗水平的不断提高，显微镜病理图像数据呈现井喷式爆发，数字化显微镜病理图像数量呈现几何式增长。
技术内容
有鉴于此，本申请提出了一种医学图像的分类方法、模型训练方法、计算设备以及存储介质。
本申请实施例提供了一种医学图像的分类方法，由计算设备执行。该方法包括：获取医学图像的数据集合；对医学图像的数据集合进行质量分析，提取医学图像的特征信息；基于提取的特征信息，利用预先训练的用于对医学图像进行异常检测分类的深度学习网络对医学图像进行分类，得到分类结果。
本申请实施例还提供了一种对医学图像进行异常检测分类的模型的训练方法，由计算设备执行，该方法包括：获取原始医学图像的数据集合以及相应的原始图像标注信息集合；对原始医学图像的数据集合进行质量分析，提取原始医学图像的特征信息；基于所述提取的原始医学图像数据集合中的特征信息和原始图像标注信息集合训练对医学图像进行异常检测分类的深度学习网络，得到训练后 的对医学图像进行异常检测分类的深度学习网络模型。
本申请实施例还提供了一种医学图像的分类装置。该装置包括获取模块、质量分析模块和分类模块。获取模块被配置为获取医学图像的数据集合。质量分析模块被配置为对医学图像的数据集合进行质量分析，提取医学图像的特征信息。分类模块被配置为基于提取的特征信息，利用预先训练的用于对所述医学图像进行异常检测分类的深度学习网络对医学图像进行分类，得到分类结果。
本申请实施例还提供了一种对医学图像进行异常检测分类的模型的训练装置。该训练装置包括获取模块、质量分析模块和训练模块。获取模块用于获取原始医学图像的数据集合以及相应原始图像标注信息集合。质量分析模块用于对原始医学图像的数据集合进行质量分析，提取原始医学图像的特征信息。训练模块用于基于提取的原始医学图像数据集合中的特征信息和原始图像标注信息集合训练对医学图像进行异常检测分类的深度学习网络，得到训练后的对医学图像进行异常检测分类的深度学习网络模型。
本申请实施例还提供了一种计算设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现本申请实施例所述的方法。本申请实施例还提供了一种医学图像分类系统，其中医学图像分类系统包括医学图像获取设备和医学图像处理设备；医学图像获取设备用于扫描医学图像，并向医学图像处理设备发送医学图像；医学图像处理设备用于执行如本申请实施例所述的任一种的方法。
本申请实施例还提供了一种非易失性计算机可读存储介质，其存储有可由计算机设备执行的计算机程序，当所述程序在计算机设备上运行时，使得所述计算机设备执行本申请实施例所述的方法。
附图说明
现在将更详细地并且参考附图来描述本申请的实施例，其中：
图1图示了根据相关技术对显微镜图像进行筛选的方法；
图2图示了根据本申请的实施例的图像分类筛选模块应用于其中的智能显微镜系统；
图3图示了根据本申请实施例的对模型进行训练的方法的示意图；
图4a图示了原始数据集和解码与文件校验模块的示意图；
图4b图示了质量分析模块的示意图；
图4c图示了插值与归一化处理模块的示意图；
图4d图示了待训练的深度学习网络的示意图；
图4e图示了加载经训练的深度学习网络模型的示意图；
图5a图示了根据本申请实施例提供的医学图像进行分类的方法所适用系统架构图；
图5b图示了根据本申请实施例对图像进行分类的方法的流程图；
图6图示了根据本申请实施例对深度学习网络模型进行训练的方法的流程图；
图7图示了根据本申请实施例对医学图像进行分类的服务器的示意图；
图8图示了根据本申请实施例对深度学习网络模型进行训练的服务器的示意图；
图9图示了一个示例系统，其包括代表可以实现本文描述的各种技术的一个或多个系统和/或设备的示例计算设备。
实施方式
下面的说明提供用于充分理解和实施本申请的各种实施例的特定细节。本领域的技术人员应当理解，本申请的技术方案可以在没有这些细节中的一些的情况下被实施。在某些情况下，并没有示出或详细描述一些熟知的结构和功能，以避免不必要地使对本申请的实施例的描述模糊不清。在本申请中使用的术语以其最宽泛的合理方式来理解，即使其是结合本申请的特定实施例被使用的。
人工智能技术是一门综合学科，涉及领域广泛，既有硬件层面的技术也有软件层面的技术。人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互系统、机电一体化等技术。人工智能软件技术主要包括计算机视觉技术、语音处理技术、自然语言处理技术以及机器学习/深度学习等几大方向。
目前，深度学习是机器学习的技术和研究领域之一，通过建立具有阶层结构 的人工神经网络，在计算机系统中实现人工智能(Artificial Intelligence，AI)。
由于深度学习(Deep Learning，DL)在视觉领域的成功应用，研究者也将其引入到图像处理领域中，例如医学图像的处理领域，通过大量的训练图像来训练深度学习神经网络模型，以使其能够进行图像处理，例如能够通过对医学图像进行质量分析，来对医学图像进行分类，例如过滤掉无效、不合格的病理图像。
目前针对显微镜病理图像的分析主要依赖人工审核，主要通过医务工作者对医学图像进行标注获取异常情况，再将异常情况反馈给医疗机构或医学专家确认。
且显微镜图像质量分析主要是通过病理科医生或其他医学专家反复精确标注以提出异常，进行质量评估和消除噪声信息。在一些情况下，是通过经培训的技术人员获取异常结果表现，并将这些噪声数据返回医疗机构或医疗专家确认。这样，不仅效率低下，而且极易混入质量低的数据或无效数据。现有显微镜图像标注技术的标注可靠性低、成本高且获取有效数据的周期长。
因此，本申请实施例提供了一种医学图像的分类方法，可以在混杂了大量无效和不合格的显微镜病理图像的数量巨大的病理图像中，通过数字图像形态学描述(例如，包括色域、饱和度、亮度、清晰度、纹理、熵值等)对病理图像进行描述，过滤掉不合格的显微镜病理图像，减少显微镜病理图像对疾病诊断所带来的噪声混淆。
在描述本申请实施例提供的医学图像的分类方法之前，先对本申请的实施例中涉及的部分用语进行说明，以便于本领域技术人员理解：
失焦：显微镜物镜在对病理切片对焦时，或相机未能达到或超过合理物距，导致显微镜图像成像模糊。
切片细胞空洞、重叠、褶皱：在切片的制作过程中，由于切片操作人员不同，导致切片细胞制作出现空洞、重叠、褶皱；这一部分细胞不会被检验使用。
成像失衡、过曝：显微镜成像相机由于自身结构问题，导致采集图像出现严重对比度失衡、过曝等情况，从而失去原始图像信息。
图1示意性示出了根据一种对显微镜图像进行筛选的方法100。方法100由医生或其他医学专家对获取的数据集101进行反复精确标注102，最后剔除103异常图片和评估图片质量，以消除噪声信息。在许多情况下，使用数据集101之前需要消耗大量时间对医学图像进行筛选。如果在第一次筛选普遍质量不高的情 况下，还需要返回给医学标注人员重新标注。即便数据暂时没有发现任何明显错误，也很可能在后期模型训练过程中发现异常结果。这不仅对模型的准确性和可靠性影响甚大，还可能需要更多的时间成本对其他数据进行核查检验才能确认数据的有效性、准确性和完整性。
图2示意性示出了根据本申请的实施例本申请的图像分类筛选模块应用于其中的智能显微镜系统200。智能显微镜系统200可以包括图像采集模块201、图像分类筛选模块202和图像分析模块203。首先，图像采集模块201获取医学图像数据集。将该医学图像数据集通过根据本申请的实施例的图像筛选模块202对医学图像进行异常检测，以便筛除异常的医学图像。异常的医学图像包括无关组织、镜头失焦模糊的图像、白平衡失效的图像等，这些异常的医学图像无法用于对疾病的检测和发现，因此需要从医学图像数据集中剔除，进而得到正常的医学图像。最后将经过筛选得到的正常的医学图像发送给图像分析模块203进一步对显微镜病理图像进行分析，以根据这些正常的医学图像对疾病进行检测和发现。
如以上参照图2描述的智能显微镜系统200主要应用于获取医学标注人员所标注的显微镜病理图像数据，不仅可以用于进行显微镜TNM(tumor，node，metastasis)分期项目的数据筛选，还用于医学病理中心对显微镜病理图像进行归档确认和整理。
具体地，针对应用于智能显微镜TNM分期项目的应用场景，在将原始的医学图像数据集应用于显微镜病理图像分析系统之前，首先，需要对医学图像数据集中的每个图像进行质量分析，以提取每个图像的特征信息，根据提取的特征信息，去除所述医学图像数据集中的无关图像，例如猫狗的图像、自然图像等非病理图像；然后，通过图像分类筛选设备进行显微镜病理图像的异常检测分类，可以得到无关组织、镜头失焦模糊的图像、细胞重叠褶皱、白平衡失效的图像、正常的医学图像以及无关图像等几大类别，筛选除去异常图像，也即无关组织、镜头失焦模糊的图像、白平衡失效的图像等无法用于对疾病的检测和发现的医学图像；最后，将筛选后的有效可信的正常的医学图像送入显微镜病理图像分析系统中进行分析，以根据这些正常的医学图像对疾病进行检测和发现。
另一种应用场景是应用于病理中心对显微镜图像进行整理和归档。采用根据本申请的图像分类筛选设备帮助或代替医生对各种类型的病理显微镜图像进行 分类，可以得到无关组织、镜头失焦模糊的图像、细胞重叠褶皱、白平衡失效的图像、正常的医学图像以及无关图像等几大类别，对分类得到这几大类别的医学图像进行归档和整理，不仅可以帮助医生减少工作量提高效率，还可以帮助检测标注结果的准确性。
图3示意性示出了根据本申请一个实施例中对医学图像进行异常检测分类的模型的训练和测试的方法300的示意图。图3中的实线代表训练数据，虚线代表测试数据。该训练和测试方法300通过建立基于数字图像处理、深度学习分类算法的数据处理流程，将混杂在医学图像的数据集合中的无关组织、相机失焦、细胞重叠褶皱、白平衡失效等图像进行区分、剔除，并对这些数据归档存储。该医学图像的数据集合为上述医学图像数据集，包括至少一张医学图像。在一些实施例中，还可以针对不同的疾病的图像进行分类，并将图像数据针对不同疾病进行归档。主要包括以下步骤：
(1)解码与校验模块302对数据集合进行解码和校验；
(2)质量分析模块303对数字图像进行色域分析、饱和度分析、亮度分析、清晰度分析、纹理分析、信息熵值分析中的一个或多个；
(3)插值与归一化处理模块305对所述医学图像的数据集合中的各医学图像进行插值和归一化；
(4)建立深度学习网络模型306，利用标签信息304所标注的数据以及质量分析模块303中所提取的信息对所建立的深度学习网络模型进行训练，并保存模型。
(5)将训练好的模型306加载到深度学习网络模型308中，并可以对需要检测的测试数据进行筛选和分类。
图4a示出了原始数据集301和解码与文件校验模块302的示意图。解码与文件校验模块302包括文件解码模块3021和文件校验模块3022。无论是训练数据还是测试数据均需要对数据本身进行合法性校验，原因在于数据集中混杂着图像本身由于存储问题等所造成的损坏。因此，在获取所述学图像的数据集合之后，文件解码模块3021将图像解码为数字图像矩阵，用于后续图像处理。文件校验模块3022对图像进行文件校验，确认文件可解析无损坏。文件校验和解码这两项校验保证了输入数据在后续处理中的有效性，避免了无关数据混入整体系统，对根据本申请的异常检测分类的整体方案非常重要。
图4b示出了质量分析模块303的示意图。质量分析主要包括数字图像矩阵的色域分析、饱和度分析、亮度分析、清晰度分析、纹理分析和熵值分析。如图4b所示，在获得经过解码3021和校验2022后的数字图像矩阵之后，需要对图像的各类属性进行分析。好处在于：一方面数据帮助模型分类训练，另一方面可以对大部分的无关图像直接滤除整理，从而使后续模型判断更为精确准。
具体地，色域-饱和度-亮度分析指的是将获取的RGB域的图像矩阵转换到HSV空间，其转换公式如下：
Figure PCTCN2020096797-appb-000001
Figure PCTCN2020096797-appb-000002
v＝max       (3)
其中(r,g,b)是图像一个点的红、绿和蓝坐标，它们的值在0到1之间的实数，max为r、g和b中的最大值，min为r、g和b中的最小值。分别计算色域值h矩阵、饱和度值s矩阵和亮度值矩阵v的均值和方差作为色域-饱和度-亮度的特征信息表达。如本领域技术人员所理解的，也可以采用HSL空间来代替HSV空间，其中H表示色相、S表示饱和度、L表示明度。
在清晰度分析中，设数字图像矩阵值为a，计算a与5×5高斯卷积核的卷积值b。然后，计算a与b的最小均方差值MSE，计算MSE的峰值信噪比PSNR。这里，n为医学图像的色阶，可以为预设的正整数，例如该预设值可以为正整数8，选用PSNR作为图像的清晰度指数。其计算公式如式(4):
Figure PCTCN2020096797-appb-000003
如本领域技术人员所理解的，除采用峰值信噪比PSNR的描述方法之外，还可以采用Brenner、Tenengrad、Laplacian、SMD、SMD2等方法进行描述。
在纹理分析中，采用Sobel等检测算子或卷积核，对灰度图像边缘进行提取 计算。如本领域技术人员所理解的，也可以采用Isotropic Sobel、Roberts、Prewit、Laplacian等算子进行处理。
在熵值分析中，根据所述医学图像的长度、所述医学图像的宽度、以及滑动窗口内中心的灰度值和所述滑动窗口内除中心像素外的灰度均值在所述医学图像中出现的次数和概率来计算熵值。
具体的，设W、H为图像的长和宽，(i,j)为一个二元组，i表示某个滑动窗口内中心的灰度值，j为该窗口内除了中心像素的灰度均值。f(i,j)表示(i,j)这个二元组在整个图像中出现的次数，P i,j为(i，j)二元组在所述医学图像中出现的概率，计算公式如下：
Figure PCTCN2020096797-appb-000004
图4c示出了插值与归一化处理模块305的示意图。在对图像质量进行分析的预处理完成之后，为了模型能够取得更好的训练效果，需要对原始图像进行插值和归一化处理。其目的在于，一方面使不同大小的图像尺寸能够统一，以便于对网络进行训练。另一方面，使所有图像在量纲上统一，便于对图像的度量和计算。也就是说，插值与归一化模块的作用主要是对图像去量纲。具体地，首先，分别在图像的R、G、B三个通道上进行插值。本申请采用的是双线性插值，以便将图像统一到相同尺寸。本申请系统采用的是299×299(RGB)的尺寸。然后，将图像矩阵线性缩放到[-1，1]的范围内。如本领域技术人员所理解的，还可以采用邻近插值、样条插值等方法进行插值计算，替代方案可以采用诸如正态归一化的方式进行数据归一化。
图4d示出了待训练的深度学习网络308的示意图。将上述经插值和归一化处理后的数据发送给待训练的深度学习网络模块，并且结合医生或医学标注人员提供的标注信息(例如，为无关组织、镜头失焦模糊、白平衡失效、低信息图像、细胞褶皱重叠等)对图像的异常检测分类网络进行训练，以获取经过训练的深度学习网络模型。在一个实施例中，深度学习网络采用深度学习中的InceptionV3结构，通过随机梯度下降算法(SGD)对模型进行优化。最后对模型进行评估，选择符合指标的模型结果并保存。具体过程如下：
a.建立深度学习深度学习网络InceptionV3结构；
b.设定训练的优化函数(即梯度下降函数SGD)，设定学习率和迭代次数；
c.开始深度学习网络模型的训练，并检测学习率和迭代次数；
d.选取验证集损失函数loss最小值所对应的模型，保存模型以便在测试时使用。
图4e示出了加载经训练的深度学习网络模型的示意图。在确认训练好之后，对模型进行加载，同时对需要进行异常分类检测的图像数据进行推断。利用经训练符合损失函数要求的模型的判断准确性，对相应图像进行过滤判断，从而实现显微镜病例图像的异常检测分类。具体地，(1)加载选定的模型，(2)输入处理好的图像数据集合，(3)输出结果并对照结果生成相应的结果标签，以及(4)按照标签保存，并归档数据。
关于图4d和4e中的模型训练与判断，在可替换实施例中，可以采用多种基于卷积神经网络的模型结构进行分类，其本质都是属于神经网络对图像进行特征提取、特征融合和特征判断，属于同一种方法。如本领域技术人员可以理解的，也可以采用SIFT、HOG等传统图像处理算法进行特征提取，采用诸如svm、mlp、adaboost等分类器进行分类处理，其本质上与本申请同属于机器学习分类算法。
请参见图5a，是本申请实施例提供的对医学图像进行分类的方法所适用的系统架构图。服务器51通过网络52与用户终端53建立连接。
在本申请的一些实例中，服务器51是对医学图像进行分类处理的后台服务器。服务器51与用户终端53一起为用户提供服务，例如，服务器51在对医学图像进行分类处理之后，将分类结果发送到用户终端53以提供给用户使用，该用户可以是医学相关人员，或者又例如服务器51还可以训练对医学图像进行异常检测分类的深度学习网络。其中，服务器51可以是单独的服务器也可以是多个服务器组成的集群服务器。
网络52可以包括有线网络和无线网络。如图5a所示，在接入网一侧，用户终端53可以通过无线的方式或者有线的方式接入到网络52；而在核心网一侧，服务器51一般是通过有线方式连接到网络52的。当然，上述服务器11也可以通过无线方式连接到网络52。
上述用户终端53可以是指具有数据计算处理功能的智能设备，例如可以对服务器提供的分类结果等信息进行展示和分析，包括但不限于(安装有通信模块的)智能手机、掌上电脑、平板电脑等。用户终端53上安装有操作系统，包括 但不限于：Android操作系统、Symbian操作系统、Windows mobile操作系统、以及苹果iPhone OS操作系统等等。
图5b示意性示出了根据本申请一个实施例对医学图像进行分类的方法500的流程图。该对医学图像进行分类的方法由计算设备执行，该计算设备可以是图5a中的服务器51，也可以是图5a中的用户终端53，如图5b所示，该方法包括以下步骤：
在步骤501中，首先获取医学图像的数据集合。
在一个实施例中，所述医学图像的数据集合包括至少一个医学图像，在读取医学数据集合之后，对所获取的医学图像的数据集合进行文件校验和解码。通过解码，将医学图像的数据集合中的至少一个医学图像转换为数字图像矩阵。
在步骤502中，对该医学图像的数据集合进行质量分析，以提取医学图像中的特征信息。
在一个实施例中，在对所述医学图像的数据集合进行质量分析，以提取所述医学图像的特征信息的过程中，执行对所述医学图像的数据集合进行色域-饱和度-亮度分析、清晰度分析、纹理分析和熵值分析中的至少一个，得到所述医学图像的特征信息，所述医学图像的特征信息包括色域的特征信息、饱和度的特征信息、亮度的特征信息、清晰度指数、灰度边缘以及熵值中的至少一个。
在一个实施例中，质量分析具体包括的关于色域-饱和度-亮度的分析、清晰度分析、纹理分析、熵值分析，可以参见图3中关于质量分析的描述。在一个实施例中，在上述质量分析之后，对医学图像数据集合进行差值和归一化处理。一方面使不同大小的图像尺寸能够统一，以便于对网络进行训练。另一方面，使所有图像在量纲上统一，便于对图像的度量和计算。也就是说，插值与归一化模块的作用主要是对图像去量纲。具体地，首先，分别在图像的R、G、B三个通道上进行插值。本申请采用的是双线性插值，以便将图像统一到相同尺寸。本申请系统采用的是299×299(RGB)的尺寸。然后，将图像矩阵线性缩放到[-1，1]的范围内。如本领域技术人员所理解的，还可以采用邻近插值、样条插值等方法进行插值计算，替代方案可以采用诸如正态归一化的方式进行数据归一化。
在步骤503中，基于提取的所述特征信息，利用预先训练的深度学习网络对所述医学图像进行分类，得到分类结果。
在一些实施例中，根据提取的所述特征信息，去除所述医学图像的数据集合中的无关图像，所述无关图像包括非医学图像；利用所述用于对所述医学图像进行异常检测分类的深度学习网络对去除所述无关图像后的所述医学图像的数据集合进行分类。
具体的，根据提取的特征信息，去除所述医学图像的数据集合中的无关图像，例如猫狗图像或者非病理的自然图像等非医学图像，利用所述用于对所述医学图像进行异常检测分类的深度学习网络对去除所述无关图像后的所述医学图像的数据集合进行分类，得到的分类结果包括正常组织、无关组织、镜头失焦和白平衡失效这些类别，这样可以提高该深度学习网络分类的精确度。
在一些实施例中，在去除这些非医学图像后，所述医学图像的数据集合中的大部分的无关图像将会被滤除，但也不能排除还会留存少部分的无关图像，利用所述用于对所述医学图像进行异常检测分类的深度学习网络对包含有这些少部分的无关图像的医学图像的数据集合进行分类，得到的分类结果中除了正常组织、无关组织、镜头失焦和白平衡失效这些类别以外，还可以得到包含无关图像这一类别。
在一些实施例中，如果所述医学图像的数据集合中存在其特征信息中的所述色域的特征信息、所述饱和度的特征信息、所述亮度的特征信息、所述清晰度指数、所述灰度边缘以及所述熵值中至少一个小于对应的阈值的医学图像，则确定所述医学图像为所述无关图像，并从所述医学图像的数据集合中去除。
具体的，如果所述医学图像的数据集合中存在一个医学图像的特征信息中所述色域的特征信息、所述饱和度的特征信息、所述亮度的特征信息、所述清晰度指数、所述灰度边缘以及所述熵值只要存在一个小于对应的阈值，则确定该医学图像为无关图像。
例如，如果存在一个医学图像的饱和度的特征信息小于该饱和度的特征信息对应的阈值，则确定该医学图像为无关图像，并将其从所述医学图像的数据集合中去除；又例如，如果存在一个医学图像的清晰度指数小于所述清晰度指数对应的阈值并且该医学图像的熵值小于所述熵值对应的阈值，则确定该医学图像为无关图像，并从所述医学图像的数据集合中去除。
在一些实施例中，所述分类结果包括正常组织、无关组织、镜头失焦、白平 衡失效以及无关图像的类别中的一个或多个。
在一些实施例中，得到分类结果后，可以根据得到的分类结果，对医学图像进行分类归档和整理。
在一些实施例中，得到分类结果后，可以利用分类结果中的正常的医学图像进行疾病的检测和发现。
本申请提供了一套完整的显微镜病理图像的处理和特征提取方法，使得无效、不合格的病理图像能够被过滤掉。因此，该方法纯化了显微镜病理图像，使基于图像的疾病诊断更加准确。另一方面，通过对图像的各种形态学属性进行描述，方便对显微镜病理图像按照质量进行归档和整理。
图6示意性示出了根据本申请一个实施例对深度学习网络模型进行训练的方法600的流程图。该对医学图像进行异常检测分类的模型的训练方法由计算设备执行，该计算设备可以是图5a中的服务器51，也可以是图5a中的用户终端53，如图6所示，该方法包括以下步骤：
在步骤601中，获取原始医学图像的数据集合以及相应原始图像标注信息集合。
在一个实施例中，原始图像标注信息集合中的标注信息可以是医生或医学标注人员提供的，包括针对所述医学图像是正常组织、无关组织、镜头失焦和白平衡失效的标签，例如为无关组织、镜头失焦模糊、白平衡失效、低信息图像、细胞褶皱重叠等。在一个实施例中，在获取医学数据集合之后，对所获取的医学图像的数据集合进行文件校验和解码。通过解码，将医学图像转换为数字图像矩阵。
在步骤602中，对原始医学图像的数据集合进行质量分析，提取所述原始医学图像的特征信息。
在一个实施例中，质量分析具体可以包括关于色域-饱和度-亮度的分析、清晰度分析、纹理分析、熵值分析等。具体参见图3中关于质量分析的描述。在一个实施例中，在上述质量分析之后，对医学图像数据集合进行差值和归一化处理。一方面使不同大小的图像尺寸能够统一，以便于对网络进行训练。另一方面，使所有图像在量纲上统一，便于对图像的度量和计算。也就是说，插值与归一化模块的作用主要是对图像去量纲。具体地，首先，分别在图像的R、G、B三个通道上进行插值。本申请采用的是双线性插值，以便将图像统一到相同尺寸。本申 请系统采用的是299×299(RGB)的尺寸。然后，将图像矩阵线性缩放到[-1，1]的范围内。如本领域技术人员所理解的，还可以采用邻近插值、样条插值等方法进行插值计算，替代方案可以采用诸如正态归一化的方式进行数据归一化。
在步骤603中，基于提取的原始医学图像数据集合中的所述特征信息和原始图像标注信息集合对所述深度学习网络进行训练，得到深度学习网络模型。
图7示意性示出了根据本申请一个实施例对医学图像进行分类的装置700的示意图。该装置700包括获取模块701、质量分析模块702和分类模块703。获取模块701，用于获取所述医学图像的数据集合。质量分析模块702用于对所述医学图像的数据集合进行质量分析，提取所述医学图像的特征信息。分类模块703用于基于提取的所述特征信息，利用预先训练的深度学习网络对所述医学图像进行分类，得到分类结果。
图8示意性示出了根据本申请一个实施例对医学图像进行异常检测分类的模型的训练装置800的示意图。装置800包括获取模块801、质量分析模块802和训练模块803。获取模块801用于获取原始医学图像的数据集合以及相应的原始图像标注信息集合。质量分析模块802用于对所述原始医学图像的数据集合进行质量分析，提取所述原始医学图像的特征信息。训练模块803用于基于提取的原始医学图像数据集合中的所述特征信息和原始图像标注信息集合训练对所述医学图像进行异常检测分类的所述深度学习网络，得到训练后的对医学图像进行异常检测分类的深度学习网络模型。
图9图示了示例系统900，其包括代表可以实现本文描述的各种技术的一个或多个系统和/或设备的示例计算设备910。计算设备910可以是例如服务提供商的服务器、与客户端(例如，客户端设备)相关联的设备、片上系统、和/或任何其他合适的计算设备或计算系统。上面关于图7用于对医学图像进行分类的服务器700或图8的用于对深度学习网络模型进行训练的服务器800可以采取计算设备910的形式。替换地，用于对医学图像进行分类的服务器700或图8的用于对模型进行训练的服务器800可以以医学图像分类应用916的形式被实现为计算机程序。
如图示的示例计算设备910包括彼此通信耦合的处理系统911、一个或多个计算机可读介质912以及一个或多个I/O接口913。尽管未示出，但是计算设备 910还可以包括系统总线或其他数据和命令传送系统，其将各种组件彼此耦合。系统总线可以包括不同总线结构的任何一个或组合，所述总线结构诸如存储器总线或存储器控制器、外围总线、通用串行总线、和/或利用各种总线架构中的任何一种的处理器或局部总线。还构思了各种其他示例，诸如控制和数据线。
处理系统911代表使用硬件执行一个或多个操作的功能。因此，处理系统911被图示为包括可被配置为处理器、功能块等的硬件元件914。这可以包括在硬件中实现为专用集成电路或使用一个或多个半导体形成的其他逻辑器件。硬件元件914不受其形成的材料或其中采用的处理机构的限制。例如，处理器可以由(多个)半导体和/或晶体管(例如，电子集成电路(IC))组成。在这样的上下文中，处理器可执行指令可以是电子可执行指令。
计算机可读介质912被图示为包括存储器/存储装置915。存储器/存储装置915表示与一个或多个计算机可读介质相关联的存储器/存储容量。存储器/存储装置915可以包括易失性介质(诸如随机存取存储器(RAM))和/或非易失性介质(诸如只读存储器(ROM)、闪存、光盘、磁盘等)。存储器/存储装置915可以包括固定介质(例如，RAM、ROM、固定硬盘驱动器等)以及可移动介质(例如，闪存、可移动硬盘驱动器、光盘等)。计算机可读介质912可以以下面进一步描述的各种其他方式进行配置。
一个或多个I/O接口913代表允许用户向计算设备910输入命令和信息并且可选地还允许使用各种输入/输出设备将信息呈现给用户和/或其他组件或设备的功能。输入设备的示例包括键盘、光标控制设备(例如，鼠标)、麦克风(例如，用于语音输入)、扫描仪、触摸功能(例如，被配置为检测物理触摸的容性或其他传感器)、相机(例如，可以采用可见或不可见的波长(诸如红外频率)将不涉及触摸的运动检测为手势)等等。输出设备的示例包括显示设备(例如，监视器或投影仪)、扬声器、打印机、网卡、触觉响应设备等。因此，计算设备910可以以下面进一步描述的各种方式进行配置以支持用户交互。
计算设备910还包括医学图像分类应用916。医学图像分类应用916可以例如是关于图7和8描述的用于对医学图像进行分类的服务器700或图8的用于用于对模型进行训练的服务器800的软件实例，并且与计算设备910中的其他元件相组合地实现本文描述的技术。
本文可以在软件硬件元件或程序模块的一般上下文中描述各种技术。一般地，这些模块包括执行特定任务或实现特定抽象数据类型的例程、程序、对象、元素、组件、数据结构等。本文所使用的术语“模块”，“功能”和“组件”一般表示软件、固件、硬件或其组合。本文描述的技术的特征是与平台无关的，意味着这些技术可以在具有各种处理器的各种计算平台上实现。
所描述的模块和技术的实现可以存储在某种形式的计算机可读介质上或者跨某种形式的计算机可读介质传输。计算机可读介质可以包括可由计算设备910访问的各种介质。作为示例而非限制，计算机可读介质可以包括“计算机可读存储介质”和“计算机可读信号介质”。
与单纯的信号传输、载波或信号本身相反，“计算机可读存储介质”是指能够持久存储信息的介质和/或设备，和/或有形的存储装置。因此，计算机可读存储介质是指非信号承载介质。计算机可读存储介质包括诸如易失性和非易失性、可移动和不可移动介质和/或以适用于存储信息(诸如计算机可读指令、数据结构、程序模块、逻辑元件/电路或其他数据)的方法或技术实现的存储设备之类的硬件。计算机可读存储介质的示例可以包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字通用盘(DVD)或其他光学存储装置、硬盘、盒式磁带、磁带，磁盘存储装置或其他磁存储设备，或其他存储设备、有形介质或适于存储期望信息并可以由计算机访问的制品。
“计算机可读信号介质”是指被配置为诸如经由网络将指令发送到计算设备910的硬件的信号承载介质。信号介质典型地可以将计算机可读指令、数据结构、程序模块或其他数据体现在诸如载波、数据信号或其他传输机制的调制数据信号中。信号介质还包括任何信息传递介质。术语“调制数据信号”是指以这样的方式对信号中的信息进行编码来设置或改变其特征中的一个或多个的信号。作为示例而非限制，通信介质包括诸如有线网络或直接连线的有线介质以及诸如声、RF、红外和其他无线介质的无线介质。
如前所述，硬件元件914和计算机可读介质912代表以硬件形式实现的指令、模块、可编程器件逻辑和/或固定器件逻辑，其在一些实施例中可以用于实现本文描述的技术的至少一些方面。硬件元件可以包括集成电路或片上系统、专用集成电路(ASIC)、现场可编程门阵列(FPGA)、复杂可编程逻辑器件(CPLD) 以及硅中的其他实现或其他硬件设备的组件。在这种上下文中，硬件元件可以作为执行由硬件元件所体现的指令、模块和/或逻辑所定义的程序任务的处理设备，以及用于存储用于执行的指令的硬件设备，例如，先前描述的计算机可读存储介质。
前述的组合也可以用于实现本文所述的各种技术和模块。因此，可以将软件、硬件或程序模块和其他程序模块实现为在某种形式的计算机可读存储介质上和/或由一个或多个硬件元件914体现的一个或多个指令和/或逻辑。计算设备910可以被配置为实现与软件和/或硬件模块相对应的特定指令和/或功能。因此，例如通过使用处理系统的计算机可读存储介质和/或硬件元件914，可以至少部分地以硬件来实现将模块实现为可由计算设备910作为软件执行的模块。指令和/或功能可以由一个或多个制品(例如，一个或多个计算设备910和/或处理系统911)可执行/可操作以实现本文所述的技术、模块和示例。
在各种实施方式中，计算设备910可以采用各种不同的配置。例如，计算设备910可以被实现为包括个人计算机、台式计算机、多屏幕计算机、膝上型计算机、上网本等的计算机类设备。计算设备910还可以被实现为包括诸如移动电话、便携式音乐播放器、便携式游戏设备、平板计算机、多屏幕计算机等移动设备的移动装置类设备。计算设备910还可以实现为电视类设备，其包括具有或连接到休闲观看环境中的一般地较大屏幕的设备。这些设备包括电视、机顶盒、游戏机等。
本文描述的技术可以由计算设备910的这些各种配置来支持，并且不限于本文所描述的技术的具体示例。功能还可以通过使用分布式系统、诸如通过如下所述的平台922而在“云”920上全部或部分地实现。
云920包括和/或代表用于资源924的平台922。平台922抽象云920的硬件(例如，服务器)和软件资源的底层功能。资源924可以包括在远离计算设备910的服务器上执行计算机处理时可以使用的应用和/或数据。资源924还可以包括通过因特网和/或通过诸如蜂窝或Wi-Fi网络的订户网络提供的服务。
平台922可以抽象资源和功能以将计算设备910与其他计算设备连接。平台922还可以用于抽象资源的分级以提供遇到的对于经由平台922实现的资源924的需求的相应水平的分级。因此，在互连设备实施例中，本文描述的功能的实现 可以分布在整个系统900内。例如，功能可以部分地在计算设备910上以及通过抽象云920的功能的平台922来实现。
应当理解，为清楚起见，参考不同的功能模块对本申请的实施例进行了描述。然而，将明显的是，在不偏离本申请的情况下，每个功能模块的功能性可以被实施在单个模块中、实施在多个模块中或作为其他功能模块的一部分被实施。例如，被说明成由单个模块执行的功能性可以由多个不同的模块来执行。因此，对特定功能模块的参考仅被视为对用于提供所描述的功能性的适当模块的参考，而不是表明严格的逻辑或物理结构或组织。因此，本申请可以被实施在单个模块中，或者可以在物理上和功能上被分布在不同的模块和电路之间。
将理解的是，尽管第一、第二、第三等术语在本文中可以用来描述各种设备、元件、或部件，但是这些设备、元件、或部件不应当由这些术语限制。这些术语仅用来将一个设备、元件、或部件与另一个设备、元件、或部件相区分。
尽管已经结合一些实施例描述了本申请，但是其不旨在被限于在本文中所阐述的特定形式。相反，本申请的范围仅由所附权利要求来限制。附加地，尽管单独的特征可以被包括在不同的权利要求中，但是这些可以可能地被有利地组合，并且包括在不同权利要求中不暗示特征的组合不是可行的和/或有利的。特征在权利要求中的次序不暗示特征必须以其工作的任何特定次序。此外，在权利要求中，词“包括”不排除其他元件，并且不定冠词“一”或“一个”不排除多个。权利要求中的附图标记仅作为明确的例子被提供，不应该被解释为以任何方式限制权利要求的范围。

Claims (20)

  1. 一种医学图像的分类方法，由计算设备执行，包括：
    获取医学图像的数据集合；
    对所述医学图像的数据集合进行质量分析，以提取所述医学图像的特征信息；
    基于提取的所述特征信息，利用预先训练的用于对所述医学图像进行异常检测分类的深度学习网络对所述医学图像的数据集合进行分类，得到分类结果。
  2. 如权利要求1所述的方法，其中，所述质量分析包括：色域-饱和度-亮度分析、清晰度分析、纹理分析和熵值分析中的至少一个；
    所述对所述医学图像的数据集合进行质量分析，以提取所述医学图像的特征信息，包括：
    对所述医学图像的数据集合进行色域-饱和度-亮度分析、清晰度分析、纹理分析和熵值分析中的至少一个，以得到所述医学图像的特征信息，所述医学图像的特征信息包括色域的特征信息、饱和度的特征信息、亮度的特征信息、清晰度指数、灰度边缘以及熵值中的至少一个。
  3. 如权利要求2所述的方法，其中，所述基于提取的所述特征信息，利用预先训练的用于对所述医学图像进行异常检测分类的深度学习网络对所述医学图像的数据集合进行分类，包括：
    根据提取的所述特征信息，去除所述医学图像的数据集合中的无关图像，所述无关图像包括非医学图像；
    利用所述用于对所述医学图像进行异常检测分类的深度学习网络对去除所述无关图像后的所述医学图像的数据集合进行分类。
  4. 如权利要求3所述的方法，其中，基于提取的所述特征信息，去除所述医学图像的数据集合中的无关图像，包括：
    如果所述医学图像的数据集合中存在其特征信息中的所述色域的特征信息、所述饱和度的特征信息、所述亮度的特征信息、所述清晰度指数、所述灰度边缘以及所述熵值中至少一个小于对应的阈值的医学图像，则确定所述医学图像为所述无关图像，并从所述医学图像的数据集合中去除。
  5. 如权利要求3所述的方法，其中，所述分类结果包括正常组织、无关组织、镜头失焦、白平衡失效以及无关图像的类别中的一个或多个。
  6. 如权利要求2所述的方法，其中所述色域-饱和度-亮度分析包括：将获取的所述医学图像的RGB空间中的红坐标r、绿坐标g、蓝坐标b值转换到HSV空间中，得到色域的特征信息h、饱和度的特征信息s和亮度的特征信息v；
    所述清晰度分析包括：计算所述数字图像矩阵的值a，将所述数字矩阵的值a与5×5的高斯卷积核进行卷积得到值c，基于a与c的最小均方差值来计算所述医学图像的清晰度指数；
    所述纹理分析包括：采用Sobel检验算子提取所述医学图像的灰度边缘；以及
    所述熵值分析包括：根据所述医学图像的长度、所述医学图像的宽度、以及滑动窗口内中心的灰度值和所述滑动窗口内除中心像素外的灰度均值在所述医学图像中出现的次数和概率来计算熵值。
  7. 如权利要求6所述的方法，其中，通过下式计算所述医学图像的清晰度指数PSNR：
    Figure PCTCN2020096797-appb-100001
    其中，MSE为所述a与c的最小均方差，所述a为计算得到的所述数字矩阵的值，所述c为将a与5×5的高斯卷积核进行卷积得到值，所述n为预设的正整数。
  8. 如权利要求6所述的方法，其中，通过下式计算熵值En：
    Figure PCTCN2020096797-appb-100002
    其中W为所述医学图像的长度，H为所述医学图像的宽度，f(i，j)表示(i，j)二元组在所述医学图像中出现的次数，i为滑动窗口内中心的灰度值，j为所述滑动窗口内除中心像素外的灰度均值，P i,j为(i，j)二元组在所述医学图像中出现的概率。
  9. 如权利要求1所述的方法，还包括：
    在进行所述质量分析之前，对获取的所述医学图像的数据集合进行文件校验和解码，以通过进行所述文件校验来确认所述医学图像的数据集合中的各医学图 像可解析，以及通过进行所述解码将所述医学图像转换为数字图像矩阵。
  10. 如权利要求1所述的方法，还包括：
    在对所述医学图像进行分类之前，对所述医学图像的数据集合的各医学图像进行插值和归一化处理。
  11. 如权利要求10所述的方法，其中，所述插值包括双线性插值。
  12. 如权利要求1所述的方法，所述方法进一步包括：选取Inception V3模型对所述深度学习网络进行训练。
  13. 一种对医学图像进行异常检测分类的模型的训练方法，由计算设备执行，包括：
    获取原始医学图像的数据集合以及相应的原始图像标注信息集合；
    对所述原始医学图像的数据集合进行质量分析，以提取所述原始医学图像的特征信息；
    基于提取的所述原始医学图像数据集合中的所述特征信息和原始图像标注信息集合来训练对医学图像进行异常检测分类的深度学习网络，得到训练后的对医学图像进行异常检测分类的深度学习网络模型。
  14. 如权利要求13所述的模型训练方法，其中，所述基于提取的所述原始医学图像数据集合中的所述特征信息和原始图像标注信息集合训练对医学图像进行异常检测分类的深度学习网络还包括：
    建立深度学习网络结构；
    设定训练优化函数、学习率和迭代次数；
    开始对所述深度学习网络的训练，并监测损失函数；以及
    选择所述损失函数最小时所对应的深度学习网络模型，并将其作为所述训练后的对医学图像进行异常检测分类的深度学习网络模型。
  15. 如权利要求13所述的模型训练方法，其中，所述原始图像标注信息集合中的标注信息包括针对所述医学图像是正常组织、无关组织、镜头失焦、白平衡失效和无关图像的标签。
  16. 一种医学图像的分类装置，包括：
    获取模块，用于获取医学图像的数据集合；
    质量分析模块，用于对所述医学图像的数据集合进行质量分析，以提取所述 医学图像的特征信息；
    分类模块，用于基于提取的所述特征信息，利用预先训练的用于对所述医学图像进行异常检测分类的深度学习网络对所述医学图像的数据集合进行分类，得到分类结果。
  17. 一种对医学图像进行异常检测分类的模型的训练装置，包括：
    获取模块，用于获取原始医学图像的数据集合以及相应的原始图像标注信息集合；
    质量分析模块，用于对所述原始医学图像的数据集合进行质量分析，提取所述原始医学图像的特征信息；
    训练模块，用于基于提取的所述原始医学图像数据集合中的所述特征信息和原始图像标注信息集合训练对所述医学图像进行异常检测分类的深度学习网络，得到训练后的对医学图像进行异常检测分类的深度学习网络模型。
  18. 一种计算设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现权利要求1～15任一项所述的方法。
  19. 一种医学图像分类系统，其中所述医学图像分类系统包括医学图像获取设备和医学图像处理设备；
    所述医学图像获取设备用于扫描医学图像，并向所述医学图像处理设备发送所述医学图像；
    所述医学图像处理设备用于执行如上所述权利要求1-15中任一项所述的方法。
  20. 一种计算机可读存储介质，其存储有可由计算机设备执行的计算机程序，当所述程序在计算机设备上运行时，使得所述计算机设备执行权利要求1-15任一项所述的方法。
PCT/CN2020/096797 2019-06-21 2020-06-18 一种医学图像的分类方法、模型训练方法、计算设备以及存储介质 WO2020253773A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/375,177 US11954852B2 (en) 2019-06-21 2021-07-14 Medical image classification method, model training method, computing device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910543018.9A CN110232719B (zh) 2019-06-21 2019-06-21 一种医学图像的分类方法、模型训练方法和服务器
CN201910543018.9 2019-06-21

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/375,177 Continuation US11954852B2 (en) 2019-06-21 2021-07-14 Medical image classification method, model training method, computing device, and storage medium

Publications (1)

Publication Number Publication Date
WO2020253773A1 true WO2020253773A1 (zh) 2020-12-24

Family

ID=67857144

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/096797 WO2020253773A1 (zh) 2019-06-21 2020-06-18 一种医学图像的分类方法、模型训练方法、计算设备以及存储介质

Country Status (3)

Country Link
US (1) US11954852B2 (zh)
CN (2) CN110232719B (zh)
WO (1) WO2020253773A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112784923A (zh) * 2021-02-07 2021-05-11 重庆渝抗医药科技有限公司 大数据环境下医疗异常影像的分类方法
CN114881929A (zh) * 2022-04-07 2022-08-09 北京医准智能科技有限公司 一种针对乳腺x线图像整体性质量异常的检测方法及装置

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110232719B (zh) * 2019-06-21 2021-10-26 腾讯科技（深圳）有限公司 一种医学图像的分类方法、模型训练方法和服务器
CN110852384B (zh) * 2019-11-12 2023-06-27 武汉联影医疗科技有限公司 医学影像质量检测方法、装置和存储介质
CN110838116B (zh) * 2019-11-14 2023-01-03 上海联影医疗科技股份有限公司 医学图像采集方法、装置、设备和计算机可读存储介质
US11545255B2 (en) * 2019-12-20 2023-01-03 Shanghai United Imaging Intelligence Co., Ltd. Systems and methods for classifying an anomaly medical image using variational autoencoder
CN110992360B (zh) * 2019-12-24 2024-01-23 北京安兔兔科技有限公司 设备性能测试方法、装置及电子设备
CN111160238A (zh) * 2019-12-27 2020-05-15 上海杏脉信息科技有限公司 显微图像质量分析方法、训练方法、系统、设备及介质
CN111310793B (zh) * 2020-01-17 2024-01-05 南方科技大学 一种医学图像分类方法、装置、移动终端及介质
CN111311545A (zh) * 2020-01-19 2020-06-19 上海箱云物流科技有限公司 集装箱检测方法、装置及计算机可读存储介质
CN111312372A (zh) * 2020-02-26 2020-06-19 广州金域医学检验中心有限公司 医学图像标准测试数据集的建立方法及装置
CN111369523B (zh) * 2020-03-02 2023-07-25 上海杏脉信息科技有限公司 显微图像中细胞堆叠的检测方法、系统、设备及介质
CN111523593B (zh) * 2020-04-22 2023-07-21 北京康夫子健康技术有限公司 用于分析医学影像的方法和装置
CN111626203B (zh) * 2020-05-27 2021-11-16 北京伟杰东博信息科技有限公司 一种基于机器学习的铁路异物识别方法及系统
CN111860143B (zh) * 2020-06-10 2022-08-02 北京航空航天大学 一种面向巡检机器人的火焰实时检测方法
CN113139935A (zh) * 2021-03-30 2021-07-20 宁波市眼科医院 裂隙灯图片质量监控系统
CN112990335B (zh) * 2021-03-31 2021-10-15 江苏方天电力技术有限公司 一种电网无人机巡检影像缺陷智能识别自学习训练方法、系统
CN113160166B (zh) * 2021-04-16 2022-02-15 宁波全网云医疗科技股份有限公司 通过卷积神经网络模型进行医学影像数据挖掘工作方法
CN113160167B (zh) * 2021-04-16 2022-01-14 深圳市铱硙医疗科技有限公司 通过深度学习网络模型进行医学影像数据提取工作方法
CN113313203B (zh) * 2021-06-22 2022-11-01 哈尔滨工程大学 一种基于可拓理论和深度学习的医学图像分类方法
CN113380381A (zh) * 2021-07-13 2021-09-10 深圳市中科微光医疗器械技术有限公司 获取医学诊断信息的方法、装置、电子设备及存储介质
CN114494238A (zh) * 2021-08-26 2022-05-13 深圳科亚医疗科技有限公司 一种病理图像的训练集处理方法和系统
CN113744845A (zh) * 2021-09-17 2021-12-03 平安好医投资管理有限公司 基于人工智能的医学影像处理方法、装置、设备及介质
CN114242209A (zh) * 2021-11-02 2022-03-25 深圳市智影医疗科技有限公司 一种医学图像预处理方法及系统
CN115148323B (zh) * 2022-09-06 2022-12-20 北京鹰瞳科技发展股份有限公司 基于医学图像进行疾病预测的装置、方法和可读存储介质
CN115205952B (zh) * 2022-09-16 2022-11-25 深圳市企鹅网络科技有限公司 一种基于深度学习的线上学习图像采集方法及系统
CN115995289B (zh) * 2023-03-20 2023-06-30 福建自贸试验区厦门片区Manteia数据科技有限公司 基于神经网络模型对医学图像的勾画方法和装置
CN116055900B (zh) * 2023-03-30 2023-06-09 北京城建智控科技股份有限公司 一种基于摄像装置的图像质量修正方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108364017A (zh) * 2018-01-24 2018-08-03 华讯方舟科技有限公司 一种图像质量分类方法、系统及终端设备
US20180253624A1 (en) * 2017-03-06 2018-09-06 Siemens Medical Solutions Usa, Inc. Defective Pixel Identification Using Machine Learning
CN109522960A (zh) * 2018-11-21 2019-03-26 泰康保险集团股份有限公司 图像评估方法、装置、电子设备及计算机可读介质
CN109815965A (zh) * 2019-02-13 2019-05-28 腾讯科技（深圳）有限公司 一种图像过滤方法、装置及存储介质
CN110232719A (zh) * 2019-06-21 2019-09-13 腾讯科技（深圳）有限公司 一种医学图像的分类方法、模型训练方法和服务器

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103581672B (zh) * 2012-08-06 2018-09-04 深圳市腾讯计算机系统有限公司 一种数据传输方法及设备
CN105100789B (zh) * 2015-07-22 2018-05-15 天津科技大学 一种视频质量评价方法
CN106651834A (zh) * 2016-10-20 2017-05-10 国网山东省电力公司电力科学研究院 无参考图像的变电设备红外热像图质量评价方法及装置
CN107066934A (zh) * 2017-01-23 2017-08-18 华东交通大学 胃部肿瘤细胞图像识别判定装置、方法及胃部肿瘤切片识别判定设备
CN106991439A (zh) * 2017-03-28 2017-07-28 南京天数信息科技有限公司 基于深度学习与迁移学习的图像识别方法
IL253532B (en) * 2017-07-17 2021-12-01 Au10Tix Ltd System, method and computer software product for analyzing images in JPEG format for forensic and other purposes
CN107862249B (zh) * 2017-10-18 2021-08-17 太原理工大学 一种分叉掌纹识别方法及装置
CN107729948A (zh) * 2017-10-31 2018-02-23 京东方科技集团股份有限公司 图像处理方法及装置、计算机产品和存储介质
CN108198171A (zh) * 2017-12-27 2018-06-22 大连理工大学 一种基于服务器表面热状态的运行状态诊断的方法
CN108615236A (zh) * 2018-05-08 2018-10-02 上海商汤智能科技有限公司 一种图像处理方法及电子设备
US11011265B2 (en) * 2018-06-28 2021-05-18 Case Western Reserve University Predicting prostate cancer risk of progression with multiparametric magnetic resonance imaging using machine learning and peritumoral radiomics
CN109118442B (zh) * 2018-07-20 2021-08-03 湖北工业大学 一种基于Sobel算子滤波的图像增强方法
CN109544503B (zh) * 2018-10-15 2020-12-01 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
US11471096B2 (en) * 2018-10-25 2022-10-18 The Chinese University Of Hong Kong Automatic computerized joint segmentation and inflammation quantification in MRI

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180253624A1 (en) * 2017-03-06 2018-09-06 Siemens Medical Solutions Usa, Inc. Defective Pixel Identification Using Machine Learning
CN108364017A (zh) * 2018-01-24 2018-08-03 华讯方舟科技有限公司 一种图像质量分类方法、系统及终端设备
CN109522960A (zh) * 2018-11-21 2019-03-26 泰康保险集团股份有限公司 图像评估方法、装置、电子设备及计算机可读介质
CN109815965A (zh) * 2019-02-13 2019-05-28 腾讯科技（深圳）有限公司 一种图像过滤方法、装置及存储介质
CN110232719A (zh) * 2019-06-21 2019-09-13 腾讯科技（深圳）有限公司 一种医学图像的分类方法、模型训练方法和服务器
CN110428475A (zh) * 2019-06-21 2019-11-08 腾讯科技（深圳）有限公司 一种医学图像的分类方法、模型训练方法和服务器

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112784923A (zh) * 2021-02-07 2021-05-11 重庆渝抗医药科技有限公司 大数据环境下医疗异常影像的分类方法
CN114881929A (zh) * 2022-04-07 2022-08-09 北京医准智能科技有限公司 一种针对乳腺x线图像整体性质量异常的检测方法及装置

Also Published As

Publication number Publication date
US20210343012A1 (en) 2021-11-04
CN110428475A (zh) 2019-11-08
CN110232719B (zh) 2021-10-26
CN110232719A (zh) 2019-09-13
US11954852B2 (en) 2024-04-09
CN110428475B (zh) 2021-02-05

Similar Documents

Publication Publication Date Title
WO2020253773A1 (zh) 一种医学图像的分类方法、模型训练方法、计算设备以及存储介质
US9710695B2 (en) Characterizing pathology images with statistical analysis of local neural network responses
CN109997147B (zh) 对鼓膜病理进行分类的系统、方法和计算机可读介质
WO2021196632A1 (zh) 一种全景数字病理图像智能分析系统及方法
US20180330522A1 (en) System and method for assessing wound
WO2021139258A1 (zh) 基于图像识别的细胞识别计数方法、装置和计算机设备
JP2021166062A (ja) 顕微鏡スライド画像のための焦点重み付き機械学習分類器誤り予測
Zhang et al. Automated semantic segmentation of red blood cells for sickle cell disease
WO2019198094A1 (en) Method and system for estimating total count of blood cells in a blood smear
WO2019102277A1 (en) Method and system for determining hematological parameters in a peripheral blood smear
US20190294863A9 (en) Method and apparatus for face classification
JP2018512567A (ja) 検査室自動化のためのサイドビューサンプルチューブ画像におけるバーコードタグ検出
CN111369523B (zh) 显微图像中细胞堆叠的检测方法、系统、设备及介质
CN108229289B (zh) 目标检索方法、装置和电子设备
US11721023B1 (en) Distinguishing a disease state from a non-disease state in an image
JP2021514228A (ja) 画像処理方法及び装置、並びにニューラルネットワークモデルのトレーニング方法
Hajabdollahi et al. Segmentation of bleeding regions in wireless capsule endoscopy images an approach for inside capsule video summarization
Kanwal et al. Quantifying the effect of color processing on blood and damaged tissue detection in whole slide images
Jia et al. GI bleeding detection in wireless capsule endoscopy images based on pattern recognition and a MapReduce framework
CN113705595A (zh) 异常细胞转移程度的预测方法、装置和存储介质
Bidaki et al. Lesion area assessment in psoriasis patients
EP4036867A2 (en) System and method for classifying dermatological images using machine learning
EP3907696A1 (en) Method and system for identifying abnormal images in a set of medical images
US11187644B2 (en) Method and system for determining total count of red blood cells in peripheral blood smear
CN113032622A (zh) 一种新型医学视频图像采集和数据管理系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20826310

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20826310

Country of ref document: EP

Kind code of ref document: A1