CN109410194A - A kind of cancer of the esophagus pathology image processing method based on deep learning - Google Patents

A kind of cancer of the esophagus pathology image processing method based on deep learning Download PDF

Info

Publication number
CN109410194A
CN109410194A CN201811222572.9A CN201811222572A CN109410194A CN 109410194 A CN109410194 A CN 109410194A CN 201811222572 A CN201811222572 A CN 201811222572A CN 109410194 A CN109410194 A CN 109410194A
Authority
CN
China
Prior art keywords
image
epithelium
layer
cancer
small
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811222572.9A
Other languages
Chinese (zh)
Other versions
CN109410194B (en
Inventor
葛菁
武鲁
赵志刚
王迪
李娜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Computer Science Center National Super Computing Center in Jinan
Original Assignee
Shandong Computer Science Center National Super Computing Center in Jinan
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Computer Science Center National Super Computing Center in Jinan filed Critical Shandong Computer Science Center National Super Computing Center in Jinan
Priority to CN201811222572.9A priority Critical patent/CN109410194B/en
Publication of CN109410194A publication Critical patent/CN109410194A/en
Application granted granted Critical
Publication of CN109410194B publication Critical patent/CN109410194B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20092Interactive image processing based on input by user
    • G06T2207/20104Interactive definition of region of interest [ROI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30096Tumor; Lesion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03Recognition of patterns in medical or anatomical images

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

Cancer of the esophagus pathology image processing method based on deep learning of the invention, comprising: a) pathological section scans;B) circle infuses upper dermatotome type, and the normal region in upper dermatotome, low level and high-level precancerous lesion region circle are outpoured;C) image preprocessing obtains the small image of epithelium;D) the small image of each epithelium is divided into n image block along its longitudinal direction by convolutional neural networks, carries out feature extraction to each image block;E) shot and long term memory network LSTM obtains the feature vector of the small image of epithelium;F) classifier is classified;G) model foundation and tuning, h) calculating of accuracy rate.Cancer of the esophagus pathology image processing method of the invention, after the processing of CNN, LSTM network and classifier, obtaining the small image of each epithelium is normal, low level and high-level precancerous lesion type probability, a kind of effective digital image processing method is provided for the scientific utilization of pathology department's cancer of the esophagus full slice, beneficial effect is significant, is suitable for the application of popularization.

Description

A kind of cancer of the esophagus pathology image processing method based on deep learning
Technical field
The present invention relates to a kind of cancer of the esophagus pathology image processing methods, more specifically, more particularly to a kind of based on depth The cancer of the esophagus pathology image processing method of degree study (CNN+LSTM).
Background technique
The cancer of the esophagus (esophageal cancer, EC) is initiated by the malignant tumor of digestive tract of Esophageal Mucosa epithelium, full generation There are about 300,000 people to die of the cancer of the esophagus every year on boundary.China is one of Esophageal Cancer, the country of high mortality in the world, and " China is swollen Tumor registration annual report " 2017 annual datas show that the cancer of the esophagus occupies the 4th of mortality of malignant tumors.China is more than 90% food at present To middle and advanced stage when pipe cancer patient makes a definite diagnosis, prognosis is poor, and quality of life is low, and respectively 5 years total survival rates of the cancer of the esophagus are big by stages Generally 14% or so, the health and life security of patient are seriously threatened.And the early stage cancer of the esophagus is usually through endoscopic minimally-invasive Treatment can eradicate, and acquirement and the comparable curative effect of surgical operation, 5 years survival rates of patient can be more than 95%.Therefore, to early stage oesophagus The clinical manifestation of cancer and pathology are analyzed, and accomplish early diagnosis, early treatment, are one of emphasis of clinical research, are survived to improving Rate is particularly significant.
With the rapid development and popularization and application of medical imaging devices, medical pathologies image data starts that exponential increasing is presented Long, manual inspection speed is slow, and manpower and material resources consuming is larger, and the rapid development of big data, artificial intelligence technology, so that depth Habit technology obtains immense success in computer vision field.Deep learning technology also becomes the master for solving medical image analysis task Want research direction.
In terms of cancer diagnosis, deep learning takes in the pathological diagnosis of cutaneum carcinoma, breast cancer, gastric cancer, colon cancer etc. Certain achievement is obtained, and can find lesion exception from X-ray, CT scan and MRI image, but in precancerous lesion of cancer of esophagus pathology It does not have progressed in diagnosis.Hospital pathology department is stored with the full pathological section of a large amount of cancer of the esophagus, these cancer of the esophagus pathological sections are just Form cancer of the esophagus pathology sample, with these samples come for the cancer of the esophagus diagnosis and screening provide science reference, so as to It assists doctor's work, improve precancerous lesion screening precision it is necessary to carry out the image analysis of science to cancer of the esophagus pathological section, lead to It crosses and effective feature extraction is carried out to image to distinguish normally and precancerous lesion, but there is presently no a kind of effective image procossings Method can be sliced the existing cancer of the esophagus of pathology department and be effectively treated.
Summary of the invention
The present invention in order to overcome the shortcomings of the above technical problems, provides a kind of cancer of the esophagus pathology figure based on deep learning As processing method.
Cancer of the esophagus pathology image processing method based on deep learning of the invention, which is characterized in that especially by following Step is realized:
A) pathological section scans, and is scanned using scanner to the sufficient amount of cancer of the esophagus pathological section of pathology department, with Obtain the digitlization pathological image of the cancer of the esophagus;B) circle infuses upper dermatotome type, digitizes pathology figure in the cancer of the esophagus that step a) is obtained As upper, the normal region in upper dermatotome, low level precancerous lesion region and high-level precancerous lesion region are enclosed outpour respectively by doctor Come, and adds corresponding type label in circle note region;C) image preprocessing, in order to reduce staining pathologic section and scanned The influence of journey carries out standards for dyeing processing to cancer of the esophagus digitlization pathological image first;Then, each circle in upper dermatotome is infused Region carries out direction adjustment according to " basal layer is located below, epithelial layer is located above ", then each circle note region is transversely cut It is divided into several small images, is denoted as the small image of epithelium;Finally, the small image of the insufficient epithelium of effective information is discarded, it is remaining on The small image of skin incorporates into as training set, verifying collection and test set;D) feature extraction of convolutional neural networks CNN, convolutional neural networks Data input layer the small image of epithelium is divided into n image block along its longitudinal direction, and according to the direction from basal layer to epithelial layer Sequence successively carries out 1 to n number to image block;Convolutional layer is using n image block of the small image of each epithelium as independent Image carries out feature extraction to each image block;E) feature extraction of shot and long term memory network LSTM, basal layer and epithelial layer Between there is High relevancy, there is in n image block of the same small image of epithelium strong link between the image block of adjacent number, it is sharp The implication relation in the small image of same epithelium between two adjacent image blocks is extracted with shot and long term memory network LSTM;On same The small image of skin obtains the feature vector that ranks are 1 × 3 after shot and long term memory network LSTM;F) classifier is classified, through step E) 1 × 3 feature vector obtained in is classified by classifier, obtains the small image discriminating of each epithelium as normal region, rudimentary The probability in other precancerous lesion region or high-level precancerous lesion region, and using the corresponding type of maximum probability sentencing as the image Other type;G) step d), model foundation and tuning are e) and f) processes for continuing iteration tuning, in model foundation, setting The number of iterations of loss value and model establishes model, and the picture number concentrated using verifying using the image data in training set It is persistently adjusted according to the parameter to convolutional neural networks CNN and long-term section memory network LSTM, until the loss value and iteration of model Until number reaches the threshold value of setting;H) accuracy rate calculates, small using the epithelium in test set after model training completion Image, the type of the statistics circle note probability consistent with type is differentiated, the as precision of model.
Cancer of the esophagus pathology image processing method based on deep learning of the invention, the image preprocessing in step c) pass through Following steps are realized:
C-1) standards for dyeingization is handled, during H&E dyeing course and digitlization, different laboratory and different It is different that scanner can to digitize pathological image color, shallow or deep or purple or powder, by standards for dyeing by original disease The color for managing sectioning image is unified, to improve the extensive level of model;C-2) the cutting in circle note region, since doctor encloses note Area size is different, it is therefore desirable to cutting processing is carried out to circle note region, to obtain the small image of the consistent epithelium of size, as The input of CNN network model;The cutting in circle note region is realized by following steps: c-2-1) adjustment circle note region direction, benefit With gradient information, the direction of circle note area image is adjusted so that each circle adjusted infuse be below region basal layer, Top is epithelial layer;C-2-2) circle note image cutting, for unified input picture size and increase the quantity of training sample, press According to 50% repetitive rate, it is that 640 pixels × 300 pixels are more that by the circle note region after adjustment direction, transversely cutting, which is height × width, A small image, small image are denoted as the small image of epithelium;C-2-3) image base is finely tuned, and finds basal layer in the small image of each epithelium The pixel position of bottom, using the position as the initial pixel of image base point position, so that the base of the small image of all epitheliums Bottom is all close to image base, abandons the white space of image base;C-2-4) image top is finely tuned, by step c-2-3) Effective information in the small image of obtained epithelium is normally at the middle and lower part of image, and there are white spaces on top, to reduce blank Influence of the region to overall model, using 515 pixels of the upward number of bottom pixel as final picture size, i.e. epithelium The size adjusting of small image is that height × width is 515 pixels × 300 pixels;C-3) image filtering is handled, to guarantee input picture Effective information is enough, and the small image of epithelium that cutting is formed is filtered, and abandons image of the effective information less than 60%, remains with Effect information is more than 60% image, the input picture as final mask.
Cancer of the esophagus pathology image processing method based on deep learning of the invention, the convolutional neural networks in step d) The feature extraction of CNN, the tandem according to data processing successively include data input layer, 3 convolutional layers, 1 pond layer, 1 A local acknowledgement's normalization layer, 2 convolutional layers, 1 pond layer, 1 local acknowledgement normalize layer;
The small image of 515 pixels that data input layer exports image preprocessing × 300 pixels epithelium, is divided into along longitudinal direction 5 height × width are the image block of 103 pixel × 300, and according to the direction sequence from basal layer to epithelial layer to image block from 1 to 5 are numbered;
5 image blocks of the small image of same epithelium are regarded as independent image and carry out feature extraction, Mei Gejuan by convolutional layer Lamination extracts the feature of image block according to formula (1),
In formula, l indicates the convolution number of plies locating for current convolution,Indicate connect l-1 layer ith feature image with The convolution kernel of l layers of jth characteristic image,For l layers of j-th of characteristic image,For l-1 layers of ith feature Image,Indicate l layers of bias term, f () indicates nonlinear activation function;
The convolution kernel of 5 convolutional layers is respectively set to 11 × 11,1 × 7,7 × 1,3 × 3,1 × 1, nonlinear activation function ReLU is used, the stride of first layer convolutional layer is set as 2, and the stride of remaining convolutional layer is disposed as 1;2 layers of pond layer are adopted With max_pooling mode, local receptor field is 3 × 3, and stride is 2;The learning rate of 2 local acknowledgements normalization layer and The number of iterations is respectively set to 0.00001 and 60 time.
Cancer of the esophagus pathology image processing method based on deep learning of the invention, shot and long term described in step e) remember net The feature extraction of network LSTM passes through formula (2) and formula (3) Lai Shixian:
ht=ot⊙tanh(Ct) (3)
Wherein, ht-1Indicate the output of a memory cell, Ct-1Indicate that the state an of memory cell updates, xtTable Show as precellular input, CtIndicate that the state of current memory cell updates, htIndicate the output of current memory cell;⊙ is indicated Point processing, tanh () are hyperbolic tangent functions, and σ () is sigmoid function, Wf、Wi、WC、WoRespectively indicate parameter ft、it otWeight matrix, bf、bi、bC、boRespectively indicate parameter ft、itotBias term.
Cancer of the esophagus pathology image processing method based on deep learning of the invention, classifier used in step f) are Softmax is a polytypic classifier, pathological image is divided into 3 classes using softmax classifier, respectively with 1,2,3 tables Show high-level precancerous lesion, low level precancerous lesion and normal;The calculation formula of classifier is as follows:
In formula, j represents type, and T is type sum, due to being divided into high-level precancerous lesion, low level precancerous lesion and just Normal three classes, therefore T value is 3;αjIndicate j-th of value of input feature value, PjIt is expressed as the probability of type j;When a disease of input When reason image data exports one 1 × 3 vector by softmax, the corresponding classification of the maximum probability of vector intermediate value is taken to make For the prediction label of this input data, i.e., affiliated type.
Cancer of the esophagus pathology image processing method based on deep learning of the invention, in step g), the loss value of model and The number of iterations is arranged in model foundation, when model loss value is greater than the number of iterations threshold of setting loss threshold value or not up to setting When value, in formula (1)W in formula (4)f、Wi、WC、WoAnd bf、bi、bC、boIt is adjusted, so that model is restrained.
The beneficial effects of the present invention are: cancer of the esophagus pathology image processing method of the invention, first by doctor to digitlization Cancer of the esophagus pathological image is labeled, by image normal, low level precancerous lesion and high-level precancerous lesion mark respectively Out, standards for dyeing, cutting and filtering processing then are carried out to the image of mark, forms the small image of epithelium and each small image N image block;Then feature extraction is carried out to each image block by convolutional neural networks CNN, then through shot and long term memory network LSTM obtains the feature vector of each small image, after the classification processing of last categorized device, obtains the small image of each epithelium and is positive Often, the probability of rudimentary and high-level precancerous lesion type obtains the classification mould for differentiating that accuracy is met the requirements after training Type provides a kind of effective digital image processing method for the scientific utilization of pathology department's cancer of the esophagus pathological section, beneficial Significant effect is suitable for the application of popularization.
Detailed description of the invention
Fig. 1 is the flow diagram of cancer of the esophagus pathology image processing method of the invention;
Fig. 2 is the process flow diagram of convolutional neural networks CNN, shot and long term memory network LSTM and classifier in the present invention;
Fig. 3 is the partial schematic diagram of the cancer of the esophagus pathology sectioning image of doctor's mark in the present invention;
Fig. 4 is the schematic diagram that each small image of epithelium is divided into 5 image blocks along longitudinal direction in the present invention;
Fig. 5 is the calculating schematic diagram that shot and long term memory network LSTM seeks the small image feature vector of epithelium in the present invention;
Fig. 6 is cell state C in memory cell in shot and long term memory network LSTM modeltWith output htCalculating process;
Fig. 7 is the flow diagram of image preprocessing in the present invention.
Specific embodiment
The invention will be further described with embodiment with reference to the accompanying drawing.
As shown in Figure 1, the flow diagram of cancer of the esophagus pathology image processing method of the invention is given, if according to mould The form of block is illustrated to bright cancer of the esophagus pathology image processing method is distributed, by image pre-processing module, CNN convolution Neural network module, LSTM shot and long term memory network module and classifier modules composition, pathology figure of the image preprocessing to input As carrying out cutting, standards for dyeingization processing, and corresponding label is stamped for each input picture, as CNN convolutional neural networks The input of module;CNN convolutional neural networks module carries out longitudinal stripping and slicing to each picture of input first, each is inputted Image cutting is 5 image blocks of same size, carries out feature extraction to this 5 image blocks, that is, passes through CNN convolutional neural networks Module obtains 5 feature vectors of an input picture;This 5 features are input to LSTM shot and long term memory net in sequence In network model, by the model, 5 feature vectors of an input picture is integrated into a feature vector and are exported, are passed through Softmax classifier modules are obtained the probability that the input picture is determined as each rank, and are made with the corresponding rank of maximum probability For the differentiation rank of the image.
As shown in Fig. 2, giving convolutional neural networks CNN in the present invention, shot and long term memory network LSTM and classifier Process flow diagram, the cancer of the esophagus pathology image processing method of the invention based on deep learning come real especially by following steps It is existing:
A) pathological section scans, and is scanned using scanner to pathology department's full pathological section of the sufficient amount of cancer of the esophagus, To obtain the digitlization pathological image of the cancer of the esophagus;
B) circle infuses upper dermatotome type, and on the cancer of the esophagus digitlization pathological image that step a) is obtained, doctor is by upper dermatotome Normal region, low level precancerous lesion region and high-level precancerous lesion region, which are enclosed to outpour respectively, to be come, and is added in circle note region Corresponding type label;
As shown in figure 3, giving the partial schematic diagram for the full pathological section image of the cancer of the esophagus that doctor marks in the present invention, institute The region of the enclosed note of the normal shown is normal region, show the region there is no canceration, shown in HGEIN mark it is high-level Precancerous lesion region.Low level precancerous lesion region is indicated with low, not comprising low level precancerous lesion region in Fig. 3.Due to Precancerous lesion of cancer of esophagus is located in the epithelial region of pathological section, and the region of the enclosed note of pathologist is also epithelial region.It will doctor The region of raw circle note exports as the pathological picture of jpeg format, the initial data of design method as of the present invention.By the area Quan Zhu When domain exports as jpeg format, the region enclosed regardless of doctor be greatly it is small, scanner carries software and automatically protects all pictures Saving as width is 640 pixels, the unfixed vertical bar shape picture of length, is not unified for identical amplification factor, therefore this The recognition methods of invention design is suitable for the precancerous lesion of cancer of esophagus pathological image of different amplification.
Pathologist has the following to be noted that during diagnosis first is that the basal layer of epithelial region is mentioned in diagnosis For main information, it is therefore desirable to complete basal layer;Second is that organized region will enough in each pathological image, Bu Nengcun In a large amount of white space;Third is that diagnosis Shi Yaocong basal layer starts to observe to epithelial layer.
C) image preprocessing first digitizes the cancer of the esophagus to reduce the influence of staining pathologic section and scanning process Pathological image carries out standards for dyeing processing;Then, by each circle in upper dermatotome note region according to " basal layer is located below, on Cortex is located above " direction adjustment is carried out, then by each circle note region, transversely cutting is several small images, is denoted as the small figure of epithelium Picture;Finally, the small image of the insufficient epithelium of effective information is discarded, the remaining small image of epithelium is incorporated into as training set, verifying collection And test set;
As shown in fig. 7, the flow diagram of image preprocessing in the present invention is given, the image preprocessing in this step, It is realized especially by following steps:
C-1) standards for dyeingization is handled, during H&E dyeing course and digitlization, different laboratory and different It is different that scanner can to digitize pathological image color, shallow or deep or purple or powder, by standards for dyeing by original disease The color for managing sectioning image is unified, to improve the extensive level of model;
C-2) the cutting in circle note region, since the area size that doctor encloses note is different, it is therefore desirable to which circle note region is carried out Cutting processing, the input to obtain the small image of the consistent epithelium of size, as CNN network model;The cutting in circle note region passes through Following steps are realized:
C-2-1) adjustment circle note region direction is adjusted the direction of circle note area image using gradient information, so that It is basal layer, top below each circle note adjusted region is epithelial layer;
C-2-2) circle note image cutting, for unified input picture size and increase the quantity of training sample, according to 50% Repetitive rate, by the circle note region after adjustment direction, transversely cutting be height × width for the multiple small figures of 640 pixels × 300 pixels Picture, small image are denoted as the small image of epithelium;
C-2-3) image base is finely tuned, and the pixel position of basal layer bottom in the small image of each epithelium is found, with this Position is the initial pixel point position of image base, so that the basal layer of the small image of all epitheliums is all close to image base, is abandoned The white space of image base;
C-2-4) image top is finely tuned, the general position of effective information in the small image of epithelium obtained by step c-2-3) In the middle and lower part of image, there are white spaces on top, to reduce influence of the white space to overall model, with bottom pixel Upward 515 pixels of number are as final picture size.
C-3) image filtering is handled, and the effective information to guarantee input picture is enough, the small image of epithelium that cutting is formed It is filtered, abandons image of the effective information less than 60%, retain the image that effective information is more than 60%, as final mask Input picture.
D) feature extraction of convolutional neural networks CNN, the data input layer of convolutional neural networks is by the small image of epithelium along it Longitudinal direction is divided into n image block, and successively carries out 1 to n volume to image block according to the direction sequence from basal layer to epithelial layer Number;Convolutional layer as independent image, carries out n image block of each epithelium small image feature to each image block and mentions It takes;
The feature extraction of convolutional neural networks CNN in this step, is realized especially by following steps: at data The tandem of reason successively includes data input layer, 3 convolutional layers, 1 pond layer, 1 local acknowledgement's normalization layer, 2 volumes Lamination, 1 pond layer, 1 local acknowledgement normalize layer;
The small image of 515 pixels that data input layer exports image preprocessing × 300 pixels epithelium, is divided into along longitudinal direction 5 height × width are 103 pixels × 300 pixels image block, and according to the direction sequence from basal layer to epithelial layer to image block It is numbered from 1 to 5;As shown in figure 4, give each small image of epithelium in the present invention is divided into 5 image blocks along longitudinal direction Schematic diagram
5 image blocks of the small image of same epithelium are regarded as independent image and carry out feature extraction, Mei Gejuan by convolutional layer Lamination extracts the feature of image block according to formula (1),
In formula, l indicates the convolution number of plies locating for current convolution,Indicate connect l-1 layer ith feature image with The convolution kernel of l layers of jth characteristic image,For l layers of j-th of characteristic image,For l-1 layers of ith feature Image,Indicate l layers of bias term, f () indicates nonlinear activation function;
The convolution kernel of 5 convolutional layers is respectively set to 11 × 11,1 × 7,7 × 1,3 × 3,1 × 1, nonlinear activation function ReLU is used, the stride of first layer convolutional layer is set as 2, and the stride of remaining convolutional layer is disposed as 1;2 layers of pond layer are adopted With max_pooling mode, local receptor field is 3 × 3, and stride is 2;The learning rate of 2 local acknowledgements normalization layer and The number of iterations is respectively set to 0.00001 and 60 time.
E) feature extraction of shot and long term memory network LSTM, between basal layer and epithelial layer have High relevancy, it is same on There is between the image block of adjacent number strong link in n image block of the small image of skin, mentioned using shot and long term memory network LSTM Take the implication relation in the small image of same epithelium between two adjacent image blocks;Image small for same epithelium, remembers through shot and long term After network LSTM, the feature vector that ranks are 1 × 3 is obtained;
LSTM model is common a kind of network model in series processing task, such as voice and handwriting recognition, especially with The highly relevant problem of time series.The present invention using five consecutive image blocks of an image as comprising five time points when Between sequence handled.Corresponding five feature vectors of five image blocks are a spy of initial pictures by LSTM model integration Levy vector.
As shown in figure 5, giving the meter that shot and long term memory network LSTM in the present invention seeks the small image feature vector of epithelium Schematic diagram is calculated, LSTM model is included multiple memory cells and is connected with each other with time sequencing, and realizes information by gate function Addition and discarding.Each memory cell includes that there are three doors, and control each memory cell by these three doors Output state.Three doors are respectively as follows: input gate, forget door and out gate.Input gate (input gate) is worked as controlling The influence that the input information at preceding moment generates the node.Forget door (forget gate) and forgets that the node is remembered for appropriate The historical information of record.With the continuous propulsion of time, some otiose historical informations are permanent to be forgotten.Out gate (output Gate) it is used to control the output of this node in current time role, because the node is remembered in certain specific situations The information of record is not the effective or main feature of required task, therefore decays to it, so that the output information of other nodes Play the role of more main.
In LSTM model, the door of each memory cell is the output h about a upper memory cellt-1With it is current The input x of celltC is updated to codetermine the state of current memory celltWith output ht, as shown in fig. 6, giving shot and long term Cell state C in memory cell in memory network LSTM modeltWith output htCalculating process, alphabetical a-quadrant in Fig. 5 and The calculating in region is as shown in Figure 6 between two letter A.
The feature extraction of shot and long term memory network LSTM in this step passes through formula (2) and formula (3) Lai Shixian:
ht=ot⊙tanh(Ct) (3)
Wherein, ht-1Indicate the output of a memory cell, Ct-1Indicate that the state an of memory cell updates, xtTable Show as precellular input, CtIndicate that the state of current memory cell updates, htIndicate the output of current memory cell;⊙ is indicated Point processing, tanh () are hyperbolic tangent functions, and σ () is sigmoid function, Wf、Wi、WC、WoRespectively indicate parameter ft、it otWeight matrix, bf、bi、bC、boRespectively indicate parameter ft、itotBias term.
F) classifier is classified, and 1 × 3 feature vector through obtaining in step e) is classified by classifier, is obtained on each The small image discriminating of skin is the probability in normal region, low level precancerous lesion region or high-level precancerous lesion region, and with highest Differentiation type of the corresponding type of probability as the image;
Classifier used in this step is softmax, is a polytypic classifier, is classified using softmax Pathological image is divided into 3 classes by device, indicates high-level precancerous lesion, low level precancerous lesion and normal with 1,2,3 respectively;Classifier Calculation formula it is as follows:
In formula, j represents type, and T is type sum, due to being divided into high-level precancerous lesion, low level precancerous lesion and just Normal three classes, therefore T value is 3;αjIndicate j-th of value of input feature value, PjIt is expressed as the probability of type j;When a disease of input When reason image data exports one 1 × 3 vector by softmax, the corresponding classification of the maximum probability of vector intermediate value is taken to make For the prediction label of this input data, i.e., affiliated type.
Such as when the row vector that LSTM exports one 1 × 3 is [0.81155 0.13428149 0.05416854], from a left side The probability in high-level precancerous lesion region, low level precancerous lesion region and normal region is respectively corresponded to the right side, it is high level general Rate highest, it is believed that the small image of this epithelium belongs to high-level.
G) step d), model foundation and tuning are e) and f) processes for continuing iteration tuning, in model foundation, setting The number of iterations of loss value and model establishes model, and the picture number concentrated using verifying using the image data in training set It is persistently adjusted according to the parameter to convolutional neural networks CNN and long-term section memory network LSTM, until the loss value and iteration of model Until number reaches the threshold value of setting;
Wherein, the loss value and the number of iterations of model are arranged in model foundation, when model loss value is greater than setting loss When threshold value or the number of iterations threshold value of not up to setting, in formula (1)W in formula (4)f、Wi、WC、WoAnd bf、 bi、bC、boIt is adjusted, so that model is restrained.
H) accuracy rate calculates, and after model training completion, using the small image of epithelium in test set, statistics circle is infused The type probability consistent with type is differentiated, the as precision of model.

Claims (6)

1. a kind of cancer of the esophagus pathology image processing method based on deep learning, which is characterized in that come especially by following steps It realizes:
A) pathological section scans, and is scanned using scanner to the sufficient amount of cancer of the esophagus pathological section of pathology department, to obtain The digitlization pathological image of the cancer of the esophagus;
B) circle infuses upper dermatotome type, and on the cancer of the esophagus digitlization pathological image that step a) is obtained, doctor is normal by upper dermatotome Region, low level precancerous lesion region and high-level precancerous lesion region, which are enclosed to outpour respectively, to be come, and is corresponded in the region addition of circle note Type label;
C) image preprocessing digitizes pathology to the cancer of the esophagus first to reduce the influence of staining pathologic section and scanning process Image carries out standards for dyeing processing;Then, by each circle in upper dermatotome note region according to " basal layer is located below, epithelial layer It is located above " direction adjustment is carried out, then by each circle note region, transversely cutting is several small images, is denoted as the small image of epithelium; Finally, the small image of the insufficient epithelium of effective information is discarded, the remaining small image of epithelium is incorporated into as training set, verifying collection and is surveyed Examination collection;
D) feature extraction of convolutional neural networks CNN, the data input layer of convolutional neural networks is by the small image of epithelium along its longitudinal direction It is divided into n image block, and successively carries out 1 to n number to image block according to the direction sequence from basal layer to epithelial layer;Volume N image block of the small image of each epithelium as independent image, is carried out feature extraction to each image block by lamination;
E) feature extraction of shot and long term memory network LSTM, has High relevancy between basal layer and epithelial layer, same epithelium is small There is between the image block of adjacent number strong link in n image block of image, extracted using shot and long term memory network LSTM same Implication relation in the small image of one epithelium between two adjacent image blocks;Image small for same epithelium, through shot and long term memory network After LSTM, the feature vector that ranks are 1 × 3 is obtained;
F) classifier is classified, and 1 × 3 feature vector through obtaining in step e) is classified by classifier, and it is small to obtain each epithelium Image discriminating is the probability in normal region, low level precancerous lesion region or high-level precancerous lesion region, and with maximum probability Differentiation type of the corresponding type as the image;
G) step d), model foundation and tuning are e) and f) processes for continuing iteration tuning, in model foundation, loss is arranged The number of iterations of value and model establishes model, and the image data pair concentrated using verifying using the image data in training set The parameter of convolutional neural networks CNN and long-term section memory network LSTM persistently adjust, until the loss value and the number of iterations of model Until the threshold value for reaching setting;
H) accuracy rate calculates, and after model training completion, utilizes the small image of epithelium in test set, the type of statistics circle note The probability consistent with type is differentiated, the as precision of model.
2. the cancer of the esophagus pathology image processing method according to claim 1 based on deep learning, which is characterized in that step C) image preprocessing in is realized by following steps:
C-1) standards for dyeingization is handled, during H&E dyeing course and digitlization, different laboratories and different scanning Instrument can make digitlization pathological image color different, and shallow or deep or purple or powder is cut original pathology by standards for dyeing The color of picture is unified, to improve the extensive level of model;
C-2) the cutting in circle note region, since the area size that doctor encloses note is different, it is therefore desirable to which cutting is carried out to circle note region Processing, the input to obtain the small image of the consistent epithelium of size, as CNN network model;The cutting in circle note region passes through following Step is realized:
C-2-1) adjustment circle note region direction is adjusted the direction of circle note area image, using gradient information so that adjustment It is basal layer, top below each circle note region is afterwards epithelial layer;
C-2-2) circle note image cutting, for unified input picture size and increase the quantity of training sample, according to 50% weight Multiple rate, it is the multiple small images of 640 pixels × 300 pixels that by the circle note region after adjustment direction, transversely cutting, which is height × width, Small image is denoted as the small image of epithelium;
C-2-3) image base is finely tuned, and the pixel position of basal layer bottom in the small image of each epithelium is found, with the position Image is abandoned so that the basal layer of the small image of all epitheliums is all close to image base for the initial pixel point position of image base The white space of bottom;
C-2-4) image top is finely tuned, and the effective information in the small image of the epithelium obtained by step c-2-3) is normally at figure The middle and lower part of picture, there are white spaces on top, upward with bottom pixel to reduce influence of the white space to overall model For 515 pixels of number as final picture size, i.e. the size adjusting of the small image of epithelium is that height × width is 515 pixel × 300 Pixel;
C-3) image filtering is handled, and the effective information to guarantee input picture is enough, and the small image of the epithelium that cutting is formed carries out Filtering abandons image of the effective information less than 60%, retains the image that effective information is more than 60%, the input as final mask Image.
3. the cancer of the esophagus pathology image processing method according to claim 2 based on deep learning, which is characterized in that step D) feature extraction of the convolutional neural networks CNN in, the tandem according to data processing successively include data input layer, 3 Convolutional layer, 1 pond layer, 1 local acknowledgement's normalization layer, 2 convolutional layers, 1 pond layer, 1 local acknowledgement normalize layer;
The small image of 515 pixels that data input layer exports image preprocessing × 300 pixels epithelium, is divided into 5 along longitudinal direction Height × width be 103 pixel × 300 image block, and according to from basal layer to epithelial layer direction sequence to image block from 1 to 5 into Row number;
5 image blocks of the small image of same epithelium are regarded as independent image and carry out feature extraction, each convolutional layer by convolutional layer The feature of image block is extracted according to formula (1),
In formula, l indicates the convolution number of plies locating for current convolution,Indicate connect l-1 layers ith feature image and l layers Jth characteristic image convolution kernel,For l layers of j-th of characteristic image,For l-1 layers of ith feature image,Indicate l layers of bias term, f () indicates nonlinear activation function;
The convolution kernel of 5 convolutional layers is respectively set to 11 × 11,1 × 7,7 × 1,3 × 3,1 × 1, and nonlinear activation function makes 2 are set as with the stride of ReLU, first layer convolutional layer, the stride of remaining convolutional layer is disposed as 1;2 layers of pond layer are all made of Max_pooling mode, local receptor field are 3 × 3, and stride is 2;2 local acknowledgements normalize the learning rate of layer and change Generation number is respectively set to 0.00001 and 60 time.
4. the cancer of the esophagus pathology image processing method according to claim 3 based on deep learning, which is characterized in that step E) feature extraction of the shot and long term memory network LSTM described in passes through formula (2) and formula (3) Lai Shixian:
ht=ot⊙tanh(Ct) (3)
Wherein, ht-1Indicate the output of a memory cell, Ct-1Indicate that the state an of memory cell updates, xtExpression is worked as Precellular input, CtIndicate that the state of current memory cell updates, htIndicate the output of current memory cell;⊙ indicates point fortune It calculates, tanh () is hyperbolic tangent function, and σ () is sigmoid function, Wf、Wi、WC、WoRespectively indicate parameter ft、itot's Weight matrix, bf、bi、bC、boRespectively indicate parameter ft、itotBias term.
5. the cancer of the esophagus pathology image processing method according to claim 4 based on deep learning, which is characterized in that step F) classifier used in is softmax, is a polytypic classifier, using softmax classifier by pathological image It is divided into 3 classes, indicates high-level precancerous lesion, low level precancerous lesion and normal with 1,2,3 respectively;The calculation formula of classifier is such as Under:
In formula, j represents type, and T is type sum, due to being divided into high-level precancerous lesion, low level precancerous lesion and normal three Class, therefore T value is 3;αjIndicate j-th of value of input feature value, PjIt is expressed as the probability of type j;When a pathology figure of input When exporting one 1 × 3 vector by softmax as data, take the corresponding classification of the maximum probability of vector intermediate value as this The prediction label of a input data, i.e., affiliated type.
6. the cancer of the esophagus pathology image processing method according to claim 5 based on deep learning, which is characterized in that step G) in, the loss value and the number of iterations of model are arranged in model foundation, when model loss value is greater than setting loss threshold value or not When reaching the number of iterations threshold value of setting, in formula (1)W in formula (4)f、Wi、WC、WoAnd bf、bi、bC、boInto Row adjustment, so that model is restrained.
CN201811222572.9A 2018-10-19 2018-10-19 Esophageal cancer pathological image processing method based on deep learning Active CN109410194B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811222572.9A CN109410194B (en) 2018-10-19 2018-10-19 Esophageal cancer pathological image processing method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811222572.9A CN109410194B (en) 2018-10-19 2018-10-19 Esophageal cancer pathological image processing method based on deep learning

Publications (2)

Publication Number Publication Date
CN109410194A true CN109410194A (en) 2019-03-01
CN109410194B CN109410194B (en) 2022-03-29

Family

ID=65467845

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811222572.9A Active CN109410194B (en) 2018-10-19 2018-10-19 Esophageal cancer pathological image processing method based on deep learning

Country Status (1)

Country Link
CN (1) CN109410194B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109805963A (en) * 2019-03-22 2019-05-28 深圳开立生物医疗科技股份有限公司 The determination method and system of one Endometrium parting
CN109948733A (en) * 2019-04-01 2019-06-28 深圳大学 More classification methods, sorter and the storage medium of alimentary tract endoscope image
CN110220902A (en) * 2019-05-09 2019-09-10 杭州迪英加科技有限公司 Digital pathological section analysis method and device
CN110659692A (en) * 2019-09-26 2020-01-07 重庆大学 Pathological image automatic labeling method based on reinforcement learning and deep neural network
CN110866908A (en) * 2019-11-12 2020-03-06 腾讯科技(深圳)有限公司 Image processing method, image processing apparatus, server, and storage medium
CN111047559A (en) * 2019-11-21 2020-04-21 万达信息股份有限公司 Method for rapidly detecting abnormal area of digital pathological section
CN111062448A (en) * 2019-12-24 2020-04-24 北京知道创宇信息技术股份有限公司 Equipment type recognition model training method, equipment type recognition method and device
CN111080654A (en) * 2019-11-18 2020-04-28 中国科学院深圳先进技术研究院 Image lesion region segmentation method and device and server
CN111462076A (en) * 2020-03-31 2020-07-28 湖南国科智瞳科技有限公司 Method and system for detecting fuzzy area of full-slice digital pathological image
WO2021070108A1 (en) * 2019-10-11 2021-04-15 International Business Machines Corporation Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory
WO2021097595A1 (en) * 2019-11-18 2021-05-27 中国科学院深圳先进技术研究院 Method and apparatus for segmenting lesion area in image, and server
US11417424B2 (en) 2019-10-11 2022-08-16 International Business Machines Corporation Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory and multiple instance learning
CN115294126A (en) * 2022-10-08 2022-11-04 南京诺源医疗器械有限公司 Intelligent cancer cell identification method for pathological image

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106780475A (en) * 2016-12-27 2017-05-31 北京市计算中心 A kind of image processing method and device based on histopathologic slide's image organizational region
CN107369151A (en) * 2017-06-07 2017-11-21 万香波 System and method are supported in GISTs pathological diagnosis based on big data deep learning
CN107423756A (en) * 2017-07-05 2017-12-01 武汉科恩斯医疗科技有限公司 Nuclear magnetic resonance image sequence sorting technique based on depth convolutional neural networks combination shot and long term memory models
CN108305249A (en) * 2018-01-24 2018-07-20 福建师范大学 The quick diagnosis and methods of marking of full size pathological section based on deep learning
CN108629768A (en) * 2018-04-29 2018-10-09 山东省计算中心(国家超级计算济南中心) The dividing method of epithelial tissue in a kind of oesophagus pathological image

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106780475A (en) * 2016-12-27 2017-05-31 北京市计算中心 A kind of image processing method and device based on histopathologic slide's image organizational region
CN107369151A (en) * 2017-06-07 2017-11-21 万香波 System and method are supported in GISTs pathological diagnosis based on big data deep learning
CN107423756A (en) * 2017-07-05 2017-12-01 武汉科恩斯医疗科技有限公司 Nuclear magnetic resonance image sequence sorting technique based on depth convolutional neural networks combination shot and long term memory models
CN108305249A (en) * 2018-01-24 2018-07-20 福建师范大学 The quick diagnosis and methods of marking of full size pathological section based on deep learning
CN108629768A (en) * 2018-04-29 2018-10-09 山东省计算中心(国家超级计算济南中心) The dividing method of epithelial tissue in a kind of oesophagus pathological image

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SJORS VAN RIEL等: "AUTOMATIC DETECTION OF EARLY ESOPHAGEAL CANCER WITH CNNS USING TRANSFER LEARNING", 《2018 年第 25 届 IEEE 图像处理国际会议 (ICIP)》 *
王旭阳: "基于深度学习的食道癌图像检测技术的研究", 《中国优秀硕士学位论文全文数据库(电子期刊) 医药卫生科技刊》 *

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109805963B (en) * 2019-03-22 2022-07-05 深圳开立生物医疗科技股份有限公司 Method and system for judging endometrium typing
CN109805963A (en) * 2019-03-22 2019-05-28 深圳开立生物医疗科技股份有限公司 The determination method and system of one Endometrium parting
CN109948733A (en) * 2019-04-01 2019-06-28 深圳大学 More classification methods, sorter and the storage medium of alimentary tract endoscope image
CN110220902A (en) * 2019-05-09 2019-09-10 杭州迪英加科技有限公司 Digital pathological section analysis method and device
CN110659692A (en) * 2019-09-26 2020-01-07 重庆大学 Pathological image automatic labeling method based on reinforcement learning and deep neural network
US11417424B2 (en) 2019-10-11 2022-08-16 International Business Machines Corporation Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory and multiple instance learning
GB2604503A (en) * 2019-10-11 2022-09-07 Ibm Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory
CN114503213B (en) * 2019-10-11 2024-03-08 玛雷迪夫美国公司 Detection of disease from weakly annotated rolled medical images using convolved long-term and short-term memory
WO2021070108A1 (en) * 2019-10-11 2021-04-15 International Business Machines Corporation Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory
US11195273B2 (en) 2019-10-11 2021-12-07 International Business Machines Corporation Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory
CN114503213A (en) * 2019-10-11 2022-05-13 国际商业机器公司 Detecting disease from weakly annotated rolled medical images using convolutional long-short term memory
GB2604503B (en) * 2019-10-11 2023-12-20 Merative Us L P Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory
CN110866908B (en) * 2019-11-12 2021-03-26 腾讯科技(深圳)有限公司 Image processing method, image processing apparatus, server, and storage medium
CN110866908A (en) * 2019-11-12 2020-03-06 腾讯科技(深圳)有限公司 Image processing method, image processing apparatus, server, and storage medium
CN111080654A (en) * 2019-11-18 2020-04-28 中国科学院深圳先进技术研究院 Image lesion region segmentation method and device and server
WO2021097595A1 (en) * 2019-11-18 2021-05-27 中国科学院深圳先进技术研究院 Method and apparatus for segmenting lesion area in image, and server
CN111047559A (en) * 2019-11-21 2020-04-21 万达信息股份有限公司 Method for rapidly detecting abnormal area of digital pathological section
CN111062448A (en) * 2019-12-24 2020-04-24 北京知道创宇信息技术股份有限公司 Equipment type recognition model training method, equipment type recognition method and device
CN111062448B (en) * 2019-12-24 2024-05-28 北京知道创宇信息技术股份有限公司 Equipment type recognition model training method, equipment type recognition method and device
CN111462076A (en) * 2020-03-31 2020-07-28 湖南国科智瞳科技有限公司 Method and system for detecting fuzzy area of full-slice digital pathological image
CN115294126A (en) * 2022-10-08 2022-11-04 南京诺源医疗器械有限公司 Intelligent cancer cell identification method for pathological image
CN115294126B (en) * 2022-10-08 2022-12-16 南京诺源医疗器械有限公司 Cancer cell intelligent identification method for pathological image

Also Published As

Publication number Publication date
CN109410194B (en) 2022-03-29

Similar Documents

Publication Publication Date Title
CN109410194A (en) A kind of cancer of the esophagus pathology image processing method based on deep learning
CN109191457B (en) Pathological image quality validity identification method
CN109670510A (en) A kind of gastroscopic biopsy pathological data screening system and method based on deep learning
CN112488234B (en) End-to-end histopathology image classification method based on attention pooling
CN110675411B (en) Cervical squamous intraepithelial lesion recognition algorithm based on deep learning
CN109389584A (en) Multiple dimensioned rhinopharyngeal neoplasm dividing method based on CNN
CN110097974A (en) A kind of nasopharyngeal carcinoma far-end transfer forecasting system based on deep learning algorithm
CN110136149A (en) Leucocyte positioning and dividing method based on deep neural network
CN110111895A (en) A kind of method for building up of nasopharyngeal carcinoma far-end transfer prediction model
CN109740626A (en) The detection method of cancerous area in breast cancer pathological section based on deep learning
CN106096654A (en) A kind of cell atypia automatic grading method tactful based on degree of depth study and combination
Tang et al. Segnet-based gland segmentation from colon cancer histology images
CN111798425B (en) Intelligent detection method for mitotic image in gastrointestinal stromal tumor based on deep learning
CN113223005B (en) Thyroid nodule automatic segmentation and grading intelligent system
CN110415230A (en) A kind of CT sectioning image semantic segmentation system and method based on deep learning
CN112263217B (en) Improved convolutional neural network-based non-melanoma skin cancer pathological image lesion area detection method
CN111104961A (en) Method for classifying breast cancer based on improved MobileNet network
CN112529892A (en) Digestive tract endoscope lesion image detection method, digestive tract endoscope lesion image detection system and computer storage medium
CN113538435A (en) Pancreatic cancer pathological image classification method and system based on deep learning
CN115471701A (en) Lung adenocarcinoma histology subtype classification method based on deep learning and transfer learning
CN114398979A (en) Ultrasonic image thyroid nodule classification method based on feature decoupling
Leon et al. Supervised classification of histopathological images using convolutional neuronal networks for gastric cancer detection
CN112233085A (en) Cervical cell image segmentation method based on pixel prediction enhancement
CN112884737B (en) Automatic mitosis detection method in breast cancer pathological image based on multistage iteration
CN113420793A (en) Improved convolutional neural network ResNeSt 50-based gastric ring cell carcinoma classification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant