CN109671053A - A kind of gastric cancer image identification system, device and its application - Google Patents
A kind of gastric cancer image identification system, device and its application Download PDFInfo
- Publication number
- CN109671053A CN109671053A CN201811371788.1A CN201811371788A CN109671053A CN 109671053 A CN109671053 A CN 109671053A CN 201811371788 A CN201811371788 A CN 201811371788A CN 109671053 A CN109671053 A CN 109671053A
- Authority
- CN
- China
- Prior art keywords
- image
- frame
- lesion
- layer
- candidate region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30092—Stomach; Gastric
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of gastric cancer image identification system, device and its application, which includes data input module, data preprocessing module, image recognition model construction module and lesion identification module；Self training may be implemented in the system simultaneously, to accurately identify the diseased region in gastric cancer image.
Description
Technical field
The invention belongs to medical domain, the technology that pathological image identification is realized using image identification system is more particularly related to
Field.
Background technique
Although the disease incidence of gastric cancer was gradually reduced from 1975, still there are within 2012 nearly 1,000,000 new cases (total
951000, the 6.8% of Zhan Suoyou cancer morbidity), make the fifth-largest most common malignant tumour in the world.Wherein, have
Case more than 70% appears in developing country, and has half to occur East Asia (mainly in China).In terms of the death rate, stomach
Cancer is the big cancer cause of the death in third place in the world (totally 723000 death accounts for the 8.8% of general mortality rate).
The prognosis of gastric cancer largely depends on its disagreement.Some researches show that 5 years survival rates of stomach morning cancer almost to surpass
90% is crossed, and the survival rate of advanced gastric carcinoma is lower than 20%.So high risk suffer from cancer crowd early detection and rule with
Examining is the most effective means for reducing incidence gastric cancer rate, improving survival.
Since the mistaken diagnosis of common white light endoscopic diagnosis gastric cancer (especially superficial flat and depressed lesion), rate of missed diagnosis are quite high, respectively
Kind endoscopic diagnosis technology is come into being.But the superb operation skill not only needed using these endoscopic assistances, it is also necessary to can
The economic support of sight.Therefore, be badly in need of researching and developing a kind of discovery, Diagnosis of Gastric morning cancer and precancerous lesion it is simple and easy to get, economical and practical simultaneously
And safe and reliable diagnostic techniques.
Summary of the invention
Inventor, in order to reduce various problems brought by artificial endoscopic diagnosis, utilizes machine in long-term medical practice
Device learning art by repeatedly developing, optimizes and training obtains the system that can be used for diagnosing gastric cancer repeatedly, is aided with system and tight
The optical sieving of lattice and pretreatment further improve trained efficiency.Diagnostic system of the invention can accurately be known very much
Cancerous lesion position in other pathological image (such as gastroscope picture and realtime graphic), it is special that discrimination even has been over internal medicine
Family doctor.
The first aspect of the invention provides a kind of gastric cancer image identification system comprising:
A, data input module, for inputting the image comprising gastric cancer diseased region, described image is preferably endoscope figure
Picture；
B, data preprocessing module, for receiving the image from data input module, and accurate frame selects the lesion of gastric cancer
Position, the part in frame choosing is defined as positive sample, and the part outside frame choosing is defined as negative sample, and exports diseased region
Coordinate information and/or lesion type information；It is preferred that the module also carries out desensitization process to image in advance before frame choosing, go
Except sufferer personal information；
Preferably, the frame choosing can generate a rectangle frame or square-shaped frame comprising lesions position；The coordinate letter
Breath is preferably the coordinate information of the point in the upper left corner and lower right corner of the rectangle frame or square-shaped frame；
It is also preferred that frame selects position to be determined by following methods: 2n scope doctors carry out frame choosing in a manner of " back-to-back ", i.e.,
2n people is randomly divided into n group, 2 people/group, while all images are randomly divided into n parts, and is randomly assigned to each group doctor and carries out frame
Choosing；When frame choosing after the completion of, compare every group of two doctors frame choosing as a result, and to frame between two doctors select the consistency of result into
Row assessment, final determination block selects position, wherein natural number of the n between 1-100, for example, 1,2,3,4,5,6,7,8,9,10,20,
30,40,50,60,70,80,90 or 100；
It is further preferred that described as follows to the standard that frame selects the consistency of result to be assessed between two doctors:
For each lesion picture, the frame for comparing every group of two doctors selects the overlapping area of result, if every group two
The 50% of the area that the union that the area (i.e. intersection) that doctor distinguishes the position lap of frame choosing is greater than the rwo is covered, then
Think that the frame of two doctors selects judging result consistency good, and by the corresponding diagonal line coordinates of above-mentioned intersection, i.e., the upper left corner and
The coordinate of the point in the lower right corner saves as the final positioning of target lesion；
If the 50% of the area that the union that the area (i.e. intersection) of lap is less than the rwo is covered, then it is assumed that two
The frame of doctor selects judging result difference larger, and such lesion picture is individually picked out, and the 2n of work is selected by all participation frames
Position doctor discusses the final position for determining target lesion jointly；
C, image recognition model construction module can be received through data preprocessing module treated image, for constructing
And training image recognition model neural network based, the neural network is preferably convolutional neural networks；
D, lesion identification module is known for image to be checked to be input to the image recognition model after training, and based on image
With the presence or absence of the position of lesion and/or lesion in the output result judgement image to be checked of other model.
In one embodiment, described image identification model building module includes feature extractor, candidate region generation
Device and target marker, in which:
The feature extractor is used to carry out feature extraction to the image from data preprocessing module to obtain feature
Figure, it is preferred that the feature extraction is carried out by convolution operation；
The candidate region generator is used to generate several candidate regions based on the characteristic pattern；
The target marker calculates the classification score of the candidate region, and the score indicates that the region belongs to the sun
The probability of property sample and/or the negative sample；Target marker can propose to adjust to the bezel locations in each region simultaneously
Value so that the bezel locations for each region are adjusted, and then accurately determines lesions position；Preferably, the classification score
With loss function (Loss function) has been used in the training of adjusted value；
It is also preferred that using the gradient descent method based on mini-batch, i.e., being instructed to each when carrying out described trained
Practice picture and generates the mini-batch comprising multiple positive and negative candidate regions；The then random sampling from every picture
256 candidate regions then calculate corresponding mini- until the ratio of positive candidate region and negative candidate region is close to 1:1
The loss function of batch；If the quantity of positive candidate region is less than 128 in a picture, go to fill out with feminine gender candidate region
Mend this mini-batch；
It is further preferred that 0.001 is set by the learning rate of preceding 50000 mini-batch, 50000 by after
The learning rate of mini-batch is set as 0.0001；Momentum term is preferably arranged to 0.9, and weight decaying is preferably arranged to 0.0005.
In another embodiment, wherein the feature extractor can be to the arbitrary dimension and/or resolution ratio of input
Image carry out feature extraction, described image can be original image size and/or resolution ratio, be also possible to change size and/or point
The image inputted after resolution obtains the characteristic pattern of multidimensional (such as 256 dimensions or 512 dimensions)；
Specifically, the feature extractor includes X convolutional layer and Y sample level, wherein i-th (i is between 1-X) volume
Lamination includes a QiIt is a having a size of m*m*piConvolution kernel, wherein m*m indicate convolution kernel length and wide pixel value, piEqual to upper
The convolution nuclear volume Q of one convolutional layeri-1, in i-th of convolutional layer, convolution kernel is with step-length L to the data (example from upper level
Such as original image, (i-1)-th convolutional layer or sample level) carry out convolution operation；Each sample level includes 1 mobile with step-length 2L,
Size is the convolution kernel of 2L*2L, carries out convolution operation to the image of convolutional layer input；Wherein, it is carried out by feature extractor special
After sign is extracted, the final characteristic pattern for obtaining Qx dimension；
Wherein X is between 1-20, for example, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19
Or 20；Y is between 1-10, such as 1,2,3,4,5,6,7,8,9 or 10；M between 2-10, such as 2,3,4,5,6,7,8,9 or
10；P between 1-1024, Q between 1-1024, the numerical value of p or Q respectively such as 1,2,3,4,5,6,7,8,9,10,11,12,
13,14,15,16,32,64,128,256,512 or 1024.
In another embodiment, wherein sliding window is arranged in the characteristic pattern in the candidate region generator,
The size of sliding window is n × n, such as 3 × 3；Slide sliding window along characteristic pattern, simultaneously for every where sliding window
There are corresponding relationships for corresponding position in one position, central point and original image, and in original image centered on the corresponding position
It is middle to generate the k candidate regions with different scale and length-width ratio；Wherein, if k candidate region has x kind (such as 3
Kind) different scale and length-width ratio, then k=x2(such as k=9).
In another embodiment, the target marker includes middle layer again, classify layer and frame recurrence layer, wherein
Middle layer is used to map sliding window operation and is formed by the data of candidate region, be multidimensional (such as 256 dimensions or 512 dimensions) to
Amount；
Classification layer and frame return layer and connect respectively with middle layer, and classification layer is for determining that the object candidate area is prospect
(i.e. positive sample) or background (i.e. negative sample), frame return the x coordinate and y seat that layer is used to generate candidate region central point
Mark and the wide w and high h of candidate region.
The second aspect of the invention provides a kind of identification device of gastric cancer image, including is stored with diagnosing gastric cancer figure
The storage unit of picture, image preprocessing program and trainable image recognition program, it is also preferable to include arithmetic elements and display
Unit；
Described device, which can utilize the image recognition program of the image comprising gastric cancer lesion to be trained, (preferably has supervision
Training), so that the image recognition program after training be enable to identify gastric cancer diseased region in image to be checked；
Preferably, the image to be checked is scope photo or real-time imaging.
In one embodiment, wherein described image preprocessor accurate frame choosing in the diagnosing gastric cancer image
The diseased region of gastric cancer, the part in frame choosing is defined as positive sample, and the part outside frame choosing is defined as negative sample, and exports
The location coordinate information and/or lesion type information of lesion；It is preferred that also carrying out desensitization process before frame choosing to image in advance, going
Except sufferer personal information；
Preferably, the frame choosing can generate a rectangle frame or square-shaped frame comprising lesions position；The coordinate letter
Breath is preferably the coordinate information of the point in the upper left corner and the lower right corner；
It is also preferred that frame selects position to be determined by following method: 2n scope doctors carry out frame choosing in a manner of " back-to-back ", i.e.,
2n people is randomly divided into n group, 2 people/group, while all images are randomly divided into n parts, and is randomly assigned to each group doctor and carries out frame
Choosing；When frame choosing after the completion of, compare every group of two doctors frame choosing as a result, and to frame between two doctors select the consistency of result into
Row assessment, final determination block selects position, wherein natural number of the n between 1-100, for example, 1,2,3,4,5,6,7,8,9,10,20,
30,40,50,60,70,80,90 or 100；
It is further preferred that described as follows to the standard that frame selects the consistency of result to be assessed between two doctors:
For each lesion image, the frame for comparing every group of 2 doctors selects the overlapping area of result, if every group of two doctors
The 50% of the area that the union that the area (i.e. intersection) that teacher distinguishes the position lap of frame choosing is greater than the rwo is covered, then recognize
It selects judging result consistency good for the frame of 2 doctors, and the corresponding diagonal line coordinates of above-mentioned intersection is saved as into target lesion
Final positioning；
If the 50% of the area that the union that the area (i.e. intersection) of lap is less than the rwo is covered, then it is assumed that 2
The frame of doctor selects judging result difference larger, and such lesion picture is individually picked out, and the 2n of work is selected by all participation frames
Position doctor discusses the final position for determining target lesion jointly.
In another embodiment, described image recognizer is trainable image recognition journey neural network based
Sequence, the neural network are preferably convolutional neural networks；Preferably, described image recognizer includes feature extractor, candidate
Region generator and target marker, in which:
The feature extractor is used to carry out feature extraction to image to obtain characteristic pattern, it is preferred that the feature mentions
It takes and is carried out by convolution operation；
The candidate region generator is used to generate several candidate regions based on the characteristic pattern；
The target marker calculates the classification score of the candidate region, and the score indicates that the region belongs to the sun
The probability of property sample and/or the negative sample；Target marker can propose to adjust to the bezel locations in each region simultaneously
Value, so that the bezel locations for each region are adjusted, to accurately determine lesions position；Preferably, the classification score
With loss function (Loss function) has been used in the training of adjusted value；
In another embodiment, wherein when carrying out described trained, declined using the gradient based on mini-batch
Method generates the mini-batch comprising multiple positive and negative candidate regions to each Zhang Xunlian picture.Then from every
256 candidate regions of random sampling are until the ratio of positive candidate region and negative candidate region is close to 1:1 in picture, then
Calculate the loss function of corresponding mini-batch.If the quantity of positive candidate region is less than 128 in a picture, with yin
Property candidate region is gone to fill up this mini-batch；
Preferably, 0.001 is set by the learning rate of preceding 50000 mini-batch, 50000 mini-batch by after
Learning rate be set as 0.0001；Momentum term is preferably arranged to 0.9, and weight decaying is preferably arranged to 0.0005.
In another embodiment, wherein the feature extractor can be to the arbitrary dimension and/or resolution ratio of input
Image carry out feature extraction, described image can be original image size and/or resolution ratio, be also possible to change size and/or point
The image inputted after resolution obtains the characteristic pattern of multidimensional (such as 256 dimensions or 512 dimensions)；
Specifically, the feature extractor includes X convolutional layer and Y sample level, wherein i-th (i is between 1-X) volume
Lamination includes a QiIt is a having a size of m*m*piConvolution kernel, wherein m*m indicate convolution kernel length and wide pixel value, piEqual to upper
The convolution nuclear volume Q of one convolutional layeri-1, in i-th of convolutional layer, convolution kernel is with step-length L to the data (example from upper level
Such as original image, (i-1)-th convolutional layer or sample level) carry out convolution operation；Each sample level includes 1 mobile with step-length 2L,
Size is the convolution kernel of 2L*2L, carries out convolution operation to the image of convolutional layer input；Wherein, it is carried out by feature extractor special
After sign is extracted, the final characteristic pattern for obtaining Qx dimension；
Wherein X is between 1-20, for example, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19
Or 20；Y is between 1-10, such as 1,2,3,4,5,6,7,8,9 or 10；M between 2-10, such as 2,3,4,5,6,7,8,9 or
10；P between 1-1024, Q between 1-1024, the numerical value of p or Q respectively such as 1,2,3,4,5,6,7,8,9,10,11,12,
13,14,15,16,32,64,128,256,512 or 1024.
In another embodiment, wherein sliding window is arranged in the characteristic pattern in the candidate region generator,
The size of sliding window is n × n, such as 3 × 3；Slide sliding window along characteristic pattern, simultaneously for every where sliding window
There are corresponding relationships for corresponding position in one position, central point and original image, and in original image centered on the corresponding position
It is middle to generate the k candidate regions with different scale and length-width ratio；Wherein, if k candidate region has x kind (such as 3
Kind) different scale and length-width ratio, then k=x2(such as k=9).
In another embodiment, the target marker includes middle layer again, classify layer and frame recurrence layer, wherein
Middle layer is used to map sliding window operation and is formed by the data of candidate region, be multidimensional (such as 256 dimensions or 512 dimensions) to
Amount；
Classification layer and frame return layer and connect respectively with middle layer, and classification layer is for determining that the object candidate area is prospect
(i.e. positive sample) or background (i.e. negative sample), frame return the x coordinate and y seat that layer is used to generate candidate region central point
Mark and the wide w and high h of candidate region.
The third aspect of the invention provides the system of first aspect of the present invention or the device of the second aspect in stomach
Purposes in the prediction and diagnosis of cancer and/or gastric precancerous lesion.
The fourth aspect of the invention provides the system of first aspect of the present invention or the device of the second aspect in stomach
Purposes in cancer image or gastric cancer image in the identification of diseased region.
The fifth aspect of the invention provides the system of first aspect of the present invention or the device of the second aspect in stomach
Purposes in the real-time diagnosis of cancer and/or gastric precancerous lesion.
The sixth aspect of the invention provides the system of first aspect of the present invention or the device of the second aspect in stomach
Purposes in cancer image or gastric cancer image in the real-time identification of diseased region.
By inventor it is long-term grope to find, due to gastric cancer diseased region there is own characteristics, i.e. diseased region not
It is enough significant, it is not clear enough with perienchyma boundary, therefore the difficulty of image recognition model training compares conventional task (as known
Not Sheng Huo in object) difficulty it is bigger, slightly will lead to training accidentally and be difficult to restrain so as to cause failure.And in the present invention
In, inventor is improved its training method and (such as is accurately defined by frame choosing by image recognition model neural network based
Target lesion position in training image, improves identification precision of image recognition model etc.), to obtain a kind of internal
The identifying system (and/or device) of the intelligent and high-efficiency identification of gastric cancer lesion in mirror picture, discrimination are higher than common scope
Doctor.Use the strengthened real-time diagnosis system of machine learning, additionally it is possible to carry out to Alimentary tract disease and its position and probability real
When monitor and identification reduces misdiagnosis rate, provide for diagnosing gastric cancer so as to significant increase general doctor to the recall rate of gastric cancer
Safe and reliable technology.
Detailed description of the invention
Fig. 1 includes the endoscopic images of gastric cancer lesions position
Fig. 2 frame selects process schematic
The lesions position for the gastric cancer that image identification system Fig. 3 of the invention is identified.
Specific embodiment
Unless otherwise stated, term used in the disclosure has generally containing for one skilled in the art's understanding
Justice.Here is the meaning of some terms in this disclosure, if having inconsistent with other definition, is subject to defined below.
Definition
Term " gastric cancer ", refers to the malignant tumour derived from gastric epithelial cell, includes early carcinoma of stomach and advanced gastric carcinoma.
Term " module " refers to that the function set that can be realized certain effects, the module can be executed only by computer,
It can also be completed together by manually performing, or by computer and manually.
Obtain lesion data
The key effect for obtaining lesion data step is to obtain the sample material for deep learning.
In one embodiment, the acquisition process can specifically include the step of acquisition and primary dcreening operation.
" acquisition " refers to that searching for acquisition in all scope databases according to the standard of " being diagnosed as gastric cancer " owns
All endoscopic diagnosis images of patient with gastric cancer, such as it is diagnosed as figure all in the affiliated file of patient of " gastric cancer "
Piece, the i.e. picture of certain patient all storages in entire endoscopic procedures, therefore be also possible that other than target site lesion
Gastrocopy picture, such as the patient is diagnosed as benign ulcer, polyp etc., but further comprises food in its file under one's name
The picture of each position storage in the checking processes such as pipe, stomach bottom, body of stomach, duodenum.
" primary dcreening operation " is the step of screening to the pathological image of the patients with gastric cancer collected, specifically can be by passing through
Scope doctor abundant is tested to describe according to the related content in case " seen in endoscopy " in combination " pathological diagnosis " to carry out
's.Since the picture for deep learning network must be that quality is clear, feature is accurate, learning difficulty increasing otherwise will lead to
Or recognition result inaccuracy.Therefore the module and/or step of lesion data primary dcreening operation can be there are clear gastric cancer lesions positions
Picture picked out from a set of inspection picture.
Importantly, primary dcreening operation can be in " pathological diagnosis " to atrophy position in conjunction with histopathological findings after patient's biopsy
Description is accurately positioned lesion, combines picture clarity, shooting angle, magnification level etc., select those clarity as far as possible
Height, magnification level is moderate, can get a glimpse of the endoscopic image of lesion overall picture.
By primary dcreening operation, it can guarantee that the picture for inputting training set is the image comprising determining diseased region of high quality,
To improve typing training data set feature accuracy, so that smart network preferably can therefrom conclude, summarize
The characteristics of image of atrophic lesion out improves accuracy rate of diagnosis.
Lesion data pretreatment
The process that accurate frame selects the lesions position of gastric cancer is completed in the pretreatment, and the part in frame choosing is defined as sun
Property sample, and the part outside frame choosing is defined as negative sample, and exports the location coordinate information and lesion type information of lesion.
In one embodiment, lesion data pretreatment is real by " image preprocessing program " institute in whole or in part
Existing.
Term " image preprocessing program " refers to the frame choosing that can be realized objective area in image, to indicate target
The program of area type and range.
In one embodiment, image preprocessing program can also carry out desensitization process to image, and removal sufferer is personal
Information.
In one embodiment, image preprocessing program is one and is able to carry out using what computer programming language was write
The software of aforementioned function.
In another embodiment, image preprocessing program is the software for being able to carry out frame and selecting function.
In a specific embodiment, executing frame selects the software of function that picture to be processed can be imported to software, and
The picture is shown in operation interface, and implementing frame selection operation personnel (such as doctor) at this time only need to be at the quasi- target lesion position outlined
Mouse is dragged along from (or other diagonal directions) direction of upper left to bottom right, to form the square for covering target lesion
Shape frame or square-shaped frame, and generate from the background simultaneously and store the accurate coordinate in the rectangle frame upper left corner and the lower right corner with uniquely fixed
Position.
In order to guarantee to pre-process the accuracy of (or frame choosing), the present invention, which is further enhanced, selects quality control to frame, this
And method/system of the invention can obtain an important guarantee of bigger accuracy, concrete mode is as follows: selection 2n
(such as 6,8,10 etc.) scope doctor carries out frame choosing in a manner of " back-to-back ", i.e. 2n people is randomly divided into n group, 2 people/group,
Simultaneously the training image after all screenings is also divided into n parts at random, and is randomly assigned to each group doctor and carries out frame choosing；When frame selects
After the completion, compare every group of 2 doctors frame choosing as a result, and select the consistency of result to assess frame between two doctors, most
Whole determination block selects position.
In one embodiment, the evaluation criterion of consistency are as follows: for same lesion picture, compare every group 2
The frame of doctor selects result namely comparison to the overlapping area of rectangle frame determined by angular coordinate, if we provide the overlapping of two rectangle frames
The 50% of the area that the union that partial area (i.e. intersection) is greater than the rwo is covered, then it is assumed that the frame of 2 doctors selects judgement
As a result consistency is good, and the corresponding diagonal line coordinates of above-mentioned intersection is saved as the final positioning of target lesion.On the contrary, if
The 50% of the area that the union that the area (i.e. intersection) of two rectangle frame laps is less than the rwo is covered, then it is assumed that 2 doctors
The frame of teacher selects judging result difference larger, then such lesion picture will individually be picked out by software backstage, later period collection
In select the doctor of work discuss the final position of determination target lesion jointly by all participation frames.
Image recognition model
Term " image recognition model " refers to the algorithm based on the building of the principle of machine learning and/or deep learning, can also
To be referred to as " trainable image recognition model " or " image recognition program ".
In one embodiment, which is a kind of neural network, and the neural network is preferably convolutional Neural net
Network；In another embodiment, the neural network is based on LeNet-5, RCNN, SPP, Fast-RCNN and/or Faster-
The convolutional neural networks of RCNN framework；Wherein faster-RCNN can regard the combination of Fast-RCNN and RPN as, in a reality
It applies in scheme, is based on faster-RCNN network.
Image recognition program includes at least following level: original image feature extraction layer, candidate region selected layer and target identification
Layer, can training parameter by the adjustment of preset algorithm.
Term " original image feature extraction layer " is to refer to pass through mathematical computations to training image to what is inputted to multidimensional
Degree extracts the level or level combination of original image information.The layer can actually indicate the combination of multiple and different functional layers.
In one embodiment, original image feature extraction layer can be based on ZF or VGG16 network.
Term " convolutional layer ", refers in original image feature extraction layer, is responsible for original input picture or passes through sample level
Image information that treated carries out convolution operation, to extract the network layer of information.The convolution operation is indeed through one
The convolution kernel (such as 3*3) of a particular size is slided on the image of input with certain step-length (such as 1 pixel), in convolution kernel
The pixel on picture is multiplied with the respective weights of convolution kernel in mobile process, finally by all product additions obtain one it is defeated
It realizes out.In image procossing, often image is expressed as the vector of pixel, therefore a secondary digital picture is considered as one
The discrete function of a two-dimensional space, such as it is expressed as f (x, y), it is assumed that have for two-dimensional convolution handling function C (u, v), then can produce
Raw output image g (x, y)=f (x, y) * C (u, v), may be implemented using convolution to image Fuzzy Processing and information extraction.
Term " training " refers to by inputting largely by the samples that manually mark, to trainable image recognition program into
The self-regulated repeatedly of row parameter realizes the diseased region in identification gastric cancer image to realize expected purpose.
In one embodiment, the present invention is based on faster-rcnn networks, and are arrived in step s 4 using following end
The training method at end:
(1) parameter of network (RPN) is generated using the model initialization object candidate area of the pre-training on ImageNet,
And the network is finely adjusted；
(2) the model initialization Fast R-CNN network parameter for equally using the pre-training on ImageNet, followed by
(1) the region proposal that RPN network extracts in is trained；
(3) the Fast R-CNN network of (2) is used to reinitialize RPN, fixed convolutional layer finely tunes RPN network, wherein only
Cls and/or reg layers of RPN in fine tuning；
(4) convolutional layer for fixing Fast R-CNN in (2), uses the region proposal couple that RPN is extracted in (3)
Fast R-CNN network is finely adjusted, wherein only finely tuning the full articulamentum of Fast R-CNN.
Term " candidate region selected layer ": refer to by algorithm realization select on the original image specific region for point
Class identification and frame return level or level combination, it is similar with original image feature extraction layer, the layer can also indicate it is multiple not
The combination of same layer.
Candidate region selected layer is directly connected to for original input layer in one embodiment.
In one embodiment, candidate region selected layer and the last layer of original image feature extraction layer are directly connected to.
In one embodiment, " candidate region selected layer " can be based on RPN.
Term " target identification layer " term " sample level " can sometimes be called pond layer, and operation is similar to convolutional layer,
Only the convolution kernel of sample level is only to take maximum value, average value of corresponding position etc. (maximum pond, average pond).
Term " characteristic pattern ", is also feature map, refers to and carries out convolution to original image image by original image feature extraction layer
The high-dimensional multichannel image of the small area obtained after operation leads to as an example, characteristic pattern can be 256 that scale is 51*39
Road image.
Term " sliding window " refers to the window of the small size (such as 2*2,3*3) generated on characteristic pattern, along characteristic pattern
Each position it is mobile, although characteristic pattern size is also and less, since the data that characteristic pattern has already passed through multilayer are extracted
(such as convolution), therefore the bigger visual field can be realized using lesser sliding window on characteristic pattern.
Term " candidate region ", is referred to as candidate window, object candidate area, reference box, bounding
Box can also be replaced also in this context with anchor or anchor box.
In one embodiment, it is positioned first by sliding window to a position of characteristic pattern, it is raw for the position
At the rectangular or square window of k different area different proportion, such as 9, and it is anchored to the center of the position, therefore also cry
Anchor or anchor box, and the relationship based on the center of each sliding window and original image in characteristic pattern are done, is formed candidate
Region, the candidate region substantially may be considered original corresponding to the sliding window (3*3) moved on the last layer convolutional layer
Graph region range.
In one embodiment of the invention, k=9 includes the following steps: when generating candidate region
(1) 9 kinds of anchor box are generated first, in accordance with different area and length-width ratio, the anchor box not with characteristic pattern or
The size of person's original input picture changes；
(2) for every input picture, according to image size calculate each sliding window corresponding to original image central point；
(3) mapping relations of sliding window position and original image position are established based on above-mentioned calculating.
Term " middle layer " refers to after forming object candidate area using sliding window, characteristic pattern is further mapped to one
In the vector of multidimensional (such as 256 dimensions or 512 dimensions), this layer can be considered as to a new level, be referred to as in the present invention as centre
Layer.Link sort layer and window return layer after middle layer.
Term " classification layer " (cls_score), exports a branch connecting with middle layer, which can export 2k
A score respectively corresponds two scores of k object candidate area, one of them is prospect (i.e. positive sample) score, and one
It is background (i.e. negative sample) score, this score may determine that the object candidate area is real target or background.Cause
, for each sliding window position, layer of classifying output can belong to prospect (i.e. positive sample from high-dimensional (such as 256 dimensions) feature for this
This) and background (i.e. negative sample) probability.
Specifically, in one embodiment, when candidate region and any (authentic specimen side ground-truth box
Boundary, that is, boundary of the object for needing to identify in original image) IOU (hand over and compare) to be greater than 0.7 be to be considered the positive
Sample or positive label, when the IOU of candidate region and any ground-truth box are less than 0.3, then it is assumed that be that background is (i.e. negative
Property sample), to be assigned with class label to each anchor.Wherein IOU from mathematics containing above represent candidate region with
The degree of overlapping of ground-truth box, calculation method are as follows:
IOU=(A ∩ B)/(A ∪ B)
Classification layer can export k+1 dimension group p, indicate the probability for belonging to k class and background.To each RoI (Region of
Interesting discrete type probability distribution) is exported, p is then calculated by the full articulamentum of k+1 class using softmax.Mathematical table
Up to as follows:
P=(p0, p1..., pk)
Term " window recurrence layer " (bbox_pred) exports another branch connecting with middle layer, simultaneously with classification layer
Column.The layer can export on each position, and 9 anchor, which correspond to window, should translate the parameter of scaling.Respectively correspond k
Object candidate area, each object candidate area have 4 bezel locations adjusted values, this 4 bezel locations adjusted values refer to mesh
Mark the x in the upper left corner of candidate regionaCoordinate, yaThe high h of coordinate and object candidate areaaWith wide waAdjusted value.The work of the branch
With being finely adjusted to object candidate area position, keep the final result position more accurate.
Window, which returns layer, can export the displacement of bounding box recurrence, export 4*K dimension group t, indicate to be belonging respectively to
When k class, it should translate the parameter of scaling.Mathematical expression is as follows:
K indicates the index of classification,Refer to the translation relative to object proposal Scale invariant,
Refer to the Gao Yukuan in log space relative to object proposal.
In one embodiment, the present invention is realized by loss function (Loss function) to classification layer and window
Training while returning layer, the function be by classification loss (i.e. classification layer softmax loss) and
Regression loss (i.e. L1loss) is by certain weight proportion composition.:
Calculate calibration result and prediction result that softmax loss needs candidate region to correspond to ground truth；It calculates
Regression loss needs three group informations:
(1) predicting candidate regional center position coordinates x, y and width high w, h；
(2) each of 9 anchor point reference boxes in candidate region periphery center position coordinate xa, yaAnd width
High wa, ha。
(3) frame (ground truth) corresponding center position coordinate x*, y* and width high w*, h* are really demarcated.
It calculates regression loss and total Loss mode is as follows:
tx=(x-xa)/wa, ty=(y-ya)/ha,
tw=log (w/wa), th=log (h/ha),
Wherein, piThe probability of target is predicted as anchor.
There are two numerical value,Be negative label equal to 0,It is positive label equal to 1.
tiIndicate the vector set of 4 parametrization coordinates of the candidate region of prediction.
Indicate the coordinate vector of the corresponding ground truth bounding box of postive anchor.
In one embodiment, in the training of loss function, using the gradient descent method based on mini-batch, i.e.,
One mini-batch comprising multiple positive and negative candidate regions (anchor) is generated to each Zhang Xunlian picture.Then from
256 anchor of random sampling are until the ratio of positive anchor and feminine gender anchor are close to 1:1, then calculating in every picture
The loss function (Loss function) of corresponding mini-batch.If the quantity of positive anchor is less than 128 in a picture
It is a, then it goes to fill up this mini-batch with feminine gender anchor.
In a specific embodiment, set 0.001 for the learning rate of preceding 50000 mini-batch, will after
The learning rate of 50000 mini-batch is set as 0.0001；Momentum term is preferably arranged to 0.9, and weight decaying is preferably arranged to
0.0005。
After above-mentioned training, by the scope picture of the target lesion for identification of the deep learning network after training.One
In a embodiment, classification scoring is set to 0.85, i.e. deep learning network validation lesion probability is more than 85% lesion
It can be labeled out, so that the picture is judged as the positive；On the contrary, if not detecting suspicious diseased region in a picture
Domain, then this picture is just judged as feminine gender.
Embodiment
1. exempting informed consent statement:
(1) the scope figure that this research obtains in previous clinic diagnosis merely with Gastroenterology dept., Beijing Friendship Hospital endoscope center
Piece and relevant clinical data carry out retrospective observational study, will not make to conditions of patients, treatment, prognosis even life security
At any influence；
(2) all data collection tasks are individually completed by one people of principal investigator, and after the completion of image data acquisition, stood
All pictures are carried out to erase personal information processing using special software, it is ensured that in subsequent doctor screening, frame choosing and artificial
In intelligence programming expert typing training, debugging and test process, the leakage of patients' privacy information is not caused；
(3) in Gastroenterology dept.'s endoscope center electronic health record inquiry system, and not set " contact method " or " home address " etc.
Entry can show, i.e. the contact details of the system not typing patient, therefore this research can not trace back to and be included in patient's signature and know
Letter of consent.
2. pathological image acquires
Inclusion criteria:
(1) it is terminated in the receiving of Beijing Friendship Hospital's digestive endoscopy center from January 1st, 2013 on June 10th, 2017
The trouble of spectroscopy (including electronic gastroscope, electronic colonoscope, endoscopic ultrasonography, electron stain scope, magnifying endoscope and Endoscopy)
Person；
(2) under mirror diagnose " gastric cancer " (including and do not distinguish early carcinoma of stomach and advanced gastric carcinoma) patient；
Exclusion criteria:
(1) malignant tumor of digestive tract involves that position is extensive or indefinite person；
(2) pancreatobiliary malignant tumour person is only suffered from；
(3) merge other systems malignant tumour person simultaneously；
(2) scope picture is unintelligible and/or the undesirable person of shooting angle.
3, experiment flow and result
(1) data acquire: being found out from Gastroenterology dept., Beijing Friendship Hospital endoscope center electronic medical record system by researcher
Receive between on June 10,1 day to 2017 January in 2013 endoscopy (including electronic gastroscope, electronic colonoscope, ultrasound in
Mirror, electron stain scope, magnifying endoscope and Endoscopy), and be diagnosed as " gastric cancer " under mirror and (including and do not distinguish early carcinoma of stomach
And advanced gastric carcinoma) patient scope picture and relevant clinical data；
(2) it erases personal information: all pictures being carried out erasing personal information processing immediately after the completion of acquisition.
(3) picture screens: treated that picture is finished to all, has filtered out clear pathological examination and has been confirmed as stomach
Scope picture corresponding to the case of cancer, and according to biopsy pathology position, finishing screen is selected in each case comprising target lesion
The few picture of the clear of position, background interference amounts to 3774；
(4) construct test data set: test totally 100, picture includes " gastric cancer " (early carcinoma of stomach of pathological examination confirmation
With advanced gastric carcinoma) 50, then " the Non-cancerous disease of the another random acquisition in the database stomach that there is pathological examination to confirm
Change " (including gastric benign ulcer, polyp, mesenchymoma, lipoma, ectopic pancreas) scope picture 50 is opened.Concrete operations include:
50 are randomly selected from all gastric cancer pictures that step (3) filters out first；
" nonneoplastic lesion " for the stomach that another random acquisition in the database has pathological examination to confirm again
(including gastric benign ulcer, polyp, mesenchymoma, lipoma, ectopic pancreas) scope picture 50 is opened, and immediately to above-mentioned
50 pictures carry out erasing personal information processing；
(5) it constructs training dataset: from the gastric cancer picture that step (3) filters out, excluding random selection in step (4) and use
In the picture of building test data set, remaining 3724 are used for deep learning network training, thus composing training data set；
(6) frame selects target lesion: 6 scope doctors are randomly divided into 3 groups in a manner of " back-to-back ", by 6 people, 2 people/group；Institute
Training picture after having screening is divided into 3 parts at random, and is randomly assigned to each group doctor and carries out frame choosing.Lesion frame selects the reality of step
It applies based on the software voluntarily write, the software can show this in operation interface after picture to be processed capable of being imported software
Picture, doctor need to drag mouse along from the direction of upper left to bottom right at the quasi- target lesion position outlined at this time, to be formed
One covers the rectangle frame of target lesion, and generates and store the accurate coordinate in the rectangle frame upper left corner and the lower right corner from the background simultaneously
Uniquely to position.
After the completion of frame choosing, the frame choosing of every group of 2 doctors is compared as a result, comparing diagonal sit for same lesion picture
The overlapping area of rectangle frame determined by marking, after test, if the area of two rectangle frame lap of final determination (is handed over
Collection) it is greater than the 50% of the area that the rwo union is covered, then it is assumed that and the frame of 2 doctors selects judging result consistency good, and
And the corresponding diagonal line coordinates of above-mentioned intersection is saved as into the final positioning of target lesion.If opposite two rectangle frame laps
The 50% of the area that the union that area (i.e. intersection) is less than the rwo is covered, then it is assumed that the frame of 2 doctors selects judging result phase
Difference is larger, then such lesion picture will individually be picked out by software backstage (or handmarking), later period concentration by
All frames that participate in select the doctor of work to discuss the final position for determining target lesion jointly.
(7) typing training: the picture typing that the framed choosing of above-mentioned institute is completed is based in faster-rcnn convolutional neural networks
It is trained, and tests two kinds of network structures of ZF and VGG16；Training is by the way of end-to-end；
Wherein ZF network connects layer and a softmax classification output layer, VGG16 network tool with 5 convolutional layers, 3 entirely
Have 13 convolutional layers, 3 connect layer and softmax classification output layer entirely, under the frame of Faster-RCNN, ZF and
VGG16 model is the basic CNN for extracting training image feature.
When training, using the gradient descent method based on mini-batch, i.e., generating one to each Zhang Xunlian picture includes
The mini-batch of multiple positive and negative candidate regions (anchor).Then random sampling 256 from every picture
Anchor then calculates the loss of corresponding mini-batch until the ratio of positive anchor and feminine gender anchor are close to 1:1
Function (Loss function).If the quantity of positive anchor is less than 128 in a picture, go to fill out with feminine gender anchor
Mend this mini-batch.
0.001 is set by the learning rate of preceding 50000 mini-batch, by the study of rear 50000 mini-batch
Rate is set as 0.0001；Momentum term is preferably arranged to 0.9, and weight decaying is preferably arranged to 0.0005.
The loss function (Loss Function) used in training is as follows:
In above formula, i represents the index of anchor in each batch, piRepresent whether anchor is target (Object)
Probability；pi* be the true tag of the anchor: when anchor is that then label is 1 to Object, on the contrary then label is 0.tiIt is one 4
Dimensional vector respectively indicates the parametrization coordinate of bounding box, and ti* it then indicates in bounding box regression forecasting
Bounding box parametrization coordinate label.
(8) test and result statistics: using test data set (including 50 gastric cancers and 50 stomaches " nonneoplastic lesion "
Picture), artificial intelligence system, Gastroenterology dept. doctor of different years are tested respectively, compares, evaluate the two in terms of diagnosis
The indexs such as sensibility, specificity, accuracy rate, consistency, and carry out statistical analysis.In test, by the depth after training
Practise network for identification the scope picture of target lesion when classification scoring be set as 0.85, i.e. deep learning network validation lesion
Probability is more than that 85% lesion just can be labeled out, so that the picture is judged as the positive；On the contrary, if not having in a picture
Have and detect suspicious lesion region, then this picture is just judged as feminine gender.
As a result as follows:
Based on the platform of national digestive disease Clinical Research Center, share under gastric cancer scope in pathological changes diagnosis test, 89
The sensibility that name participates in doctor's totality is fluctuated in 48%~100% range, median 88%, average sensitivity 87%；
In 10%~98% range, (its median 78%, average specificity is 74%), accuracy rate is then fluctuated in 51% for specificity fluctuation
~91% range (its median 82%, Average Accuracy 80%).And the identification sensibility of deep learning network model diagnosis
It is 90%, specificity is 50%, accuracy rate 70%.Therefore in terms of the diagnosing gastric cancer based on gastroscope picture, artificial intelligence is quick
Perception is higher than overall physician level, but specificity is relatively low relative to Median levels, and accuracy rate is also slightly less than the middle position of doctor
Number is horizontal, it is contemplated that deep learning network model diagnostic model has splendid stability in identification, and difference is cured
Teacher has greatly fluctuation and unstability in terms of specificity, accuracy rate, therefore still can using artificial intelligent recognition lesion
Enough effective doctor's individual difference brings that excludes diagnose deviation, thus have a good application prospect.
Wherein, sensibility is also referred to as susceptibility (sensitivity, SEN), also known as true positive rate (true positive
Rate, TPR), i.e., actual diseased is diagnosed the percentage that standard is correctly diagnosed again.
Specificity, also referred to as specificity (specificity, SPE), also known as true negative rate (true negative rate,
TNR), the ability that Screen test determines non-patient is reflected.
Accuracy rate=the individual sum correctly identified/individual sum identified.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention
It is interior.
Claims (16)
1. a kind of gastric cancer image identification system comprising:
A, data input module, for inputting the image comprising gastric cancer diseased region, described image is preferably endoscopic images；
B, data preprocessing module, for receiving the image from data input module, and accurate frame selects the diseased region of gastric cancer,
Part in frame choosing is defined as positive sample, and the part outside frame choosing is defined as negative sample, and exports the seat of diseased region
Mark information and/or lesion type information；It is preferred that the module also carries out desensitization process, removal disease to image in advance before frame choosing
Suffer from personal information；
Preferably, the frame choosing can generate a rectangle frame or square-shaped frame comprising diseased region；The coordinate information is excellent
It is selected as the coordinate information in the upper left corner of the rectangle frame or square-shaped frame and the point in the lower right corner；
It is also preferred that the position of frame choosing is determined by following methods: 2n scope doctors carry out frame choosing in a manner of " back-to-back ", i.e., will
2n people is randomly divided into n group, 2 people/group, while all images are randomly divided into n parts, and is randomly assigned to each group doctor and carries out frame
Choosing；When frame choosing after the completion of, compare every group of two doctors frame choosing as a result, and to frame between two doctors select the consistency of result into
Row assessment, final determination block selects position, wherein natural number of the n between 1-100, for example, 1,2,3,4,5,6,7,8,9,10,20,
30,40,50,60,70,80,90 or 100；
It is further preferred that described as follows to the standard that frame selects the consistency of result to be assessed between two doctors:
For each lesion picture, the frame for comparing every group of two doctors selects the overlapping area of result, if every group of two doctors
The 50% of the area that the union that the area (i.e. intersection) of the position lap of frame choosing is greater than the rwo respectively is covered, then it is assumed that
The frame of two doctors selects judging result consistency good, and by the corresponding diagonal line coordinates of above-mentioned intersection, the i.e. upper left corner and bottom right
The coordinate of the point at angle saves as the final positioning of target lesion；
If the 50% of the area that the union that the area (i.e. intersection) of lap is less than the rwo is covered, then it is assumed that two doctors
Frame select judging result difference it is larger, such lesion picture is individually picked out, and selects the position 2n of work to cure by all participation frames
Shi Gongtong discusses the final position for determining target lesion；
C, image recognition model construction module can be received through data preprocessing module treated image, for constructing and instructing
Practice image recognition model neural network based, the neural network is preferably convolutional neural networks；
D, lesion identification module for image to be checked to be input to the image recognition model after training, and is based on image recognition mould
With the presence or absence of the position of lesion and/or lesion in the output result judgement image to be checked of type.
2. system according to claim 1, it includes feature extractor, candidate region that described image identification model, which constructs module,
Generator and target marker, in which:
The feature extractor is used to carry out feature extraction to the image from data preprocessing module to obtain characteristic pattern, excellent
Choosing, the feature extraction is carried out by convolution operation；
The candidate region generator is used to generate several candidate regions based on the characteristic pattern；
The target marker calculates the classification score of the candidate region, and the score indicates that the region belongs to the positive sample
The probability of this and/or the negative sample；Target marker can propose adjusted value to the bezel locations in each region simultaneously, from
And it is adjusted for the bezel locations in each region, and then accurately determine lesions position；Preferably, the classification score and adjustment
Loss function (Loss function) has been used in the training of value；
It is also preferred that using the gradient descent method based on mini-batch, i.e., scheming to each Zhang Xunlian when carrying out described trained
Piece generates the mini-batch comprising multiple positive and negative candidate regions；The then random sampling 256 from every picture
A candidate region then calculates corresponding mini- until the ratio of positive candidate region and negative candidate region is close to 1:1
The loss function of batch；If the quantity of positive candidate region is less than 128 in a picture, go to fill out with feminine gender candidate region
Mend this mini-batch；
It is further preferred that 0.001 is set by the learning rate of preceding 50000 mini-batch, 50000 mini- by after
The learning rate of batch is set as 0.0001；Momentum term is preferably arranged to 0.9, and weight decaying is preferably arranged to 0.0005.
3. system according to claim 2, wherein the feature extractor being capable of arbitrary dimension to input and/or resolution
The image of rate carries out feature extraction, and described image can be original image size and/or resolution ratio, be also possible to change size and/or
The image inputted after resolution ratio obtains the characteristic pattern of multidimensional (such as 256 dimensions or 512 dimensions)；
Specifically, the feature extractor includes X convolutional layer and Y sample level, wherein i-th of (i is between 1-X) convolutional layer
Include a QiIt is a having a size of m*m*piConvolution kernel, wherein m*m indicate convolution kernel length and wide pixel value, piEqual to upper one
The convolution nuclear volume Q of convolutional layeri-1, in i-th of convolutional layer, convolution kernel is (such as former to the data from upper level with step-length L
Figure, (i-1)-th convolutional layer or sample level) carry out convolution operation；Each sample level includes 1 mobile with step-length 2L, size
For the convolution kernel of 2L*2L, convolution operation is carried out to the image of convolutional layer input；Wherein, feature is carried out by feature extractor to mention
After taking, the final characteristic pattern for obtaining Qx dimension；
Wherein X is between 1-20, for example, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 or 20；
Y is between 1-10, such as 1,2,3,4,5,6,7,8,9 or 10；M is between 2-10, such as 2,3,4,5,6,7,8,9 or 10；P exists
Between 1-1024, Q between 1-1024, the numerical value of p or Q respectively such as 1,2,3,4,5,6,7,8,9,10,11,12,13,14,
15,16,32,64,128,256,512 or 1024.
4. system according to claim 2 or 3, wherein sliding is arranged in the candidate region generator in the characteristic pattern
Window, the size of sliding window are n × n, such as 3 × 3；Slide sliding window along characteristic pattern, simultaneously for sliding window institute
Each position, there are corresponding relationships for the corresponding position in central point and original image, and centered on the corresponding position
The k candidate regions with different scale and length-width ratio are generated in original image；Wherein, if k candidate region has x kind
(such as 3 kinds) different scale and length-width ratio, then k=x2(such as k=9).
5. the system according to any one of claim 2-4, the target marker include middle layer again, classify layer and side
Frame returns layer, and wherein middle layer is used to map sliding window operation and is formed by the data of candidate region, is multidimensional (such as 256
Dimension or 512 dimension) vector；
Classification layer and frame return layer connect respectively with middle layer, classification layer be used for determine the object candidate area be prospect (i.e.
Positive sample) or background (i.e. negative sample), frame return layer be used to generate candidate region central point x coordinate and y-coordinate,
And the wide w and high h of candidate region.
6. a kind of identification device of gastric cancer image, including be stored with diagnosing gastric cancer image, image preprocessing program and can train
Image recognition program storage unit, it is also preferable to include arithmetic elements and display unit；
Described device, which can utilize the image recognition program of the image comprising gastric cancer lesion to be trained, (preferably has supervision to instruct
Practice), so that the image recognition program after training be enable to identify gastric cancer diseased region in image to be checked；
Preferably, the image to be checked is scope photo or real-time imaging.
7. device according to claim 6, wherein described image preprocessor is smart in the diagnosing gastric cancer image
True frame selects the diseased region of gastric cancer, and the part in frame choosing is defined as positive sample, and the part outside frame choosing is defined as negative sample,
And export the location coordinate information and/or lesion type information of lesion；It is preferred that also being carried out at desensitization to image in advance before frame choosing
Reason removes sufferer personal information；
Preferably, the frame choosing can generate a rectangle frame or square-shaped frame comprising lesions position；The coordinate information is excellent
It is selected as the coordinate information of the point in the upper left corner and the lower right corner；
It is also preferred that frame selects position to be determined by following method: 2n scope doctors carry out frame choosing in a manner of " back-to-back ", i.e., by 2n
People is randomly divided into n group, 2 people/group, while all images are randomly divided into n parts, and is randomly assigned to each group doctor and carries out frame choosing；
After the completion of frame choosing, the frame for comparing every group of two doctors is selected as a result, and selecting the consistency of result to carry out frame between two doctors
Assessment, final determination block selects position, wherein natural number of the n between 1-100, for example, 1,2,3,4,5,6,7,8,9,10,20,
30,40,50,60,70,80,90 or 100；
It is further preferred that described as follows to the standard that frame selects the consistency of result to be assessed between two doctors:
For each lesion image, the frame for comparing every group of 2 doctors selects the overlapping area of result, if every group of two doctors point
The 50% of the area that the union that the area (i.e. intersection) of the position lap of other frame choosing is greater than the rwo is covered, then it is assumed that 2
The frame of position doctor selects judging result consistency good, and it is final that the corresponding diagonal line coordinates of above-mentioned intersection saved as target lesion
Positioning；
If the 50% of the area that the union that the area (i.e. intersection) of lap is less than the rwo is covered, then it is assumed that 2 doctors
Frame select judging result difference it is larger, such lesion picture is individually picked out, and selects the position 2n of work to cure by all participation frames
Shi Gongtong discusses the final position for determining target lesion.
8. device according to claim 6 or 7, described image recognizer is trainable image neural network based
Recognizer, the neural network are preferably convolutional neural networks；Preferably, described image recognizer includes feature extraction
Device, candidate region generator and target marker, in which:
The feature extractor is used to carry out feature extraction to image to obtain characteristic pattern, it is preferred that the feature extraction is logical
Cross convolution operation progress；
The candidate region generator is used to generate several candidate regions based on the characteristic pattern；
The target marker calculates the classification score of the candidate region, and the score indicates that the region belongs to the positive sample
The probability of this and/or the negative sample；Target marker can propose adjusted value to the bezel locations in each region simultaneously, from
And be adjusted for the bezel locations in each region, to accurately determine lesions position；Preferably, the classification score and adjustment
Loss function (Loss function) has been used in the training of value.
9. the device according to any one of claim 6 to 8, wherein when carrying out described trained, using based on mini-
The gradient descent method of batch generates the mini- comprising multiple positive and negative candidate regions to each Zhang Xunlian picture
batch.Then ratio of 256 candidate regions of random sampling until positive candidate region and negative candidate region from every picture
Example then calculates the loss function of corresponding mini-batch close to 1:1.If the quantity of positive candidate region is few in a picture
In 128, then go to fill up this mini-batch with feminine gender candidate region；
Preferably, 0.001 is set by the learning rate of preceding 50000 mini-batch, by rear 50000 mini-batch
Habit rate is set as 0.0001；Momentum term is preferably arranged to 0.9, and weight decaying is preferably arranged to 0.0005.
10. device according to claim 8 or claim 9, wherein the feature extractor can to the arbitrary dimension of input and/or
The image of resolution ratio carries out feature extraction, and described image can be original image size and/or resolution ratio, is also possible to change size
And/or the image inputted after resolution ratio, obtain the characteristic pattern of multidimensional (such as 256 dimensions or 512 dimensions)；
Specifically, the feature extractor includes X convolutional layer and Y sample level, wherein i-th of (i is between 1-X) convolutional layer
Include a QiIt is a having a size of m*m*piConvolution kernel, wherein m*m indicate convolution kernel length and wide pixel value, piEqual to upper one
The convolution nuclear volume Q of convolutional layeri-1, in i-th of convolutional layer, convolution kernel is (such as former to the data from upper level with step-length L
Figure, (i-1)-th convolutional layer or sample level) carry out convolution operation；Each sample level includes 1 mobile with step-length 2L, size
For the convolution kernel of 2L*2L, convolution operation is carried out to the image of convolutional layer input；Wherein, feature is carried out by feature extractor to mention
After taking, the final characteristic pattern for obtaining Qx dimension；
Wherein X is between 1-20, for example, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 or 20；
Y is between 1-10, such as 1,2,3,4,5,6,7,8,9 or 10；M is between 2-10, such as 2,3,4,5,6,7,8,9 or 10；P exists
Between 1-1024, Q between 1-1024, the numerical value of p or Q respectively such as 1,2,3,4,5,6,7,8,9,10,11,12,13,14,
15,16,32,64,128,256,512 or 1024.
11. the device according to any one of claim 8 to 10, wherein the candidate region generator is in the characteristic pattern
Middle setting sliding window, the size of sliding window are n × n, such as 3 × 3；Slide sliding window along characteristic pattern, simultaneously for
There are corresponding relationships for corresponding position in each position where sliding window, central point and original image, and with described corresponding
The k candidate regions with different scale and length-width ratio are generated centered on position in original image；Wherein, if k candidate regions
Domain has x kind (such as 3 kinds) different scale and length-width ratio, then k=x2(such as k=9).
12. the device according to any one of claim 8 to 11, the target marker includes middle layer again, layer of classifying
Return layer with frame, wherein middle layer is used to map sliding window operation and is formed by the data of candidate region, be a multidimensional (such as
256 dimension or 512 dimension) vector；
Classification layer and frame return layer connect respectively with middle layer, classification layer be used for determine the object candidate area be prospect (i.e.
Positive sample) or background (i.e. negative sample), frame return layer be used to generate candidate region central point x coordinate and y-coordinate,
And the wide w and high h of candidate region.
13. system according to any one of claim 1 to 5 or the described in any item devices of claim 6 to 12 are in stomach
Purposes in the prediction and diagnosis of cancer and/or gastric precancerous lesion.
14. system according to any one of claim 1 to 5 or the described in any item devices of claim 6 to 12 are in stomach
Purposes in cancer image or gastric cancer image in the identification of diseased region.
15. system according to any one of claim 1 to 5 or the described in any item devices of claim 6 to 12 are in stomach
Purposes in the real-time diagnosis of cancer and/or gastric precancerous lesion.
16. system according to any one of claim 1 to 5 or the described in any item devices of claim 6 to 12 are in stomach
Purposes in cancer image or gastric cancer image in the real-time identification of diseased region.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811371788.1A CN109671053A (en) | 2018-11-15 | 2018-11-15 | A kind of gastric cancer image identification system, device and its application |
| CN202210461368.2A CN115345819A (en) | 2018-11-15 | 2018-11-15 | Gastric cancer image recognition system, device and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811371788.1A CN109671053A (en) | 2018-11-15 | 2018-11-15 | A kind of gastric cancer image identification system, device and its application |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210461368.2A Division CN115345819A (en) | 2018-11-15 | 2018-11-15 | Gastric cancer image recognition system, device and application thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109671053A true CN109671053A (en) | 2019-04-23 |
Family
ID=66141781
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210461368.2A Pending CN115345819A (en) | 2018-11-15 | 2018-11-15 | Gastric cancer image recognition system, device and application thereof |
| CN201811371788.1A Pending CN109671053A (en) | 2018-11-15 | 2018-11-15 | A kind of gastric cancer image identification system, device and its application |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210461368.2A Pending CN115345819A (en) | 2018-11-15 | 2018-11-15 | Gastric cancer image recognition system, device and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN115345819A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110176002A (en) * | 2019-06-05 | 2019-08-27 | 深圳大学 | A kind of the lesion detection method and terminal device of radioscopic image |
| CN110490262A (en) * | 2019-08-22 | 2019-11-22 | 京东方科技集团股份有限公司 | Image processing model generation method, image processing method, device and electronic equipment |
| CN110866897A (en) * | 2019-10-30 | 2020-03-06 | 上海联影智能医疗科技有限公司 | Image detection method and computer readable storage medium |
| CN110880177A (en) * | 2019-11-26 | 2020-03-13 | 北京推想科技有限公司 | Image identification method and device |
| CN110974306A (en) * | 2019-12-17 | 2020-04-10 | 山东大学齐鲁医院 | System for discernment and location pancreas neuroendocrine tumour under ultrasonic endoscope |
| WO2020215807A1 (en) * | 2019-04-25 | 2020-10-29 | 天津御锦人工智能医疗科技有限公司 | Deep-learning-based method for improving colonoscope adenomatous polyp detection rate |
| CN112568864A (en) * | 2020-12-03 | 2021-03-30 | 牡丹江医学院 | Uterine cavity operation monitoring system |
| CN113642679A (en) * | 2021-10-13 | 2021-11-12 | 山东凤和凰城市科技有限公司 | Multi-type data identification method |
| CN114463246A (en) * | 2020-11-06 | 2022-05-10 | 广达电脑股份有限公司 | Circle selection system and circle selection method |
| CN115517682A (en) * | 2022-11-25 | 2022-12-27 | 四川大学华西医院 | Cognitive dysfunction prediction system based on gastrointestinal electric signals and construction method |
| TWI825643B (en) * | 2022-03-30 | 2023-12-11 | 緯創資通股份有限公司 | Medical auxiliary information generation method and medical auxiliary information generation system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017101142A1 (en) * | 2015-12-17 | 2017-06-22 | 安宁 | Medical image labelling method and system |
| CN107368859A (en) * | 2017-07-18 | 2017-11-21 | 北京华信佳音医疗科技发展有限责任公司 | Training method, verification method and the lesion pattern recognition device of lesion identification model |
| CN107563123A (en) * | 2017-09-27 | 2018-01-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for marking medical image |
| CN108665456A (en) * | 2018-05-15 | 2018-10-16 | 广州尚医网信息技术有限公司 | The method and system that breast ultrasound focal area based on artificial intelligence marks in real time |
| CN108734694A (en) * | 2018-04-09 | 2018-11-02 | 华南农业大学 | Thyroid tumors ultrasonoscopy automatic identifying method based on faster r-cnn |
-
2018
- 2018-11-15 CN CN202210461368.2A patent/CN115345819A/en active Pending
- 2018-11-15 CN CN201811371788.1A patent/CN109671053A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017101142A1 (en) * | 2015-12-17 | 2017-06-22 | 安宁 | Medical image labelling method and system |
| CN107368859A (en) * | 2017-07-18 | 2017-11-21 | 北京华信佳音医疗科技发展有限责任公司 | Training method, verification method and the lesion pattern recognition device of lesion identification model |
| CN107563123A (en) * | 2017-09-27 | 2018-01-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for marking medical image |
| CN108734694A (en) * | 2018-04-09 | 2018-11-02 | 华南农业大学 | Thyroid tumors ultrasonoscopy automatic identifying method based on faster r-cnn |
| CN108665456A (en) * | 2018-05-15 | 2018-10-16 | 广州尚医网信息技术有限公司 | The method and system that breast ultrasound focal area based on artificial intelligence marks in real time |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020215807A1 (en) * | 2019-04-25 | 2020-10-29 | 天津御锦人工智能医疗科技有限公司 | Deep-learning-based method for improving colonoscope adenomatous polyp detection rate |
| CN110176002A (en) * | 2019-06-05 | 2019-08-27 | 深圳大学 | A kind of the lesion detection method and terminal device of radioscopic image |
| CN110176002B (en) * | 2019-06-05 | 2022-04-01 | 深圳大学 | Focus detection method of X-ray image and terminal device |
| CN110490262B (en) * | 2019-08-22 | 2022-06-03 | 京东方科技集团股份有限公司 | Image processing model generation method, image processing device and electronic equipment |
| CN110490262A (en) * | 2019-08-22 | 2019-11-22 | 京东方科技集团股份有限公司 | Image processing model generation method, image processing method, device and electronic equipment |
| US11887303B2 (en) | 2019-08-22 | 2024-01-30 | Beijing Boe Technology Development Co., Ltd. | Image processing model generation method, image processing method and device, and electronic device |
| CN110866897A (en) * | 2019-10-30 | 2020-03-06 | 上海联影智能医疗科技有限公司 | Image detection method and computer readable storage medium |
| CN110880177A (en) * | 2019-11-26 | 2020-03-13 | 北京推想科技有限公司 | Image identification method and device |
| CN110974306A (en) * | 2019-12-17 | 2020-04-10 | 山东大学齐鲁医院 | System for discernment and location pancreas neuroendocrine tumour under ultrasonic endoscope |
| US12014813B2 (en) | 2020-11-06 | 2024-06-18 | Quanta Computer Inc. | Contouring system |
| CN114463246A (en) * | 2020-11-06 | 2022-05-10 | 广达电脑股份有限公司 | Circle selection system and circle selection method |
| CN112568864B (en) * | 2020-12-03 | 2022-03-18 | 牡丹江医学院 | Uterine cavity operation monitoring system |
| CN112568864A (en) * | 2020-12-03 | 2021-03-30 | 牡丹江医学院 | Uterine cavity operation monitoring system |
| CN113642679A (en) * | 2021-10-13 | 2021-11-12 | 山东凤和凰城市科技有限公司 | Multi-type data identification method |
| TWI825643B (en) * | 2022-03-30 | 2023-12-11 | 緯創資通股份有限公司 | Medical auxiliary information generation method and medical auxiliary information generation system |
| CN115517682A (en) * | 2022-11-25 | 2022-12-27 | 四川大学华西医院 | Cognitive dysfunction prediction system based on gastrointestinal electric signals and construction method |
| CN115517682B (en) * | 2022-11-25 | 2023-01-31 | 四川大学华西医院 | Cognitive dysfunction prediction system based on gastrointestinal electric signals and construction method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115345819A (en) | 2022-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109584218A (en) | A kind of construction method of gastric cancer image recognition model and its application | |
| CN109671053A (en) | A kind of gastric cancer image identification system, device and its application | |
| CN109544526A (en) | A kind of atrophic gastritis image identification system, device and method | |
| CN109523535A (en) | A kind of preprocess method of lesion image | |
| Li et al. | A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification and detection approaches | |
| Silva-Rodríguez et al. | Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection | |
| Adegun et al. | Deep learning-based system for automatic melanoma detection | |
| Zhou et al. | A comprehensive review for breast histopathology image analysis using classical and deep neural networks | |
| US11257209B2 (en) | Cancer risk stratification based on histopathological tissue slide analysis | |
| JP6000345B2 (en) | Analysis method of biological samples by spectral imaging | |
| CN109670530A (en) | A kind of construction method of atrophic gastritis image recognition model and its application | |
| Du et al. | Identification of COPD from multi-view snapshots of 3D lung airway tree via deep CNN | |
| Turkmen et al. | Classification of laryngeal disorders based on shape and vascular defects of vocal folds | |
| Fan et al. | Effect of image noise on the classification of skin lesions using deep convolutional neural networks | |
| Hu et al. | A state-of-the-art survey of artificial neural networks for whole-slide image analysis: from popular convolutional neural networks to potential visual transformers | |
| CN106096654A (en) | A kind of cell atypia automatic grading method tactful based on degree of depth study and combination | |
| CN110097974A (en) | A kind of nasopharyngeal carcinoma far-end transfer forecasting system based on deep learning algorithm | |
| Wang et al. | Study on automatic detection and classification of breast nodule using deep convolutional neural network system | |
| CN101714153A (en) | Visual perception based interactive mammography image searth method | |
| CN114600155A (en) | Weakly supervised multitask learning for cell detection and segmentation | |
| CN112508884A (en) | Comprehensive detection device and method for cancerous region | |
| Lee et al. | Model architecture and tile size selection for convolutional neural network training for non-small cell lung cancer detection on whole slide images | |
| WO2023249552A1 (en) | System and method for classification of basal cell carcinoma based on confocal microscopy | |
| CN115861598A (en) | Polyp detection method based on convolutional neural network | |
| Ahmad et al. | Colorectal cancer image classification using image pre-processing and multilayer Perceptron |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190423 |