CN110473195A - It is a kind of can automatic customization medicine lesion detection framework and method - Google Patents
It is a kind of can automatic customization medicine lesion detection framework and method Download PDFInfo
- Publication number
- CN110473195A CN110473195A CN201910743751.5A CN201910743751A CN110473195A CN 110473195 A CN110473195 A CN 110473195A CN 201910743751 A CN201910743751 A CN 201910743751A CN 110473195 A CN110473195 A CN 110473195A
- Authority
- CN
- China
- Prior art keywords
- layer
- lesion
- candidate feature
- candidate
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003902 lesion Effects 0.000 title claims abstract description 105
- 238000001514 detection method Methods 0.000 title claims abstract description 84
- 239000003814 drug Substances 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000013508 migration Methods 0.000 claims abstract description 28
- 230000005012 migration Effects 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims abstract description 24
- 230000002708 enhancing effect Effects 0.000 claims abstract description 8
- 230000008447 perception Effects 0.000 claims abstract description 7
- 239000011248 coating agent Substances 0.000 claims description 32
- 238000000576 coating method Methods 0.000 claims description 32
- 238000010606 normalization Methods 0.000 claims description 28
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000012549 training Methods 0.000 claims description 14
- 239000000284 extract Substances 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 11
- 239000000203 mixture Substances 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 8
- 238000013461 design Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 6
- 238000003475 lamination Methods 0.000 claims description 4
- 230000000644 propagated effect Effects 0.000 claims description 4
- 238000000844 transformation Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000007689 inspection Methods 0.000 claims description 3
- 238000009434 installation Methods 0.000 claims description 2
- 238000009415 formwork Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 11
- 238000013527 convolutional neural network Methods 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 3
- 239000012141 concentrate Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 229940079593 drug Drugs 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20088—Trinocular vision calculations; trifocal tensor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The invention discloses it is a kind of can automatic customization medicine lesion detection framework and method, the detection framework include: candidate feature extraction module, feature extraction is carried out to medical image；Lesion detects network header automatic customization module, defines search space and merges the perception relationship between candidate region, and using can micro- NAS algorithm obtain best lesion detection network header；Lesion detects the optimal module of network header, utilize a convolutional layer, and new candidate feature is obtained by a standard cell and two contractive cells, binary classification is carried out to candidate feature by two articulamentums and prediction block returns, the weight M that candidate feature in binary classification is classified is exported as high-level semantic information to knowledge migration module；Knowledge migration module transmits relevant contextual information, the candidate feature enhanced in conjunction with semantic relation and in different regions, and the candidate feature of enhancing and former candidate feature is merged, and carries out multivariate classification and recurrence finally by full articulamentum.
Description
Technical field
The present invention relates to the technical fields such as image recognition, target detection and deep learning, can be automatic more particularly to one kind
The medicine lesion detection framework and method of customization.
Background technique
Object detection task is position and the size found out all interested targets in image, and determine them.Medicine
Lesion Detection task is position and the size found out lesion all in medical image, and determine lesion；This is area of computer aided
The important prerequisite of detection/diagnosis (CADe/CADx).Currently, with deep learning algorithm, especially convolutional neural networks (CNN)
Fast development, to medicine lesion detection bring significant progress.However, most methods are all directly by natural figure at present
The various CNN pre-training target detection models such as RetinaNet of picture, full convolutional network (R-FCN) based on region etc., for curing
Learn the lesion detection of image；But since there is huge field difference, the detections of medicine lesion for medical image and natural image
Can have that such as lesion and the high similitude of background, lesion classification is unequal and the field is specifically challenged based on small lesion, institute
Directly to use the method limited performance of traditional natural image detection model, performance can be caused to drop because of above-mentioned challenge
It is low.Therefore, the network architecture that customization detects dedicated for medicine lesion is very important.
Recently, neural framework search (Neural Architecture Search, NAS) is in image classification, semantic segmentation
Very competitive performance is achieved in tasks with natural image processing etc., and the purpose of NAS is searched automatically according to task object
The optimal neural network framework of rope, the limitation of researcher's hand-designed is broken through with this, to realize more preferably performance.Existing
Target detection NAS work only moves to the network architecture searched in picture classification task in detection skeleton, Er Qiexu
Consume a large amount of GPU video memory and time.
Traditional target detection frame mainly includes three parts: feature extractor, region candidate network (RPN) and be based on area
The head CNN in domain.It is generally known that facilitating lesion inspection with the feature extractor and RPN of natural image such as ImageNet pre-training
It surveys, is the important component of medicine lesion detection network.Currently, there are many researchs on the head CNN based on region, such as attached drawing 1
It is shown, it can be broadly divided into following three kinds: 1) receptive field head (RFH), consider that multiple size and shape of receptive field are leaned on protrusion
The importance of nearly central area, and improve the insensitivity mobile to small space.2) head (FCH) is connected entirely, parameter redundancy, and
Ignore spatial information.3) residual error bottleneck head (RBH) enhances candidate information using residual error bottleneck module, and skip floor connection fusion is not
The feature of ad eundem and gradient disperse is avoided, but performance can be limited by single receptive field.
Summary of the invention
In order to overcome the deficiencies of the above existing technologies, purpose of the present invention is to provide it is a kind of can automatic customization medicine
Lesion detects framework and method, to realize that one kind can share relevant information with a variety of lesion types, and it is real in a seamless fashion
Now unified polymorphic type lesion detects network.
In order to achieve the above object, the present invention propose it is a kind of can automatic customization medicine lesion detection framework, comprising:
Candidate feature extraction module, for carrying out feature extraction to the medical image of input, the candidate for extracting image is special
Sign；
Lesion detects network header automatic customization module, for according to medical image characteristic, focus characteristic and target detection
Relevant knowledge, define new search space, described search space includes that largely there is flexible receptive field, skip floor the subnets such as to connect
The advanced operation of network framework, and increase a non local operation, the perception relationship between candidate region is merged, according to time
Select feature and using can micro- NAS algorithm search for suitable operation and connection type in the search space of design and makes its composition
It is suitble to the best lesion of medical image to detect network header；
Lesion detects the optimal module of network header, detects the best of network header automatic customization Custom modules for the lesion
Lesion detect network header, by the candidate feature extraction module export candidate feature, first pass around a convolution kernel be 3 ×
Then 3 convolutional layer obtains new candidate feature by a standard cell and two contractive cells, and passes through two articulamentums
Binary classification is carried out to new candidate feature and prediction block returns, the weight M that candidate feature in binary classification is classified is as height
Layer semantics information is exported to knowledge migration module；
Knowledge migration module, on the basis of the components graph that best lesion detection network header learns, in conjunction with language
Adopted relationship, and relevant contextual information is transmitted in different regions, to obtain the candidate feature of an enhancing, and will enhancing
Candidate feature and original candidate feature afterwards merges the relevant information for sharing a variety of lesion types, finally by complete
Articulamentum carries out multivariate classification and recurrence.
Preferably, described search space includes following 9 kinds of operations: 1) connectionless；2) skip floor connects；3) 3 × 3 average pond
Change；4) non local；5) 1 × 3 and 3 × 1 convolution；6) 3 × 3 depth separates convolution；7) 5 × 5 depth separates convolution；
8) 3 × 3 empty convolution that expansion rate is 3；9) 3 × 3 empty convolution that expansion rate is 5.
Preferably, it is described can be in micro- NAS algorithm, it is necessary first to it is then fixed according to the suitable search space of task design
The module of justice search, including standard cell module and contractive cell module, wherein standard cell module step-length is 1, to keep defeated
The resolution ratio same with input out, while port number is constant, contractive cell module step-length is 2, resolution ratio is reduced half, simultaneously
Port number is double, and each module, that is, cell regards a directed acyclic graph as, defines its branch's number, one spy of each branching representation
Sign is schemed, and the connection type between branch indicates operation；There are two input and an output from branch before, In for each branch
After completing above-mentioned search definition, Initialize installation is carried out, while its discrete topology serialization is made by softmax function, it
Gradient passback is carried out using gradient descent algorithm afterwards and updates its weight, finally after the search by certain time, first 9
Kind, which operates, retains the maximum operation of weight in connection, i.e., intensively connected by one and become partially connected, then select weight
Maximum two are connected to the input of the branch, and their result are incorporated as exporting.
Preferably, 9 kinds of candidate operations set are defined as follows:
1) it connectionless operation: is not connected between branch；
2) skip floor attended operation: branch is directly connected to, and does not pass through any operation；
3) 3 × 3 average pondization operation: the average pond that Chi Huahe size is 3 × 3；
4) 1 × 3 and 3 × 1 convolution operation: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 1 × 3, one layer batch
Normalize layer, one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 1, one layer batch of normalization layer；
5) the separable convolution operation of 3 × 3 depth: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 3, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolution that one layer of convolution kernel is 3 × 3
Layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
6) the separable convolution operation of 5 × 5 depth: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 5 × 5, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolution that one layer of convolution kernel is 5 × 5
Layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
7) 3 × 3 empty convolution operations that expansion rate is 3: one layer of ReLU active coating, one layer of convolution kernel are 3 × 3 and expansion rate
For 3 convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
8) 3 × 3 empty convolution operations that expansion rate is 5: one layer of ReLU active coating, one layer of convolution kernel are 3 × 3 and expansion rate
For 5 convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
9) non local operation: the purpose is to compile to the semantic information between region candidate relevant to target detection
Code.
Preferably, in the non local operation, by interregional relationship be expressed as a region to region non-directed graph
G:G=<N, E>, the corresponding region candidate of each node in N, each edge eI, j∈ E encodes the pass between two nodes
System, the input that the Non-local is operated areThe adjacency matrix of non-directed graph G can be by matrix multiplication E=
softmax(φ(X)φ(X)T) can be calculated, wherein φ () is the nonlinear transformation for having ReLU activation primitive；Later
Each node in E is propagated with picture scroll lamination Y=σ (Ef (X) W), wherein f (), W are a nonlinear transformations, and σ is one
Activation primitive, finally connecting full articulamentum is consistent its input and output size.
Preferably, the search strategy of the lesion detection network header automatic customization module is to be calculated using stochastic gradient descent
Method goes to optimize by the parameter expression of discrete topology serialization, goes one group of continuous framework weight of studyThe output tensor of branchFor the weighted blend of candidate operations, it is expressed as Wherein weightFor
Configuration parameters.
Preferably, training data is divided into two Uncrossed subsets, i.e. training set and verifying collection, optimization process is by following
Two step iteration carry out: 1) passing throughUpdate network weight w；2) pass throughMore new architecture power
Weight a.Ltrain(w, a) and Lval(w a) is respectively training set and the loss for verifying collection, wherein Ltrain(w, a) and Lval(w a) divides
Not Wei training set and verifying collection loss.
Preferably, the knowledge migration module realizes that process is as follows:
A. the weight and deviation of original binary classification are collectedAs high-level semantic information, P is
AutoRCNN Head exports the dimension of tensor, and uses soft mappingDemapping M, wherein
sijThe region i obtained from original two classification layer is classified as the score of j.
B. the feature f that figure reasoning is enhanced is carried out by matrix multiplicationo, i.e. fo=E ΓsMWo, wherein
It is weight transfer matrix, o is the output dimension of knowledge migration module, and by Enhanced feature foIt is combined with primitive character f
Improve position and the classification performance of the detection of polymorphic type lesion.
In order to achieve the above objectives, the present invention also provides it is a kind of can automatic customization medicine lesion detection method, including it is as follows
Step:
Step S1 carries out feature extraction using medical image of the candidate feature extraction module to input, extracts image
Candidate feature；
Step S2 is exported candidate feature extraction module using the lesion detection optimal module of network header of automatic customization
Candidate feature first passes around the convolutional layer that a convolution kernel is 3 × 3, then passes through a standard cell and two contractive cells
New candidate feature is obtained, and binary classification and prediction block recurrence are carried out to new candidate feature by two articulamentums, by two
The weight M of candidate feature classification is exported as high-level semantic information to knowledge migration module in member classification；
Step S3 utilizes the basis for the components graph that knowledge migration module learns in best lesion detection network header
On, in conjunction with semantic relation, and relevant contextual information is transmitted in different regions, it is special with the candidate for obtaining an enhancing
Then enhanced candidate feature is merged the related letter for sharing a variety of lesion types by sign to original candidate feature
Breath carries out multivariate classification and recurrence finally by full articulamentum.
Preferably, the method also includes:
It is empty to define new search according to the relevant knowledge of medical image characteristic, focus characteristic and target detection by step S0
Between, described search space includes the advanced operation largely with sub-networks frameworks such as flexible receptive field, skip floor connections, and increases by one
A non local operation merges the perception relationship between candidate region, according to candidate feature and using can micro- NAS algorithm exist
Suitable operation and connection type are searched in the search space of design makes it form the inspection of the best lesion of a suitable medical image
Survey network header.
Compared with prior art, one kind of the present invention can automatic customization medicine lesion detection framework and method by being automatically
Medicine lesion Detection task has customized the detection network architecture head of a suitable medical image, and by knowledge migration module with
Seamless mode realizes a unified polymorphic type lesion detection network, not only increases the precision of lesion detection, also increases
The function of polymorphic type lesion detection.The medicine lesion detection network architecture head and knowledge migration module that the present invention realizes can
Based on the detection network skeleton that any natural image is general, and performance has different degrees of promotion, all realizes detection net
The optimum performance of network skeleton.
Detailed description of the invention
Fig. 1 is the RCNN Head architecture diagram of the prior art；
Fig. 2 be one kind of the present invention can automatic customization medicine lesion detection framework configuration diagram；
Fig. 3 is the standard cell architecture diagram in the specific embodiment of the invention in AutoRCNN Head；
Fig. 4 is the contractive cell architecture diagram in the specific embodiment of the invention in AutoRCNN Head；
Fig. 5 be one kind of the present invention can automatic customization medicine lesion detection method step flow chart.
Specific embodiment
Below by way of specific specific example and embodiments of the present invention are described with reference to the drawings, those skilled in the art can
Understand further advantage and effect of the invention easily by content disclosed in the present specification.The present invention can also pass through other differences
Specific example implemented or applied, details in this specification can also be based on different perspectives and applications, without departing substantially from
Various modifications and change are carried out under spirit of the invention.
Fig. 2 be one kind of the present invention can automatic customization medicine lesion detection framework configuration diagram.As shown in Fig. 2, this
Invent it is a kind of can automatic customization medicine lesion detection framework, comprising:
Candidate feature extraction module 201 extracts the candidate of image for carrying out feature extraction to the medical image of input
Feature.In the specific embodiment of the invention, candidate feature extraction module 201 using it is ready-made include feature extractor and RPN
General detection network extracts the candidate feature of image, and since medical image is single channel grayscale image, candidate feature is extracted
Module 201 can using the key frame of medical image and its before and after frames composition triple channel be used as input picture I, by feature extractor with
Feature pyramid extracts the semantic feature of image different levels, extracts the candidate feature of image by RPN network later, passes through
After RoI alignment, network header (AutoRCNN Head) optimal module 203 is detected into lesion, due to candidate feature of the present invention
The image characteristics extraction of extraction module 201 is using the prior art, and it will not be described here
Lesion detect network header (AutoRCNN Head) automatic customization module 202, for according to medical image characteristic,
The relevant knowledge of focus characteristic and target detection, defines new search space, and described search includes a large amount of with flexibly impression
The advanced operation of the sub-networks frameworks such as wild, skip floor connection, and increase by one non local (Non-local) operation, it goes candidate regions
Perception relationship between domain merges, according to candidate feature using can micro- NAS algorithm conjunction is searched in the search space of design
Suitable operation and connection type make its composition one detection network head, i.e., using can micro- NAS algorithm go to search out one automatically
The best lesion of a suitable medical image detects network header AutoRCNN Head.
In object detection task, the identification of character representation is can be enhanced in the relationship between the size and centrality of receptive field
Property and robustness, therefore, the present invention be can micro- NAS algorithm devise a new search space, including following 9 kinds of operations: 1)
It is connectionless；2) skip floor connects；3) 3 × 3 average pond；4) non local (Non-local)；5) 1 × 3 and 3 × 1 convolution；6)3
× 3 depth separates convolution；7) 5 × 5 depth separates convolution；8) 3 × 3 empty convolution that expansion rate is 3；9) expansion rate
For 53 × 3 empty convolution.
Can be in micro- NAS algorithm, it is necessary first to according to the suitable search space of task design, 9 kinds of such as above-mentioned operations, so
Need to define the module of search, i.e. standard cell module and contractive cell module afterwards, wherein standard cell module step-length is 1, with
It keeps output and inputs same resolution ratio, while port number is constant, contractive cell module step-length is 2, and resolution ratio is reduced by one
Half, at the same port number is double.Each module, that is, cell can regard a directed acyclic graph as, need to define its branch's number, often
One characteristic pattern of a branching representation, the connection type between branch indicate operation, such as skip floor connection, 3 × 3 average pond；It is each
There are two inputs and an output from branch before to carry out initialization after completing above-mentioned search definition and set for branch
It sets, there is the input from previous all branches in each branch, while having all candidate OP to connect between the branch of every two connection
It connects, such as candidate OP has nine kinds, then 9 kinds of connection relationships is initialized as between the branch for having connection, and their weight is arranged and is
1, while its discrete topology serialization is made by softmax function, gradient passback, which is carried out, using gradient descent algorithm later updates
Its weight.Finally after the search by certain time, retain the maximum behaviour of weight in 9 kinds of operation connections first
Make, i.e., partially connected is become by an intensive connection, then select weight maximum two inputs for being connected to the branch, together
When their result is incorporated as exporting.
Assuming that each module, that is, cell is the directed acyclic graph comprising B branch, there are two is come from for each branch
The input of branch before and an output；Branching representation characteristic tensor, line indicate the operation between characteristic tensor.5 yuan can be used
Group (X1,X2,OP1,OP2,) indicate branch b, wherein X in cell c1, To input tensor；OP1,OP2∈ OP points
It Wei not X1,X2Respective operations, OP are 9 kinds of candidate operations set proposed by the present invention；Cell c last output Yc=merge
Specifically, the present invention be can micro- NAS algorithm devise a new search space, keep it big using receptive field
The identity and robustness that the small relationship between centrality goes Enhanced feature to indicate.9 kinds of candidate operations set, that is, OP are specific
It is defined as follows:
1) it connectionless operation: is not connected between branch；
2) skip floor attended operation: branch is directly connected to, and does not pass through any operation；
3) 3 × 3 average pondization operation: the average pond that Chi Huahe size is 3 × 3；
4) 1 × 3 and 3 × 1 convolution operation: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 1 × 3, one layer batch
Normalize layer, one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 1, one layer batch of normalization layer；
5) the separable convolution operation of 3 × 3 depth: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 3, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolution that one layer of convolution kernel is 3 × 3
Layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
6) the separable convolution operation of 5 × 5 depth: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 5 × 5, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolution that one layer of convolution kernel is 5 × 5
Layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
7) 3 × 3 empty convolution operations that expansion rate is 3: one layer of ReLU active coating, one layer of convolution kernel are 3 × 3 and expansion rate
For 3 convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
8) 3 × 3 empty convolution operations that expansion rate is 5: one layer of ReLU active coating, one layer of convolution kernel are 3 × 3 and expansion rate
For 5 convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer.
9) non local (Non-local) operation: the purpose is to the semanteme between region candidate relevant to target detection
Information is encoded.Specifically, by interregional relationship be expressed as a region to region non-directed graph G:G=<N, E>, in N
In the corresponding region candidate of each node, each edge eI, j∈ E encodes the relationship between two nodes.Of the invention specific
In embodiment, the input of Non-local operation isThe adjacency matrix of non-directed graph G can be by matrix multiplication E=
softmax(φ(X)φ(X)T) can be calculated, wherein φ () is the nonlinear transformation for having ReLU activation primitive；Later
Each node in E is propagated with picture scroll lamination Y=σ (Ef (X) W), wherein f (), W are a nonlinear transformations, and σ is one
Activation primitive, finally connecting full articulamentum is consistent its input and output size.
The search strategy that lesion detects network header (AutoRCNN Head) automatic customization module 202 is to use stochastic gradient
Descent algorithm goes to optimize by the parameter expression of discrete topology serialization, goes one group of continuous framework weight of studyBranch it is defeated
Tensor outIt is the weighted blend of candidate operations, is represented by Wherein weigh
WeightIt is configuration parameters.In lesion detection network header AutoRCNN Head of the invention, contractive cell is in 1/3 and arrives
The position of 2/3 depth, other positions are standard cell.Their corresponding configuration parameters are respectively (aIt shrinks, aStandard), wherein
All standard cells in AutoRCNN Head share weight aStandard, all contractive cells share weight aIt shrinks。
After by discrete topology serialization, the task of framework search is reduced to one group of continuous variable of studyIt can be with having
The stochastic gradient descent algorithm of effect optimizes.Loss function include lesion classification intersect entropy function and lesion detection it is absolute
Loss function updates network weight by iteration and framework weight carries out.The present invention by training data be divided into two it is Uncrossed
Subset, i.e. training set and verifying collection, their corresponding losses are LtrainAnd Lval.Optimization process is by following two steps iteration
It carries out:
1) pass throughUpdate network weight w；
2) pass throughMore new architecture weight a；
After search, pass throughRetain the operation OP of a maximum weightijIt goes to obtain
A discrete framework is obtained, i.e., searches out the standard cell and contractive cell of a suitable medical image automatically, will finally be shunk
Cell is placed on 1/3 to 2/3 depth of entire head, remaining position is stacked up for standard cell, and group becomes medical image
The best lesion of customization detects network header AutoRCNN Head.The optimization criteria cell and contractive cell architecture diagram searched
It shows in figs. 3 and 4.In cell architecture diagram, c_ { k } indicates to work as precellular output, and c_ { k-1 } and c_ { k-2 } are respectively
Indicate the output of the first two cell.Wherein 0,/1/,2/3 the first/bis-/tri-/tetra- branches in cell are respectively indicated, between branch
Connection be search after, candidate operations concentrate the operation of remain one maximum weight.I.e. each cell comes from
The output of the first two cell has an output as input.In initialization, there are 9 kinds of operation connections between each branch,
After the search for carrying out gradient updating, every connection path first only retains the operation of a maximum weight, finally in these behaviour
Reselection retains input of the operation of two maximum weights from different inputs as the branch in work.Finally, all branches
Output of the splicing of output as entire cell.Such as in the standard cell architecture diagram of attached drawing 3, the input of branch 0 is respectively
3 × 3 empty convolution operations that the expansion rate of skip floor attended operation and c_ { k-2 } from c_ { k-1 } is 5；The input of branch 1 point
It is not the non local operation of skip floor attended operation and c_ { k-2 } from c_ { k-1 }；The input of branch 2 is from c_ { k- respectively
1 } 3 × 3 depth separates the expansion rate of convolution operation and branch 1 as 33 × 3 empty convolution operations；The input of branch 3
It is 3 × 3 empty convolution operations that the expansion rate of non local operation and branch 1 from branch 0 is 3 respectively；Finally the standard is thin
The output of born of the same parents is that the output of branch 0/1/2/3 is spliced.In the contractive cell architecture diagram of attached drawing 4, the input of branch 0 comes respectively
5 × 5 depth of 1 × 3 and 3 × 1 convolution operation and c_ { k-2 } from c_ { k-1 } separates convolution operation；Branch 1 it is defeated
Enter be respectively from c_ { k-1 } 1 × 3 and 3 × 1 convolution operation and branch 05 × 5 depth separate convolution operation；Point
The input of branch 2 is 1 × 3 and 3 × 1 convolution operation of 3 × 3 average pondization operation and branch 1 from branch 0 respectively；Point
The input of branch 3 is 1 × 3 and 3 × 1 convolution behaviour that 5 × 5 depth from branch 1 separates convolution operation and branch 2 respectively
Make；The finally output of the standard cell is that the output of branch 0/1/2/3 is spliced.
Lesion detects network header (AutoRCNN Head) optimal module 203, detects network header for lesion
The best lesion of (AutoRCNN Head) automatic customization module 202 customization detects network header AutoRCNN Head, will wait
The candidate feature for selecting characteristic extracting module 201 to export first passes around the convolutional layer that a convolution kernel is 3 × 3, then passes through one
Standard cell and two contractive cells obtain new candidate feature, and carry out binary to new candidate feature by two articulamentums
Classification and prediction block return, and the weight M that candidate feature in binary classification is classified is exported as high-level semantic information to knowledge
Transferring module 204, to map the candidate feature for moving to multivariate classification module and being enhanced by knowledge migration module 204, and will
Enhanced feature and candidate feature are combined, and carry out polymorphic type lesion to the candidate feature after merging by most latter two articulamentum
Detection, i.e. multivariate classification and prediction block return.In the specific embodiment of the invention, the standard cell of AutoRCNN Head module
Step-length is 1, and to keep output and input same resolution ratio, while port number is constant；Contractive cell step-length is 2, by resolution ratio
Half is reduced, while port number is double.
Knowledge migration module 204, in the components graph that best lesion detection network header AutoRCNN Head learns
On the basis of, in conjunction with semantic relation, and relevant contextual information is transmitted in different regions, to obtain the time of an enhancing
Feature is selected, enhanced candidate feature and original candidate feature are merged to the phase for sharing a variety of lesion types later
Information is closed, carries out multivariate classification and recurrence finally by full articulamentum, so that the present invention be made to realize one in a seamless fashion
Unified polymorphic type lesion detects network.
In the present invention, the not instead of not simply one polymorphic type classifier of fine tuning of knowledge migration module 204, further grinds
Study carefully the ability of best lesion detection network header AutoRCNN Head.Since in multiple search experiment, Non-local can
It appears in the network architecture finally selected, therefore by E=softmax (φ (X) φ (X)T) graph structure acquired, by pushing away
Binary classification is switched to polymorphic type classification by reason.
In the specific embodiment of the invention, knowledge migration module 204 has used figure reasoning algorithm to realize, specifically,
Realization process is as follows:
A. the weight and deviation of original binary classification are collectedAs high-level semantic information, P is
The dimension of AutoRCNN Head output tensor.Since figure G is extracted in operating from Non-local in AutoRCNN Head
A region to the figure in region, then need to find the input node f from the high-level semantic information of classification to knowledge migration modulei
The most suitable mapping of ∈ f.For the error for avoiding original binary classification from generating, the present invention uses soft mappingDemapping M, wherein sijThe region i obtained from original two classification layer is classified as point of j
Number.
B. the feature f that figure reasoning is enhanced is carried out by matrix multiplicationo, i.e. fo=E ΓsMWo, wherein
It is weight transfer matrix, o is the output dimension of knowledge migration module.By Enhanced feature foIt is combined and mentions with primitive character f
The position of high polymorphic type lesion detection and classification performance.
Fig. 5 be one kind of the present invention can automatic customization medicine lesion detection method step flow chart.As shown in figure 5, this
Invent it is a kind of can automatic customization medicine lesion detection method, include the following steps:
Step S1 carries out feature extraction using medical image of the candidate feature extraction module to input, extracts image
Candidate feature.In the specific embodiment of the invention, candidate feature extraction module using it is ready-made include feature extractor and RPN
General detection network extracts the candidate feature of image, and since medical image is single channel grayscale image, candidate feature is extracted
Module can be using the key frame of medical image and its before and after frames composition triple channel as input picture I, by feature extractor and spy
Sign pyramid extracts the semantic feature of image different levels, extracts the candidate feature of image by RPN network later, passes through
After RoI alignment, network header (AutoRCNN Head) module is detected into lesion, since candidate feature of the present invention extracts mould
The image characteristics extraction of block is using the prior art, and it will not be described here.
Step S2, will be candidate special using lesion detection network header (AutoRCNN Head) the optimal module of automatic customization
The candidate feature for levying extraction module output first passes around the convolutional layer that a convolution kernel is 3 × 3, then thin by a standard
Born of the same parents and two contractive cells obtain new candidate feature, and by two articulamentums to new candidate feature carry out binary classification and
Prediction block returns, and the weight M that candidate feature in binary classification is classified is exported as high-level semantic information to knowledge migration mould
Block.
Step S3, using knowledge migration module in the area that best lesion detection network header AutoRCNN Head learns
On the basis of the relational graph of domain, in conjunction with semantic relation, and relevant contextual information is transmitted in different regions, to obtain one
Enhanced candidate feature and original candidate feature are merged share a variety of lesions later by the candidate feature of enhancing
The relevant information of type carries out multivariate classification and recurrence finally by full articulamentum, to realize one in a seamless fashion
Unified polymorphic type lesion detects network.
Preferably, one kind of the present invention can automatic customization medicine lesion detection method, further includes:
It is empty to define new search according to the relevant knowledge of medical image characteristic, focus characteristic and target detection by step S0
Between, described search space includes the advanced operation largely with sub-networks frameworks such as flexible receptive field, skip floor connections, and increases by one
A non local (Non-local) operation, goes to merge the perception relationship between candidate region, and being utilized according to candidate feature can
Micro- NAS algorithm searches for suitable operation and connection type in the search space of design makes the head of one detection network of its composition
Portion, i.e., using can micro- NAS algorithm go to search out the best lesion of a suitable medical image automatically and detect network header
AutoRCNN Head。
In object detection task, the identification of character representation is can be enhanced in the relationship between the size and centrality of receptive field
Property and robustness, therefore, the present invention be can micro- NAS algorithm devise a new search space, including following 9 kinds of operations: 1)
It is connectionless；2) skip floor connects；3) 3 × 3 average pond；4) non local (Non-local)；5) 1 × 3 and 3 × 1 convolution；6)3
× 3 depth separates convolution；7) 5 × 5 depth separates convolution；8) 3 × 3 empty convolution that expansion rate is 3；9) expansion rate
For 53 × 3 empty convolution.9 kinds of candidate operations set, that is, OP are defined as follows:
1) it connectionless operation: is not connected between branch；
2) skip floor attended operation: branch is directly connected to, and does not pass through any operation；
3) 3 × 3 average pondization operation: the average pond that Chi Huahe size is 3 × 3；
4) 1 × 3 and 3 × 1 convolution operation: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 1 × 3, one layer batch
Normalize layer, one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 1, one layer batch of normalization layer；
5) the separable convolution operation of 3 × 3 depth: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 3, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolution that one layer of convolution kernel is 3 × 3
Layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
6) the separable convolution operation of 5 × 5 depth: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 5 × 5, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolution that one layer of convolution kernel is 5 × 5
Layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
7) 3 × 3 empty convolution operations that expansion rate is 3: one layer of ReLU active coating, one layer of convolution kernel are 3 × 3 and expansion rate
For 3 convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
8) 3 × 3 empty convolution operations that expansion rate is 5: one layer of ReLU active coating, one layer of convolution kernel are 3 × 3 and expansion rate
For 5 convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer.
9) non local (Non-local) operation: the purpose is to the semanteme between region candidate relevant to target detection
Information is encoded.Specifically, by interregional relationship be expressed as a region to region non-directed graph G:G=<N, E>, in N
In the corresponding region candidate of each node, each edge eI, j∈ E encodes the relationship between two nodes.Of the invention specific
In embodiment, the input of Non-local operation isThe adjacency matrix of non-directed graph G can be by matrix multiplication E=
softmax(φ(X)φ(X)T) can be calculated, wherein φ () is the nonlinear transformation for having ReLU activation primitive；Later
Each node in E is propagated with picture scroll lamination Y=σ (Ef (X) W), wherein f (), W are a nonlinear transformations, and σ is one
Activation primitive, finally connecting full articulamentum is consistent its input and output size.
In step S0, the search strategy used is to go optimization by discrete topology serialization with stochastic gradient descent algorithm
Parameter expression, go study one group of continuous framework weightThe output tensor of branchIt is the weighted blend of candidate operations,
It is represented byWherein weightIt is configuration parameters.In disease of the invention
Stove detects in network header AutoRCNN Head, and contractive cell is in the position of 1/3 to 2/3 depth, and other positions are that standard is thin
Born of the same parents.Their corresponding configuration parameters are respectively (aIt shrinks, aStandard), wherein all standard cells in AutoRCNN Head are shared
Weight aStandard, all contractive cells share weight aIt shrinks。
After by discrete topology serialization, the task of framework search is reduced to one group of continuous variable of studyIt can be with having
The stochastic gradient descent algorithm of effect optimizes.Loss function include lesion classification intersect entropy function and lesion detection it is absolute
Loss function updates network weight by iteration and framework weight carries out.The present invention by training data be divided into two it is Uncrossed
Subset, i.e. training set and verifying collection, their corresponding losses are LtrainAnd Lval.Optimization process is by following two steps iteration
It carries out:
1) pass throughUpdate network weight w；
2) pass throughMore new architecture weight a；
After search, pass throughRetain the operation OP of a maximum weightijIt goes to obtain
A discrete framework is obtained, i.e., searches out the standard cell and contractive cell of a suitable medical image automatically, will finally be shunk
Cell is placed on 1/3 to 2/3 depth of entire head, remaining position is stacked up for standard cell, and group becomes medical image
The best lesion of customization detects network header AutoRCNN Head.The optimization criteria cell and contractive cell architecture diagram searched
It shows in figs. 3 and 4.In cell architecture diagram, c_ { k } indicates to work as precellular output, and c_ { k-1 } and c_ { k-2 } are respectively
Indicate the output of the first two cell.Wherein 0,/1/,2/3 the first/bis-/tri-/tetra- branches in cell are respectively indicated, between branch
Connection be search after, candidate operations concentrate the operation of remain one maximum weight.I.e. each cell comes from
The output of the first two cell has an output as input.In initialization, there are 9 kinds of operation connections between each branch,
After the search for carrying out gradient updating, every connection path first only retains the operation of a maximum weight, finally in these behaviour
Reselection retains input of the operation of two maximum weights from different inputs as the branch in work.
Preferably, in step S3, the knowledge migration module has used figure reasoning algorithm to realize, specifically, in fact
Existing process is as follows:
A. the weight and deviation of original binary classification are collectedAs high-level semantic information, P is
The dimension of AutoRCNN Head output tensor.Since figure G is extracted in operating from Non-local in AutoRCNN Head
A region to the figure in region, then need to find the input node f from the high-level semantic information of classification to knowledge migration modulei
The most suitable mapping of ∈ f.For the error for avoiding original binary classification from generating, the present invention uses soft mappingDemapping M, wherein sijThe region i obtained from original two classification layer is classified as point of j
Number.
B. the feature f that figure reasoning is enhanced is carried out by matrix multiplicationo, i.e. fo=E ΓsMWo, wherein
It is weight transfer matrix, o is the output dimension of knowledge migration module.By Enhanced feature foIt is combined and mentions with primitive character f
The position of high polymorphic type lesion detection and classification performance.
In conclusion one kind of the present invention can automatic customization medicine lesion detection framework and method by automatically for medicine disease
Stove Detection task has customized the detection network architecture head of a suitable medical image, and by knowledge migration module with seamless
Mode realizes a unified polymorphic type lesion detection network, not only increases the precision of lesion detection, also adds multiclass
The function of type lesion detection.The medicine lesion detection network architecture head and knowledge migration module that the present invention realizes can be based on appointing
The general detection network skeleton of meaning natural image, and performance has different degrees of promotion, all realizes detection network skeleton
Optimum performance.
The above-described embodiments merely illustrate the principles and effects of the present invention, and is not intended to limit the present invention.Any
Without departing from the spirit and scope of the present invention, modifications and changes are made to the above embodiments by field technical staff.Therefore,
The scope of the present invention, should be as listed in the claims.
Claims (10)
1. one kind can automatic customization medicine lesion detect framework, comprising:
Candidate feature extraction module extracts the candidate feature of image for carrying out feature extraction to the medical image of input；
Lesion detects network header automatic customization module, for the phase according to medical image characteristic, focus characteristic and target detection
Knowledge is closed, new search space is defined, described search space includes largely having the sub-networks framves such as flexible receptive field, skip floor connection
The advanced operation of structure, and increase a non local operation, the perception relationship between candidate region is merged, according to candidate special
Levy and utilize can micro- NAS algorithm search for suitable operation and connection type in the search space of design and be suitble to its composition one
The best lesion of medical image detects network header；
Lesion detects the optimal module of network header, and the best lesion of network header automatic customization Custom modules is detected for the lesion
Network header is detected, the candidate feature that the candidate feature extraction module is exported, first passing around a convolution kernel is 3 × 3
Then convolutional layer obtains new candidate feature by a standard cell and two contractive cells, and passes through two articulamentums pair
New candidate feature carries out binary classification and prediction block returns, and the weight M that candidate feature in binary classification is classified is as high level
Secondary semantic information is exported to knowledge migration module；
Knowledge migration module is closed on the basis of the components graph that best lesion detection network header learns in conjunction with semanteme
System, and relevant contextual information is transmitted in different regions, to obtain the candidate feature of an enhancing, and will be enhanced
Candidate feature and original candidate feature merge the relevant information for sharing a variety of lesion types, finally by full connection
Layer carries out multivariate classification and recurrence.
2. one kind as described in claim 1 can the medicine lesion of automatic customization detect framework, it is characterised in that: described search is empty
Between include following 9 kinds operation: 1) it is connectionless；2) skip floor connects；3) 3 × 3 average pond；4) non local；5) 1 × 3 and 3 × 1
Convolution；6) 3 × 3 depth separates convolution；7) 5 × 5 depth separates convolution；8) 3 × 3 cavity volumes that expansion rate is 3
Product；9) 3 × 3 empty convolution that expansion rate is 5.
3. one kind as claimed in claim 2 can automatic customization medicine lesion detect framework, it is characterised in that: it is described can be micro-
In NAS algorithm, it is necessary first to according to the suitable search space of task design, then define the module of search, including standard cell
Module and contractive cell module, wherein standard cell module step-length is 1, to keep output and input same resolution ratio, simultaneously
Port number is constant, and contractive cell module step-length is 2, resolution ratio is reduced half, while port number is double, each module, that is, thin
Born of the same parents regard a directed acyclic graph as, define its branch's number, one characteristic pattern of each branching representation, and the connection type between branch indicates
Operation；There are two inputs and an output from branch before to carry out after completing above-mentioned search definition for each branch
Initialize installation, while its discrete topology serialization is made by softmax function, gradient is carried out using gradient descent algorithm later
Passback updates its weight, and finally after the search by certain time, it is maximum to retain weight in 9 kinds of operation connections first
One operation becomes partially connected by an intensive connection, then selects weight maximum two and be connected to the branch
Input, and their result is incorporated as exporting.
4. one kind as claimed in claim 3 can automatic customization medicine lesion detect framework, it is characterised in that: it is described 9 kinds time
Selection operation set is defined as follows:
1) it connectionless operation: is not connected between branch；
2) skip floor attended operation: branch is directly connected to, and does not pass through any operation；
3) 3 × 3 average pondization operation: the average pond that Chi Huahe size is 3 × 3；
4) 1 × 3 and 3 × 1 convolution operation: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 1 × 3, one layer batch of normalizing
Change layer, one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 1, one layer batch of normalization layer；
5) 3 × 3 depth separates convolution operation: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 3, one layer of volume
The convolutional layer that product core is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 3 × 3, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer；
6) 5 × 5 depth separates convolution operation: one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 5 × 5, one layer of volume
The convolutional layer that product core is 1 × 1, one layer batch of normalization layer, one layer of ReLU active coating, the convolutional layer that one layer of convolution kernel is 5 × 5, one
The convolutional layer that layer convolution kernel is 1 × 1, one layer batch of normalization layer；
7) 3 × 3 empty convolution operations that expansion rate is 3: one layer of ReLU active coating, one layer of convolution kernel is 3 × 3 and expansion rate is 3
Convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
8) 3 × 3 empty convolution operations that expansion rate is 5: one layer of ReLU active coating, one layer of convolution kernel is 3 × 3 and expansion rate is 5
Convolutional layer, the convolutional layer that one layer of convolution kernel is 1 × 1, one layer batch of normalization layer；
9) non local operation: the purpose is to encode to the semantic information between region candidate relevant to target detection.
5. one kind as claimed in claim 4 can automatic customization medicine lesion detect framework, it is characterised in that: in the non-office
In portion's operation, by interregional relationship be expressed as a region to region non-directed graph G:G=< N, E >, each section in N
The corresponding region candidate of point, each edge eI, j∈ E encodes the relationship between two nodes, the input of the Non-local operation
ForThe adjacency matrix of non-directed graph G can be by matrix multiplication E=softmax (φ (X) φ (X)T) can be calculated, wherein
φ () is the nonlinear transformation for having ReLU activation primitive；It is propagated in E with picture scroll lamination Y=σ (Ef (X) W) later
Each node, wherein f (), W are a nonlinear transformations, and σ is an activation primitive, and finally connecting full articulamentum makes its input
Output size is consistent.
6. one kind as claimed in claim 5 can the medicine lesion of automatic customization detect framework, it is characterised in that: lesion inspection
The search strategy for surveying network header automatic customization module is to go optimization by discrete topology serialization using stochastic gradient descent algorithm
Parameter expression, go study one group of continuous framework weightThe output tensor of branchFor the weighted blend of candidate operations,
It is expressed asWherein weightFor configuration parameters.
7. one kind as claimed in claim 6 can automatic customization medicine lesion detect framework, which is characterized in that by training data
It is divided into two Uncrossed subsets, i.e. training set and verifying collection, optimization process is carried out by following two steps iteration: 1) being passed throughUpdate network weight w；2) pass throughMore new architecture weight a.Ltrain(w, a) and Lval(w,
It a) is respectively training set and the loss for verifying collection, wherein Ltrain(w, a) and Lval(w a) is respectively training set and the damage for verifying collection
It loses.
8. one kind as claimed in claim 7 can automatic customization medicine lesion detect framework, which is characterized in that the knowledge is moved
Shifting formwork block realizes that process is as follows:
A. the weight and deviation of original binary classification are collectedAs high-level semantic information, P AutoRCNN
Head exports the dimension of tensor, and uses soft mappingDemapping M, wherein sijFrom original two
The region i that classification layer obtains is classified as the score of j.
B. the feature f that figure reasoning is enhanced is carried out by matrix multiplicationo, i.e. fo=E ΓsMWo, wherein It is power
Weight transformation matrix, o are the output dimension of knowledge migration module, and by Enhanced feature foIt is combined and improves with primitive character f
The position of polymorphic type lesion detection and classification performance.
9. one kind can automatic customization medicine lesion detection method, include the following steps:
Step S1 carries out feature extraction using medical image of the candidate feature extraction module to input, extracts the candidate of image
Feature；
Step S2, the candidate for being exported candidate feature extraction module using the lesion detection optimal module of network header of automatic customization
Feature first passes around the convolutional layer that a convolution kernel is 3 × 3, then obtains by a standard cell and two contractive cells
New candidate feature, and binary classification and prediction block recurrence are carried out to new candidate feature by two articulamentums, by binary point
The weight M that candidate feature is classified in class is exported as high-level semantic information to knowledge migration module；
Step S3, using knowledge migration module on the basis of the components graph that best lesion detection network header learns,
In conjunction with semantic relation, and relevant contextual information is transmitted in different regions, to obtain the candidate feature of an enhancing, so
Enhanced candidate feature and original candidate feature are merged to the relevant information for sharing a variety of lesion types afterwards, most
Multivariate classification and recurrence are carried out by full articulamentum afterwards.
10. one kind as claimed in claim 9 can automatic customization medicine lesion detection method, which is characterized in that the method
Further include:
Step S0 defines new search space, institute according to the relevant knowledge of medical image characteristic, focus characteristic and target detection
Stating search space includes the advanced operation largely with sub-networks frameworks such as flexible receptive field, skip floor connections, and increase by one is non-
Partial operation merges the perception relationship between candidate region, according to candidate feature and using can micro- NAS algorithm designing
Search space in the suitable operation of search and connection type so that it is formed the best lesion of suitable medical image detection net
Headstall portion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910743751.5A CN110473195B (en) | 2019-08-13 | 2019-08-13 | Medical focus detection framework and method capable of being customized automatically |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910743751.5A CN110473195B (en) | 2019-08-13 | 2019-08-13 | Medical focus detection framework and method capable of being customized automatically |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110473195A true CN110473195A (en) | 2019-11-19 |
| CN110473195B CN110473195B (en) | 2023-04-18 |
Family
ID=68510625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910743751.5A Active CN110473195B (en) | 2019-08-13 | 2019-08-13 | Medical focus detection framework and method capable of being customized automatically |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110473195B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652226A (en) * | 2020-04-30 | 2020-09-11 | 平安科技（深圳）有限公司 | Image-based target identification method and device and readable storage medium |
| CN111666262A (en) * | 2020-05-28 | 2020-09-15 | 重庆渝抗医药科技有限公司 | Working method for extracting feature points of massive medical images in network attached storage NAS state |
| CN112101428A (en) * | 2020-08-27 | 2020-12-18 | 上海交通大学 | Image classification method and system based on convolutional neural network |
| CN113256500A (en) * | 2021-07-02 | 2021-08-13 | 北京大学第三医院（北京大学第三临床医学院） | Deep learning neural network model system for multi-modal image synthesis |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109284820A (en) * | 2018-10-26 | 2019-01-29 | 北京图森未来科技有限公司 | A kind of search structure method and device of deep neural network |
| CN109543754A (en) * | 2018-11-23 | 2019-03-29 | 中山大学 | The parallel method of target detection and semantic segmentation based on end-to-end deep learning |
| WO2019144575A1 (en) * | 2018-01-24 | 2019-08-01 | 中山大学 | Fast pedestrian detection method and device |
-
2019
- 2019-08-13 CN CN201910743751.5A patent/CN110473195B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019144575A1 (en) * | 2018-01-24 | 2019-08-01 | 中山大学 | Fast pedestrian detection method and device |
| CN109284820A (en) * | 2018-10-26 | 2019-01-29 | 北京图森未来科技有限公司 | A kind of search structure method and device of deep neural network |
| CN109543754A (en) * | 2018-11-23 | 2019-03-29 | 中山大学 | The parallel method of target detection and semantic segmentation based on end-to-end deep learning |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652226A (en) * | 2020-04-30 | 2020-09-11 | 平安科技（深圳）有限公司 | Image-based target identification method and device and readable storage medium |
| CN111652226B (en) * | 2020-04-30 | 2024-05-10 | 平安科技（深圳）有限公司 | Picture-based target identification method and device and readable storage medium |
| CN111666262A (en) * | 2020-05-28 | 2020-09-15 | 重庆渝抗医药科技有限公司 | Working method for extracting feature points of massive medical images in network attached storage NAS state |
| CN111666262B (en) * | 2020-05-28 | 2021-06-22 | 重庆中联信息产业有限责任公司 | Working method for extracting feature points of massive medical images in network attached storage NAS state |
| CN112101428A (en) * | 2020-08-27 | 2020-12-18 | 上海交通大学 | Image classification method and system based on convolutional neural network |
| CN112101428B (en) * | 2020-08-27 | 2023-11-10 | 上海交通大学 | Image classification method and system based on convolutional neural network |
| CN113256500A (en) * | 2021-07-02 | 2021-08-13 | 北京大学第三医院（北京大学第三临床医学院） | Deep learning neural network model system for multi-modal image synthesis |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110473195B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564029B (en) | Face attribute recognition method based on cascade multitask learning deep neural network | |
| Shen et al. | A survey on label-efficient deep image segmentation: Bridging the gap between weak supervision and dense prediction | |
| CN110473195A (en) | It is a kind of can automatic customization medicine lesion detection framework and method | |
| Lei et al. | Shallow convolutional neural network for image classification | |
| CN108062551A (en) | A kind of figure Feature Extraction System based on adjacency matrix, figure categorizing system and method | |
| CN111462282A (en) | Scene graph generation method | |
| CN107818302A (en) | Non-rigid multi-scale object detection method based on convolutional neural network | |
| CN105320961A (en) | Handwriting numeral recognition method based on convolutional neural network and support vector machine | |
| CN112308115B (en) | Multi-label image deep learning classification method and equipment | |
| CN112733866A (en) | Network construction method for improving text description correctness of controllable image | |
| CN112396106A (en) | Content recognition method, content recognition model training method, and storage medium | |
| Liu et al. | Manifold regularized matrix completion for multi-label learning with ADMM | |
| CN108596240B (en) | Image semantic segmentation method based on discriminant feature network | |
| CN110263855B (en) | Method for classifying images by utilizing common-basis capsule projection | |
| Gu et al. | A self-training hierarchical prototype-based ensemble framework for remote sensing scene classification | |
| CN113220897A (en) | Knowledge graph embedding model based on entity-relation association graph | |
| CN110084136A (en) | Context based on super-pixel CRF model optimizes indoor scene semanteme marking method | |
| Zhou et al. | Satellite image scene classification via convnet with context aggregation | |
| CN113807176A (en) | Small sample video behavior identification method based on multi-knowledge fusion | |
| Liu et al. | Iterative deep neighborhood: a deep learning model which involves both input data points and their neighbors | |
| Wu et al. | Semantic-aware graph matching mechanism for multi-label image recognition | |
| Zheng et al. | Multi-class indoor semantic segmentation with deep structured model | |
| Osman et al. | Few-shot learning network for out-of-distribution image classification | |
| Panda et al. | Compositional Zero-Shot Learning using Multi-Branch Graph Convolution and Cross-layer Knowledge Sharing | |
| Cortés et al. | A deep neural network architecture to estimate node assignment costs for the graph edit distance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |