CN108629764A - A kind of good pernicious method and device of determining Lung neoplasm - Google Patents
A kind of good pernicious method and device of determining Lung neoplasm Download PDFInfo
- Publication number
- CN108629764A CN108629764A CN201810345323.2A CN201810345323A CN108629764A CN 108629764 A CN108629764 A CN 108629764A CN 201810345323 A CN201810345323 A CN 201810345323A CN 108629764 A CN108629764 A CN 108629764A
- Authority
- CN
- China
- Prior art keywords
- patient
- lung
- lung neoplasm
- image
- good pernicious
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention discloses a kind of determining good pernicious method and devices of Lung neoplasm.The method includes：After extraction obtains Lung neoplasm image in the lung CT image of patient, default feature extraction neural network model may be used, feature extraction is carried out to the Lung neoplasm image of the patient, and obtain the corresponding feature vector of the patient；Further, patient's corresponding feature vector input can be preset good pernicious Classification Neural model, the Lung neoplasm that the patient to obtain the default good pernicious Classification Neural model output suffers from it is good pernicious.So, compared with the prior art middle doctor determined by way of artificially diagnosing Lung neoplasm it is good it is pernicious for, the embodiment of the present invention is using default feature extraction neural network model, and the good pernicious of Lung neoplasm is determined in conjunction with default good pernicious Classification Neural, the Error Diagnostics rate caused by doctor's level difference can be reduced, to improve the good pernicious accuracy of determining Lung neoplasm.
Description
Technical field
The present invention relates to technical field of data processing more particularly to a kind of good pernicious method and devices of determining Lung neoplasm.
Background technology
Lung neoplasm is a kind of epithelium granulomatous lesion of non-caseous necrosis.Lung neoplasm be divided into it is benign and malignant, it is benign
Lung neoplasm can voluntarily alleviate, it is relatively small to the harm of human body；It is extensive that pernicious Lung neoplasm easily causes functional lesion, lung
Fibrosis even results in lung cancer, to the very harmful of human body.Therefore, it when patient suffers from Lung neoplasm, needs accurately to judge
The Lung neoplasm that patient is suffered from is benign Lung neoplasm or malign lung nodules.
In the prior art, doctor determine Lung neoplasm it is good pernicious when, often by observation Chest Image, this diagnosis
Method needs doctor artificially to judge, less efficient, and is easy by regional healthcare level and doctor's personal experience's water
Flat influence, accuracy are relatively low.
Based on this, there is an urgent need for a kind of determining good pernicious methods of Lung neoplasm at present, and the good pernicious effect of Lung neoplasm is determined to improve
Rate and accuracy.
Invention content
The embodiment of the present invention provides a kind of good pernicious method and device of determining Lung neoplasm, and the good evil of Lung neoplasm is determined to improve
The efficiency and accuracy of property.
The embodiment of the present invention provides a kind of good pernicious method of determining Lung neoplasm, the method includes：
Obtain lung's CT scan CT images of patient；
Determine position of the Lung neoplasm of the patient in the lung CT image of the patient, and from the lung of the patient
Extraction obtains the Lung neoplasm image of the patient in CT images；Wherein, the lung CT image and the Lung neoplasm image are
3-D view；
Feature extraction is carried out to the Lung neoplasm image of the patient using default feature extraction neural network model, obtains institute
State the corresponding feature vector of patient；The parameter of the default feature extraction neural network model is by the multiple patient
Lung neoplasm image is trained；
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, and is obtained described default
The Lung neoplasm that the patient of good pernicious Classification Neural model output suffers from is corresponding good pernicious；Wherein, described preset
The parameter of good pernicious Classification Neural model is the lung knot by being suffered to the corresponding feature vector of multiple patients, each patient
Save corresponding good pernicious be trained.
In this way, the embodiment of the present invention, which uses, presets feature extraction neural network model, and combine default good pernicious classification god
Through network come to Lung neoplasm it is good it is pernicious analyze, since model above is by being trained to obtain to mass data
, so that it is relatively reasonable by the result that model obtains, and there is certain scientific basis.It is examined compared to traditional doctor
For disconnected mode, the Error Diagnostics rate caused by doctor's level difference can be reduced, it is good pernicious to improve determining Lung neoplasm
Accuracy；Further, Lung neoplasm is analyzed by neural network model, it is good pernicious that determining Lung neoplasm can be greatly improved
Efficiency.
In one possible implementation, by the default good pernicious classification nerve of the corresponding feature vector input of the patient
Network model, and the Lung neoplasm that the patient for obtaining the default good pernicious Classification Neural model output suffers from is corresponding
It is good pernicious, including：
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, the patient is obtained and suffers from
There are the corresponding confidence level of benign Lung neoplasm and the patient to suffer from the corresponding confidence level of malign lung nodules；
If the patient is greater than or equal to the patient with evil with the corresponding confidence level of benign Lung neoplasm
Property the corresponding confidence level of Lung neoplasm, then it is benign to obtain Lung neoplasm that the patient suffers from；
If the patient is less than the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung knot
Corresponding confidence level is saved, then it is pernicious to obtain the Lung neoplasm that the patient suffers from.
In one possible implementation, the parameter of the preset good pernicious Classification Neural model is by right
Lung neoplasm that the corresponding feature vector of multiple patients, each patient suffer from it is corresponding it is good it is pernicious be trained, including：
The corresponding feature vector of the multiple patient is input to initial good pernicious Classification Neural model, is obtained every
The corresponding prediction of Lung neoplasm that a patient suffers from is good pernicious；
The corresponding Lung neoplasm predicted good pernicious and each patient and suffered from of Lung neoplasm suffered from according to each patient
Corresponding reality is good pernicious, carries out reverse train, generates the default good pernicious Classification Neural model.
In this way, by predict the good pernicious and practical Lung neoplasm of Lung neoplasm it is good it is pernicious between comparison, can accurately adjust
The parameter for presetting good pernicious Classification Neural model, that improves generation presets the accurate of good pernicious Classification Neural model
Degree.
In one possible implementation, determine the Lung neoplasm of the patient in the lung CT image of the patient
Position, and extraction obtains the Lung neoplasm image of the patient from the lung CT image of the patient, including：
Determine centre coordinate and radius of the Lung neoplasm of the patient in the lung CT image of the patient；
According to the centre coordinate and the radius, and extraction obtains the patient from the lung CT image of the patient
Lung neoplasm image.
In one possible implementation, the default feature extraction neural network model includes N number of convolution module；N
Less than or equal to first threshold；
Wherein, each convolution module includes convolutional layer, BN layers of the normalization being connect with the convolutional layer, connects with described BN layers
The activation primitive layer connect and pooling layers of max being connect with the activation primitive layer.
The embodiment of the present invention provides a kind of good pernicious device of determining Lung neoplasm, and described device includes：
Acquiring unit, lung's CT scan CT images for obtaining patient；
Processing unit, for determining position of the Lung neoplasm of the patient in the lung CT image of the patient, and from
Extraction obtains the Lung neoplasm image of the patient in the lung CT image of the patient；Wherein, the lung CT image and described
Lung neoplasm image is 3-D view；
The processing unit, for using preset feature extraction neural network model to the Lung neoplasm image of the patient into
Row feature extraction obtains the corresponding feature vector of the patient；The parameter of the default feature extraction neural network model is logical
It crosses and the Lung neoplasm image of the multiple patient is trained；And the corresponding feature vector of the patient is inputted pre-
If good pernicious Classification Neural model, and the patient for obtaining the default good pernicious Classification Neural model output suffers from
Some Lung neoplasms are corresponding good pernicious；Wherein, the parameter of the preset good pernicious Classification Neural model is by more
Lung neoplasm that the corresponding feature vector of a patient, each patient suffer from is corresponding good pernicious to be trained.
In one possible implementation, the processing unit is specifically used for：
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, the patient is obtained and suffers from
There are the corresponding confidence level of benign Lung neoplasm and the patient to suffer from the corresponding confidence level of malign lung nodules；
If the patient is greater than or equal to the patient with evil with the corresponding confidence level of benign Lung neoplasm
Property the corresponding confidence level of Lung neoplasm, then it is benign to obtain Lung neoplasm that the patient suffers from；
If the patient is less than the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung knot
Corresponding confidence level is saved, then it is pernicious to obtain the Lung neoplasm that the patient suffers from.
In one possible implementation, the processing unit is specifically used for：
The corresponding feature vector of the multiple patient is input to initial good pernicious Classification Neural model, is obtained every
The corresponding prediction of Lung neoplasm that a patient suffers from is good pernicious；
The corresponding Lung neoplasm predicted good pernicious and each patient and suffered from of Lung neoplasm suffered from according to each patient
Corresponding reality is good pernicious, carries out reverse train, generates the default good pernicious Classification Neural model.
In one possible implementation, the processing unit is specifically used for：
Determine centre coordinate and radius of the Lung neoplasm of the patient in the lung CT image of the patient；
According to the centre coordinate and the radius, and extraction obtains the patient from the lung CT image of the patient
Lung neoplasm image.
In one possible implementation, the default feature extraction neural network model includes N number of convolution module；N
Less than or equal to first threshold；
Wherein, each convolution module includes convolutional layer, BN layers of the normalization being connect with the convolutional layer, connects with described BN layers
The activation primitive layer connect and pooling layers of max being connect with the activation primitive layer.
The embodiment of the present invention also provides a kind of device, and described device can be to determine the good pernicious device of Lung neoplasm, described
Device includes：
Memory, for storing software program；
Processor, for reading the software program in the memory and executing institute in above-mentioned various possible realization methods
The good pernicious method of determination Lung neoplasm of description.
The embodiment of the present invention also provides a kind of computer storage media, stores software program in the storage medium, this is soft
Part program is realized described in above-mentioned various possible realization methods really when being read and executed by one or more processors
Determine the good pernicious method of Lung neoplasm.
The embodiment of the present invention also provides a kind of computer program product including instruction, when run on a computer,
So that computer executes the good pernicious method of determination Lung neoplasm described in above-mentioned various possible realization methods.
Description of the drawings
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced.
Fig. 1 is the flow diagram corresponding to a kind of determining good pernicious method of Lung neoplasm provided in an embodiment of the present invention；
Fig. 2 a are a kind of schematic diagram of lung CT image provided in an embodiment of the present invention；
Fig. 2 b are the schematic diagram of the Lung neoplasm image after a kind of extraction provided in an embodiment of the present invention；
Fig. 3 is a kind of structural schematic diagram of default feature extraction neural network model provided in an embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram for presetting good pernicious Classification Neural model provided in an embodiment of the present invention；
Fig. 5 is a kind of structural schematic diagram of the determining good pernicious device of Lung neoplasm provided in an embodiment of the present invention.
Specific implementation mode
The application is specifically described with reference to the accompanying drawings of the specification, the concrete operation method in embodiment of the method also may be used
To be applied in device embodiment.
Fig. 1 illustrates the stream corresponding to a kind of determining good pernicious method of Lung neoplasm provided in an embodiment of the present invention
Journey schematic diagram, as shown in Figure 1, including the following steps：
Step 101, lung's CT scan CT images of patient are obtained.
Step 102, position of the Lung neoplasm of the patient in the lung CT image of the patient is determined, and from the trouble
Extraction obtains the Lung neoplasm image of the patient in the lung CT image of person.
Step 103, feature is carried out to the Lung neoplasm image of the patient using default feature extraction neural network model to carry
It takes, obtains the corresponding feature vector of the patient.
Step 104, good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, and obtains
The Lung neoplasm that the patient of the default good pernicious Classification Neural model output suffers from is corresponding good pernicious.
In this way, the embodiment of the present invention, which uses, presets feature extraction neural network model, and combine default good pernicious classification god
Through network come to Lung neoplasm it is good it is pernicious analyze, since model above is by being trained to obtain to mass data
, so that it is relatively reasonable by the result that model obtains, and there is certain scientific basis.It is examined compared to traditional doctor
For disconnected mode, the Error Diagnostics rate caused by doctor's level difference can be reduced, it is good pernicious to improve determining Lung neoplasm
Accuracy；Further, Lung neoplasm is analyzed by neural network model, it is good pernicious that determining Lung neoplasm can be greatly improved
Efficiency.
Specifically, in step 101 and step 102, Lung neoplasm image refers to the figure extracted from lung CT image
Picture.As shown in Figure 2 a, it is a kind of schematic diagram of lung CT image.It can see from Fig. 2 a, in the location A of the lung CT image
There are a Lung neoplasms.
Further, there are many modes that Lung neoplasm image is extracted from lung CT image, for example, it may be directly from lung
Figure is scratched in portion's CT images obtains Lung neoplasm image；Alternatively, can also be according to Lung neoplasm in the lung CT image of patient in
Heart coordinate and radius obtain Lung neoplasm image from lung CT image.
For obtaining Lung neoplasm image according to centre coordinate and radius, as shown in Figure 2 b, provided for the embodiment of the present invention
A kind of extraction after Lung neoplasm image schematic diagram.Wherein, the centre coordinate of position A of the Lung neoplasm in the lung CT image
For (x0, y0, z0).According to fig. 2 the content shown in b it is found that the Lung neoplasm be irregular figure, if in the Lung neoplasm, distance center
Coordinate (x0, y0, z0) nearest distance is D1, and distance center coordinate (x0, y0, z0) nearest distance is D2, then it can basis
Centre coordinate (x0, y0, z0) centered on point, using 2 times of D2 as the length of side, obtained region (the i.e. area that dotted line is outlined in Fig. 2
Domain) it is the corresponding Lung neoplasm image of the Lung neoplasm.
In view of in actual conditions, the size of each Lung neoplasm is not consistent, and each Lung neoplasm may be irregular component
Shape.Based on the above situation, in the embodiment of the present invention, can centered on the centre coordinate of Lung neoplasm point, with Lung neoplasm
2.5 times, 3 times of major diameter are the length of side, and using obtained region as Lung neoplasm image.It so, it is possible to ensure obtained lung
Nodule image can include the region where entire Lung neoplasm, avoid the parts of images for omitting Lung neoplasm.
It should be noted that：(1) the corresponding region of Lung neoplasm image can be various shapes, cuboid as described above
The corresponding region of Lung neoplasm image be only a kind of example, in other possible examples, the corresponding region of Lung neoplasm image
Can be sphere or other shapes；(2) since lung CT image is 3-D view, it extracts and obtains from lung CT image
Lung neoplasm image also be 3-D view.
Further, it after obtaining Lung neoplasm image, can be for further processing to Lung neoplasm image, to expand
The training sample amount of default feature extraction neural network model, and then improve and preset the accurate of feature extraction neural network model
Degree.In the embodiment of the present invention, to there are many processing modes of Lung neoplasm image, for example, processing mode can be horizontal translation, on
Lower translation, horizontal mirror image, vertical mirror, rotation, scaling etc., in specific implementation process, those skilled in the art can be only with
A kind of above-mentioned processing mode handles Lung neoplasm image, or can also use a variety of processing modes to Lung neoplasm image into
Row processing, does not limit specifically.
It should be noted that the above-mentioned processing mode to Lung neoplasm image can also be applied in the processing to lung CT image
On.For example, the processing modes such as translation, rotation, scaling can be taken to carry out deformation process lung CT image, so as to expand
The data volume of lung CT image.
In step 103, the parameter for presetting feature extraction neural network model can be by the Lung neoplasm to multiple patients
What image was trained.Wherein, it can be shallow-layer neural network model to preset feature extraction neural network model, i.e., this is pre-
If feature extraction neural network may include N number of convolution module, and, N is less than or equal to first threshold.Those skilled in the art can
Rule of thumb to set the concrete numerical value of first threshold with actual conditions, do not limit herein.
In order to which according to default feature extraction neural network model referred to above is clearly described, Fig. 3 is exemplary
Show a kind of structural schematic diagram of default feature extraction neural network model provided in an embodiment of the present invention.The default feature carries
It may include three convolution modules to take neural network model.As shown in figure 3, three convolution modules are respectively the first convolution module
301, the second convolution module 302 and third convolution module 303；Wherein, each convolution module may include convolutional layer and convolution again
Normalization (Batch Normalization, BN) layer of layer connection, the activation primitive layer that is connect with BN layers and with activate letter
Pooling layer of the max of several layers of connection, the first convolution module 301 as shown in Figure 3 include the first convolutional layer the 3011, the first BN layers
3012, the first activation primitive layer 3013 and the first max pooling layers 3014, the second convolution module 302 include the second convolutional layer
3021, the 2nd BN layers 3022, the second activation primitive layer 3023 and the 2nd max pooling layers 3024, third convolution module 303 are wrapped
Include third convolutional layer 3031, the 3rd BN layers 3032, third activation primitive layer 3033 and the 3rd max pooling layers 3034.
It should be noted that：(1) activation primitive shown in Fig. 3 can be a plurality of types of activation primitives, for example, can be with
For line rectification function (Rectified Linear Unit, ReLU), do not limit specifically；(2) each volume shown in Fig. 3
The convolution kernel size of lamination, the convolution kernel size of pooling layers of max, the extraction of each convolution module feature channel numerical value can be with
It rule of thumb sets for those skilled in the art and adjusts with actual conditions, do not limit specifically；(3) due to the present invention
The image inputted in embodiment is 3-D view, and therefore, the default feature extraction neural network model in the embodiment of the present invention can
Think (3Dimensions, 3D) convolutional neural networks, correspondingly, the corresponding convolution kernel size of 3D convolutional neural networks can be
M*m*m, wherein m is the integer more than or equal to 1.
The method of determination of the corresponding feature vector of patient as described above is only a kind of possible realization method, other
In possible realization method, the corresponding feature vector of patient can also be determined otherwise, is not limited specifically.
In step 104, the parameter of preset good pernicious Classification Neural model is by the corresponding spy of multiple patients
Lung neoplasm that sign vector, each patient suffer from is corresponding good pernicious to be trained.
Fig. 4 illustrates a kind of structure for presetting good pernicious Classification Neural model provided in an embodiment of the present invention
Schematic diagram, as shown in figure 4, it includes the first full articulamentum 401, the second full articulamentum that this, which presets good pernicious Classification Neural model,
402 and softmax layers 403.It is complete to wait for that the corresponding feature vector of patient diagnosed can pass sequentially through the first full articulamentum 401, second
After articulamentum 402 is calculated, then output category result after being classified by softmax layers 403, suffer to obtaining patient
Lung neoplasm it is good pernicious.
It should be noted that preset good pernicious Classification Neural model illustrated in fig. 4 is only a kind of possible knot
Structure, in other examples, those skilled in the art can modify to content illustrated in fig. 4, for example, preset good pernicious
Classification Neural model may include three full articulamentums, not limit specifically.
In the embodiment of the present invention, training is described preset good pernicious Classification Neural model during, can will be more
The corresponding feature vector of a patient is input to initial good pernicious Classification Neural model, obtains the lung knot that each patient suffers from
It is good pernicious to save corresponding prediction, and good pernicious and each patient is predicted according to the Lung neoplasm that each patient suffers from is corresponding
The corresponding reality of Lung neoplasm suffered from is good pernicious, carries out reverse train, generates the default good pernicious Classification Neural model.
It is one kind of the training set of preset good pernicious Classification Neural model as shown in table 1 in specific training process
Example.The feature vector of patient 1 is X1, the Lung neoplasm suffered from is benign Lung neoplasm；The feature vector of patient 2 is X2, the lung that suffers from
Tubercle is malign lung nodules；The feature vector of patient 3 is X3, the Lung neoplasm suffered from is malign lung nodules；The feature vector of patient 4
For X4, the Lung neoplasm suffered from is benign Lung neoplasm；The feature vector of patient 5 is X5, the Lung neoplasm suffered from is malign lung nodules.
Table 1：A kind of example of the training set of preset good pernicious Classification Neural model
| Number | Feature vector | Lung neoplasm is good pernicious |
| Patient 1 | X1 | It is benign |
| Patient 2 | X2 | It is pernicious |
| Patient 3 | X3 | It is pernicious |
| Patient 4 | X4 | It is benign |
| Patient 5 | X5 | It is pernicious |
| …… | …… | …… |
Further, by the feature vector of multiple patients shown in table 1, and the Lung neoplasm that suffers from of each patient is good
Pernicious input is preset in good pernicious Classification Neural model, it may be determined that presets the ginseng of good pernicious Classification Neural model
Number.Specifically, the corresponding feature vector of multiple patients can be first input to initial good pernicious Classification Neural model,
The prediction Lung neoplasm for obtaining the Lung neoplasm that each patient suffers from is good pernicious, the prediction of the Lung neoplasm then suffered from according to each patient
The practical Lung neoplasm for the Lung neoplasm that Lung neoplasm is good pernicious and each patient suffers from is good pernicious, carries out reverse train, generates default
Good pernicious Classification Neural model.
It should be noted that when good pernicious Classification Neural model is preset in training, Lung neoplasm that multiple patients suffer from
Reality good pernicious can be determined by doctor.
For example, by taking patient 1 shown in table 1 as an example, by 1 corresponding feature vector, X of patient1Input is default good pernicious
In Classification Neural, by propagated forward, the result vector of one 2 dimension can be obtainedWherein, y1For benign lung knot
Save corresponding confidence level；y2For the corresponding confidence level of malign lung nodules.According to the result vector of the good pernicious Classification NeuralIt is good pernicious to obtain the corresponding prediction of patient 1；Further, if 1 corresponding prediction Lung neoplasm of patient is good pernicious for evil
Property, and the corresponding practical good pernicious content according to shown in table 1 of Lung neoplasm of patient 1 is benign, good pernicious classification god default in this way
Error is there is between prediction result and actual result through network model, that is, loses (loss) functional value.In turn, it can use
Back-propagation algorithm, according to stochastic gradient descent (Stochastic gradient descent, SGD) algorithm along loss
(loss) parameter of good pernicious Classification Neural model is preset in the direction adjustment that functional value declines.In this way, by predicting good evil
Property confidence level and practical good pernicious confidence level between comparison, can accurately adjust default good pernicious Classification Neural
The parameter of model improves the accuracy for presetting good pernicious Classification Neural model of generation.
It further, can be by patient couple after obtaining trained default good pernicious Classification Neural model
Answer feature vector input preset good pernicious Classification Neural model, obtain the patient with benign Lung neoplasm confidence level and
Confidence level with malign lung nodules；If the patient is greater than or equal to described with the corresponding confidence level of benign Lung neoplasm
The patient suffers from the corresponding confidence level of malign lung nodules, it is determined that the Lung neoplasm that the patient suffers from is benign；If the institute
It states patient and is less than the patient with the corresponding confidence level of malign lung nodules, then with the corresponding confidence level of benign Lung neoplasm
Determine that the Lung neoplasm that the patient suffers from is pernicious.
In this way, compared with the prior art in doctor determined by way of artificially diagnosing Lung neoplasm it is good it is pernicious for, this
Inventive embodiments determine lung knot using presetting feature extraction neural network model, and in conjunction with default good pernicious Classification Neural
That saves is good pernicious, can reduce the Error Diagnostics rate caused by doctor's level difference, good pernicious to improve determining Lung neoplasm
Accuracy；Further, Lung neoplasm is analyzed by neural network model, it is good pernicious that determining Lung neoplasm can be greatly improved
Efficiency.
Based on identical inventive concept, the embodiment of the present invention provides a kind of good pernicious device of determining Lung neoplasm, such as Fig. 5 institutes
Show, which includes acquiring unit 501 and processing unit 502；Wherein,
Acquiring unit 501, lung's CT scan CT images for obtaining patient；
Processing unit 502, for determining position of the Lung neoplasm of the patient in the lung CT image of the patient, and
Extraction obtains the Lung neoplasm image of the patient from the lung CT image of the patient；Wherein, the lung CT image and institute
It is 3-D view to state Lung neoplasm image；
The processing unit 502, for the Lung neoplasm figure using default feature extraction neural network model to the patient
As carrying out feature extraction, the corresponding feature vector of the patient is obtained；The parameter of the default feature extraction neural network model
It is to be trained by the Lung neoplasm image to the multiple patient；And it is the corresponding feature vector of the patient is defeated
Enter default good pernicious Classification Neural model, and obtains the trouble of the default good pernicious Classification Neural model output
The Lung neoplasm that person suffers from is corresponding good pernicious；Wherein, the parameter of the preset good pernicious Classification Neural model is to pass through
The Lung neoplasm that suffers to the corresponding feature vector of multiple patients, each patient is corresponding good pernicious to be trained.
In one possible implementation, the processing unit 502 is specifically used for：
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, the patient is obtained and suffers from
There are the corresponding confidence level of benign Lung neoplasm and the patient to suffer from the corresponding confidence level of malign lung nodules；
If the patient is greater than or equal to the patient with evil with the corresponding confidence level of benign Lung neoplasm
Property the corresponding confidence level of Lung neoplasm, then it is benign to obtain Lung neoplasm that the patient suffers from；
If the patient is less than the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung knot
Corresponding confidence level is saved, then it is pernicious to obtain the Lung neoplasm that the patient suffers from.
In one possible implementation, the processing unit 502 is specifically used for：
The corresponding feature vector of the multiple patient is input to initial good pernicious Classification Neural model, is obtained every
The corresponding prediction of Lung neoplasm that a patient suffers from is good pernicious；
The corresponding Lung neoplasm predicted good pernicious and each patient and suffered from of Lung neoplasm suffered from according to each patient
Corresponding reality is good pernicious, carries out reverse train, generates the default good pernicious Classification Neural model.
In one possible implementation, the processing unit 502 is specifically used for：
Determine centre coordinate and radius of the Lung neoplasm of the patient in the lung CT image of the patient；
According to the centre coordinate and the radius, and extraction obtains the patient from the lung CT image of the patient
Lung neoplasm image.
In one possible implementation, the default feature extraction neural network model includes N number of convolution module；N
Less than or equal to first threshold；
Wherein, each convolution module includes convolutional layer, BN layers of the normalization being connect with the convolutional layer, connects with described BN layers
The activation primitive layer connect and pooling layers of max being connect with the activation primitive layer.
The embodiment of the present invention also provides a kind of device, and described device can be to determine the good pernicious device of Lung neoplasm, described
Device includes：
Memory, for storing software program；
Processor, for reading the software program in the memory and executing institute in above-mentioned various possible realization methods
The good pernicious method of determination Lung neoplasm of description.
The embodiment of the present invention also provides a kind of computer storage media, stores software program in the storage medium, this is soft
Part program is realized described in above-mentioned various possible realization methods really when being read and executed by one or more processors
Determine the good pernicious method of Lung neoplasm.
The embodiment of the present invention also provides a kind of computer program product including instruction, when run on a computer,
So that computer executes the good pernicious method of determination Lung neoplasm described in above-mentioned various possible realization methods.
It should be understood by those skilled in the art that, the embodiment of the present invention can be provided as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, the present invention can be used in one or more wherein include computer usable program code computer
The computer program production implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The present invention be with reference to according to the method for the embodiment of the present invention, the flow of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions every first-class in flowchart and/or the block diagram
The combination of flow and/or box in journey and/or box and flowchart and/or the block diagram.These computer programs can be provided
Instruct the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine so that the instruction executed by computer or the processor of other programmable data processing devices is generated for real
The device for the function of being specified in present one flow of flow chart or one box of multiple flows and/or block diagram or multiple boxes.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
God and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of good pernicious method of determining Lung neoplasm, which is characterized in that the method includes：
Obtain lung's CT scan CT images of patient；
Determine position of the Lung neoplasm of the patient in the lung CT image of the patient, and from the lung CT figure of the patient
Extraction obtains the Lung neoplasm image of the patient as in；Wherein, the lung CT image and the Lung neoplasm image are three-dimensional
Image；
Feature extraction is carried out to the Lung neoplasm image of the patient using default feature extraction neural network model, obtains the trouble
The corresponding feature vector of person；The parameter of the default feature extraction neural network model is by the lung knot to the multiple patient
Section image is trained；
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, and obtains the default good evil
Property the output of Classification Neural model the Lung neoplasm that suffers from of the patient it is corresponding good pernicious；Wherein, the preset good evil
The parameter of property Classification Neural model is the Lung neoplasm pair by being suffered to the corresponding feature vector of multiple patients, each patient
That answers good pernicious is trained.
2. according to the method described in claim 1, it is characterized in that, good evil is preset in the corresponding feature vector input of the patient
Property Classification Neural model, and the lung that the patient for obtaining the default good pernicious Classification Neural model output suffers from
Tubercle is corresponding good pernicious, including：
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, obtains the patient with good
Property the corresponding confidence level of Lung neoplasm and the patient suffer from the corresponding confidence level of malign lung nodules；
If the patient is greater than or equal to the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung
The corresponding confidence level of tubercle, then it is benign to obtain the Lung neoplasm that the patient suffers from；
If the patient is less than the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung nodules pair
The confidence level answered, then it is pernicious to obtain the Lung neoplasm that the patient suffers from.
3. according to the method described in claim 1, it is characterized in that, the ginseng of the preset good pernicious Classification Neural model
Number is good pernicious to be trained to obtain by the way that the Lung neoplasm for suffering from the corresponding feature vector of multiple patients, each patient is corresponding
, including：
The corresponding feature vector of the multiple patient is input to initial good pernicious Classification Neural model, obtains each trouble
The corresponding prediction of Lung neoplasm that person suffers from is good pernicious；
The corresponding Lung neoplasm for predicting that good pernicious and each patient suffers from of Lung neoplasm suffered from according to each patient corresponds to
Reality it is good pernicious, carry out reverse train, generate it is described preset good pernicious Classification Neural model.
4. according to the method described in claim 1, it is characterized in that, determining lung of the Lung neoplasm in the patient of the patient
Position in CT images, and extraction obtains the Lung neoplasm image of the patient from the lung CT image of the patient, including：
Determine centre coordinate and radius of the Lung neoplasm of the patient in the lung CT image of the patient；
According to the centre coordinate and the radius, and extraction obtains the lung of the patient from the lung CT image of the patient
Nodule image.
5. method according to claim 1 to 4, which is characterized in that the default feature extraction neural network
Model includes N number of convolution module；N is less than or equal to first threshold；
Wherein, each convolution module includes convolutional layer, BN layers of normalization being connect with the convolutional layer, connect with described BN layers
Activation primitive layer and pooling layers of max being connect with the activation primitive layer.
6. a kind of good pernicious device of determining Lung neoplasm, which is characterized in that described device includes：
Acquiring unit, lung's CT scan CT images for obtaining patient；
Processing unit, for determining position of the Lung neoplasm of the patient in the lung CT image of the patient, and from described
Extraction obtains the Lung neoplasm image of the patient in the lung CT image of patient；Wherein, the lung CT image and the lung knot
It is 3-D view to save image；
The processing unit, for special to the Lung neoplasm image progress of the patient using feature extraction neural network model is preset
Sign extraction, obtains the corresponding feature vector of the patient；The parameter of the default feature extraction neural network model is by right
What the Lung neoplasm image of the multiple patient was trained；And it is the corresponding feature vector input of the patient is default good
Pernicious Classification Neural model, and obtain what the patient that the default good pernicious Classification Neural model exports suffered from
Lung neoplasm is corresponding good pernicious；Wherein, the parameter of the preset good pernicious Classification Neural model is by multiple trouble
Lung neoplasm that the corresponding feature vector of person, each patient suffer from is corresponding good pernicious to be trained.
7. device according to claim 6, which is characterized in that the processing unit is specifically used for：
Good pernicious Classification Neural model is preset into the corresponding feature vector input of the patient, obtains the patient with good
Property the corresponding confidence level of Lung neoplasm and the patient suffer from the corresponding confidence level of malign lung nodules；
If the patient is greater than or equal to the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung
The corresponding confidence level of tubercle, then it is benign to obtain the Lung neoplasm that the patient suffers from；
If the patient is less than the patient with the corresponding confidence level of benign Lung neoplasm suffers from malign lung nodules pair
The confidence level answered, then it is pernicious to obtain the Lung neoplasm that the patient suffers from.
8. device according to claim 6, which is characterized in that the processing unit is specifically used for：
The corresponding feature vector of the multiple patient is input to initial good pernicious Classification Neural model, obtains each trouble
The corresponding prediction of Lung neoplasm that person suffers from is good pernicious；
The corresponding Lung neoplasm for predicting that good pernicious and each patient suffers from of Lung neoplasm suffered from according to each patient corresponds to
Reality it is good pernicious, carry out reverse train, generate it is described preset good pernicious Classification Neural model.
9. device according to claim 6, which is characterized in that the processing unit is specifically used for：
Determine centre coordinate and radius of the Lung neoplasm of the patient in the lung CT image of the patient；
According to the centre coordinate and the radius, and extraction obtains the lung of the patient from the lung CT image of the patient
Nodule image.
10. the device according to any one of claim 6 to 9, which is characterized in that the default feature extraction neural network
Model includes N number of convolution module；N is less than or equal to first threshold；
Wherein, each convolution module includes convolutional layer, BN layers of normalization being connect with the convolutional layer, connect with described BN layers
Activation primitive layer and pooling layers of max being connect with the activation primitive layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810345323.2A CN108629764A (en) | 2018-04-17 | 2018-04-17 | A kind of good pernicious method and device of determining Lung neoplasm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810345323.2A CN108629764A (en) | 2018-04-17 | 2018-04-17 | A kind of good pernicious method and device of determining Lung neoplasm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108629764A true CN108629764A (en) | 2018-10-09 |
Family
ID=63705290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810345323.2A Pending CN108629764A (en) | 2018-04-17 | 2018-04-17 | A kind of good pernicious method and device of determining Lung neoplasm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108629764A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109363698A (en) * | 2018-10-16 | 2019-02-22 | 杭州依图医疗技术有限公司 | A kind of method and device of breast image sign identification |
| CN109461144A (en) * | 2018-10-16 | 2019-03-12 | 杭州依图医疗技术有限公司 | A kind of method and device of breast image identification |
| CN109528152A (en) * | 2019-01-22 | 2019-03-29 | 湖南兰茜生物科技有限公司 | A kind of novel tuberculosis intelligence aided detection method and system |
| CN109741312A (en) * | 2018-12-28 | 2019-05-10 | 上海联影智能医疗科技有限公司 | A kind of Lung neoplasm discrimination method, device, equipment and medium |
| CN110175979A (en) * | 2019-04-08 | 2019-08-27 | 杭州电子科技大学 | A kind of Lung neoplasm classification method based on collaboration attention mechanism |
| CN110321827A (en) * | 2019-06-27 | 2019-10-11 | 嘉兴深拓科技有限公司 | A kind of pain level appraisal procedure based on face pain expression video |
| CN110610488A (en) * | 2019-08-29 | 2019-12-24 | 上海杏脉信息科技有限公司 | Classification training and detecting method and device |
| CN110992312A (en) * | 2019-11-15 | 2020-04-10 | 上海联影智能医疗科技有限公司 | Medical image processing method, device, storage medium and computer equipment |
| CN111145140A (en) * | 2018-11-02 | 2020-05-12 | 西门子医疗有限公司 | Determining malignancy of lung nodules using deep learning |
| CN112017787A (en) * | 2020-08-17 | 2020-12-01 | 杭州依图医疗技术有限公司 | Medical information AI platform and computing equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700118A (en) * | 2015-03-18 | 2015-06-10 | 中国科学院自动化研究所 | Pulmonary nodule benignity and malignancy predicting method based on convolutional neural networks |
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| WO2017202185A1 (en) * | 2016-05-25 | 2017-11-30 | 上海伯豪医学检验所有限公司 | Peripheral blood gene marker for screening benign and malignant small pulmonary nodules and use thereof |
| CN107451609A (en) * | 2017-07-24 | 2017-12-08 | 上海交通大学 | Lung neoplasm image identification system based on depth convolutional neural networks |
| CN107578402A (en) * | 2017-08-04 | 2018-01-12 | 上海斐讯数据通信技术有限公司 | A kind of detecting system and its application method of the pulmonary nodule based on deep learning |
| CN107616796A (en) * | 2017-08-31 | 2018-01-23 | 北京医拍智能科技有限公司 | The good pernicious detection method and device of Lung neoplasm based on deep neural network |
-
2018
- 2018-04-17 CN CN201810345323.2A patent/CN108629764A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700118A (en) * | 2015-03-18 | 2015-06-10 | 中国科学院自动化研究所 | Pulmonary nodule benignity and malignancy predicting method based on convolutional neural networks |
| WO2017202185A1 (en) * | 2016-05-25 | 2017-11-30 | 上海伯豪医学检验所有限公司 | Peripheral blood gene marker for screening benign and malignant small pulmonary nodules and use thereof |
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| CN107451609A (en) * | 2017-07-24 | 2017-12-08 | 上海交通大学 | Lung neoplasm image identification system based on depth convolutional neural networks |
| CN107578402A (en) * | 2017-08-04 | 2018-01-12 | 上海斐讯数据通信技术有限公司 | A kind of detecting system and its application method of the pulmonary nodule based on deep learning |
| CN107616796A (en) * | 2017-08-31 | 2018-01-23 | 北京医拍智能科技有限公司 | The good pernicious detection method and device of Lung neoplasm based on deep neural network |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109461144B (en) * | 2018-10-16 | 2021-02-23 | 杭州依图医疗技术有限公司 | Method and device for identifying mammary gland image |
| CN109461144A (en) * | 2018-10-16 | 2019-03-12 | 杭州依图医疗技术有限公司 | A kind of method and device of breast image identification |
| CN109363698A (en) * | 2018-10-16 | 2019-02-22 | 杭州依图医疗技术有限公司 | A kind of method and device of breast image sign identification |
| CN111145140B (en) * | 2018-11-02 | 2023-11-07 | 西门子医疗有限公司 | Determining malignancy of lung nodules using deep learning |
| CN111145140A (en) * | 2018-11-02 | 2020-05-12 | 西门子医疗有限公司 | Determining malignancy of lung nodules using deep learning |
| CN109741312A (en) * | 2018-12-28 | 2019-05-10 | 上海联影智能医疗科技有限公司 | A kind of Lung neoplasm discrimination method, device, equipment and medium |
| CN109528152A (en) * | 2019-01-22 | 2019-03-29 | 湖南兰茜生物科技有限公司 | A kind of novel tuberculosis intelligence aided detection method and system |
| CN110175979A (en) * | 2019-04-08 | 2019-08-27 | 杭州电子科技大学 | A kind of Lung neoplasm classification method based on collaboration attention mechanism |
| CN110321827A (en) * | 2019-06-27 | 2019-10-11 | 嘉兴深拓科技有限公司 | A kind of pain level appraisal procedure based on face pain expression video |
| CN110610488A (en) * | 2019-08-29 | 2019-12-24 | 上海杏脉信息科技有限公司 | Classification training and detecting method and device |
| CN110992312A (en) * | 2019-11-15 | 2020-04-10 | 上海联影智能医疗科技有限公司 | Medical image processing method, device, storage medium and computer equipment |
| CN110992312B (en) * | 2019-11-15 | 2024-02-27 | 上海联影智能医疗科技有限公司 | Medical image processing method, medical image processing device, storage medium and computer equipment |
| CN112017787A (en) * | 2020-08-17 | 2020-12-01 | 杭州依图医疗技术有限公司 | Medical information AI platform and computing equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108629764A (en) | A kind of good pernicious method and device of determining Lung neoplasm | |
| US20210192758A1 (en) | Image processing method and apparatus, electronic device, and computer readable storage medium | |
| CN108648179A (en) | A kind of method and device of analysis Lung neoplasm | |
| CN108564044A (en) | A kind of method and device of determining Lung neoplasm density | |
| CN110574077A (en) | Image analysis device and method using virtual three-dimensional deep neural network | |
| CN109035187A (en) | A kind of mask method and device of medical image | |
| CN109363698A (en) | A kind of method and device of breast image sign identification | |
| US10366488B2 (en) | Image processing used to estimate abnormalities | |
| CN109447088A (en) | A kind of method and device of breast image identification | |
| CN104484886B (en) | A kind of dividing method and device of MR images | |
| CN111768375B (en) | Asymmetric GM multi-mode fusion significance detection method and system based on CWAM | |
| JP2022517769A (en) | 3D target detection and model training methods, equipment, equipment, storage media and computer programs | |
| CN109461144A (en) | A kind of method and device of breast image identification | |
| CN114708255A (en) | Multi-center children X-ray chest image lung segmentation method based on TransUNet model | |
| CN113782184A (en) | Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning | |
| CN111179235A (en) | Image detection model generation method and device, and application method and device | |
| CN116091490A (en) | Lung nodule detection method based on YOLOv4-CA-CBAM-K-means++ -SIOU | |
| CN113239951A (en) | Ultrasonic breast lesion classification method and device and storage medium | |
| CN112036298A (en) | Cell detection method based on double-segment block convolutional neural network | |
| JP7334900B2 (en) | Discriminator, learning device, method, program, trained model and storage medium | |
| Kwon et al. | Multistage probabilistic approach for the localization of cephalometric landmarks | |
| CN113506274B (en) | Detection system for human cognitive condition based on visual saliency difference map | |
| CN112101438B (en) | Left-right eye classification method, device, server and storage medium | |
| CN109087357A (en) | Scan orientation method, apparatus, computer equipment and computer readable storage medium | |
| CN109191425A (en) | medical image analysis method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190620 Address after: 310000 Room 177, Building No. 16, Zhuantang Science and Technology Economic Zone, Xihu District, Hangzhou City, Zhejiang Province Applicant after: According to the Hangzhou Medical Technology Co., Ltd. Address before: 310000 Room 177, Building No. 16, Zhuantang Science and Technology Economic Zone, Xihu District, Hangzhou City, Zhejiang Province Applicant before: According to the Hangzhou Medical Technology Co., Ltd. Applicant before: Hangzhou Yi chart Network Technology Co., Ltd. Applicant before: Guangzhou Yi chart Medical Technology Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181009 |
|
| RJ01 | Rejection of invention patent application after publication |