CN108090903A

CN108090903A - Lung neoplasm detection model training method and device, pulmonary nodule detection method and device

Info

Publication number: CN108090903A
Application number: CN201711497927.0A
Authority: CN
Inventors: 塔巴克希尼玛; 申在律; 杰雅色兰劳拉; 严行健; 梁建明; 丁晓伟
Original assignee: Suzhou Voxel Mdt Infotech Ltd
Current assignee: Suzhou Voxel Mdt Infotech Ltd
Priority date: 2017-12-29
Filing date: 2017-12-29
Publication date: 2018-05-29

Abstract

This application involves a kind of Lung neoplasm detection model training method and device, pulmonary nodule detection method and devices.In the program, Lung neoplasm detection model includes at least the candidate generator based on 2D Faster R CNN networks and the FPR models based on 3D CNN networks, wherein, background, Lung neoplasm candidate region and false positive candidate region three-dimensional are carried out using candidate generator to the dense graph picture for stacking generation by the CT gray level images of continuous level to classify, then classified using FPR models to Lung neoplasm candidate region, obtain Lung neoplasm and false positive, since false positive is individually divided into one kind by candidate generator, reduce the quantity of false positive, so as to improve the sensitivity of detection.Again due to also returning out translation vector by FPR models, accordingly, position movement of the position of the Lung neoplasm of prediction to actual Lung neoplasm can be improved detection sensitivity during detection, further reduce the quantity of false positive.

Description

Lung neoplasm detection model training method and device, pulmonary nodule detection method and device

Technical field

This application involves technical field of medical image processing more particularly to a kind of Lung neoplasm detection model training method and dresses It puts, pulmonary nodule detection method and device.

Background technology

The main reason for lung cancer is cancer mortality, therefore early detection and treatment are most important.Judge that lung whether there is Lung neoplasm is a strong indicator for judging cancer.At present, can sentence by chest thin layer (thin-section, CT) image Disconnected to whether there is Lung neoplasm, this considerably increases the workloads of doctor.To mitigate the burden of doctor, realize to lung knot in CT images The automatic identification of section has become very crucial technology, in current Lung neoplasm detection technique, based on convolutional neural networks Lung neoplasm in (Convolutional Neural Network, CNN) identification CT images, but since the variation of Lung neoplasm is more Sample has all size, variously-shaped, and there are many objects easily obscured with Lung neoplasm in CT images, causes detection spirit Sensitivity it is not high and detection result in false positive it is higher.

The content of the invention

To overcome the problems, such as at least to a certain extent present in correlation technique, the application provides a kind of Lung neoplasm detection mould Type training method and device, pulmonary nodule detection method and device.

According to the embodiment of the present application in a first aspect, providing a kind of Lung neoplasm detection model training method, the Lung neoplasm Detection model includes at least the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks； The Lung neoplasm detection model training method, including：

CT gray level images are stacked, wherein, the M being continuously stacked layer CT gray level images is storied according to preset strategy heap Into one layer of dense graph picture；The value of M is positive integer；

By the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, candidate's life is input to It grows up to be a useful person, carries out convergence training, output candidate region and the judgement to candidate region, be respectively background area, Lung neoplasm candidate regions Domain and false positive candidate region；Wherein, it is described to mark position and the diameter for including Lung neoplasm；

The Lung neoplasm candidate region that the candidate generator is exported is compared with the mark, separates Lung neoplasm candidate region With false positive candidate region, input the FPR models, carry out convergence training, classification output Lung neoplasm and false positive and according to The vector regression that the place-centric of the Lung neoplasm of classification output is moved to the place-centric of actual Lung neoplasm goes out translation vector, so as to The prediction of translation vector is carried out during detection, according to the translation vector of prediction, by the place-centric of the Lung neoplasm of prediction to actual lung The place-centric movement of tubercle.

It is preferred that the mark by actual Lung neoplasm present in the dense graph picture, the dense graph picture, is input to The candidate generator carries out convergence training, output candidate region and the judgement to candidate region, is respectively background area, lung Nodule candidate region and false positive candidate region, including：

By the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, candidate's life is input to It grows up to be a useful person, carries out the convergence training of first stage, classification output background area and Lung neoplasm candidate region；

Collect the vacation sun in the Lung neoplasm candidate region for the convergence training output that the candidate generator carries out the first stage Property candidate region, and obtain the position of false positive candidate region；

By the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, false positive candidate region Position is input to the candidate generator, carries out the convergence training of second stage, output background area, Lung neoplasm candidate region With false positive candidate region.

It is preferred that the 2D Faster R-CNN networks include RPN sub-networks and 2D Fast R-CNN sub-networks；

The mark by actual Lung neoplasm present in the dense graph picture, the dense graph picture is input to the time Maker is selected, carries out convergence training, including：

The dense graph picture is input to the RPN sub-networks, carries out convergence training, exports candidate region and candidate region Probability score comprising Lung neoplasm；

Probability score is more than to the candidate region of predetermined probabilities scoring threshold value, is input to the 2D Fast R-CNN subnets Network carries out convergence training.

It is preferred that described stack one layer of dense graph picture of generation by the M being continuously stacked layer CT gray level images according to preset strategy, Including：

The average value of the gray value of the corresponding pixel points for the M layer CT gray level images being continuously stacked is calculated, by corresponding pixel points Gray value average value, the gray value of the corresponding pixel points as dense graph picture.

It is preferred that the candidate generator has R passages, G passages and channel B；

It is described that the dense graph picture is input to the candidate generator, including：

By 3 layers of continuous dense graph picture, the R passages, G passages and channel B of the candidate generator are separately input to.

It is preferred that it is described by 3 layers of continuous dense graph picture, it is separately input to R passages, the G of the candidate generator Before passage and channel B, this method is further included the compact image magnification preset multiple.

It is preferred that the Lung neoplasm candidate region by candidate generator output is compared with the mark, Lung neoplasm is separated Candidate region and false positive candidate region, input FPR models, including：

The Lung neoplasm candidate region of candidate generator output with mark is compared, separates Lung neoplasm candidate region and false positive Candidate region；

For Lung neoplasm candidate region：One candidate region is converted into a 3D cube, is input to the FPR moulds Type；Or merge continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, obtain one The candidate region of merging is input to the FPR models；

For false positive candidate region：One candidate region is converted into a 3D cube, is input to the FPR moulds Type；Or merge continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, obtain one The candidate region of merging is input to the FPR models.

According to the second aspect of the embodiment of the present application, a kind of pulmonary nodule detection method is provided, including：

Obtain the CT gray level images to be detected of continuous level；

The CT gray level images to be detected are input in Lung neoplasm detection model；The Lung neoplasm detection model is at least Including the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks；The Lung neoplasm inspection It is being trained by the training method as described in any of the above item to survey model；

The CT gray level images to be detected are detected using the candidate generator, generate candidate region and are obtained Judgement to candidate region is respectively background area, Lung neoplasm candidate region and false positive candidate region；

The Lung neoplasm candidate region is screened using the FPR models, during screening, each candidate region is carried out Following iteration：

It is proceeded as follows according to the place-centric of this candidate region：According to the position of this candidate region, to be detected A cube is obtained in CT gray level images, the FPR models is input to, obtains the translation of Lung neoplasm and false positive and prediction Vector；

According to the translation vector of prediction, the place-centric of this mobile candidate region；According to the new position of this candidate region Center repeats the operation, until the size of the translation vector of prediction is less than pre-set dimension threshold value or carries out the behaviour The number of work is more than preset times threshold value.

According to the third aspect of the embodiment of the present application, a kind of Lung neoplasm detection model training device, the Lung neoplasm are provided Detection model includes at least the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks； The training device, including：

Image stack module, for being stacked to CT gray level images, wherein, by the M being continuously stacked layer CT gray level images One layer of dense graph picture of generation is stacked according to preset strategy；The value of M is positive integer；

First training module, for by the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, The candidate generator is input to, convergence training, output candidate region and the judgement to candidate region is carried out, is respectively background area Domain, Lung neoplasm candidate region and false positive candidate region；Wherein, it is described to mark position and the diameter for including Lung neoplasm；

Second training module, Lung neoplasm candidate region and the mark for the candidate generator to be exported compare, Lung neoplasm candidate region and false positive candidate region are separated, inputs the FPR models, carries out convergence training, classification output lung knot Section and false positive and the vector moved according to the place-centric of the Lung neoplasm of classification output to the place-centric of actual Lung neoplasm Translation vector is returned out, the prediction of translation vector is carried out during to detect, according to the translation vector of prediction, by the Lung neoplasm of prediction Place-centric from place-centric to actual Lung neoplasm move.

It is preferred that first training module, is specifically used for：

It is preferred that described image stack module, is specifically used for：

It is described by the dense graph picture, when being input to the candidate generator, second training module is specifically used for：

It is preferred that it is described by 3 layers of continuous dense graph picture, it is separately input to R passages, the G of the candidate generator Before passage and channel B, second training module is additionally operable to the compact image magnification preset multiple.

It is preferred that the Lung neoplasm candidate region by candidate generator output is compared with the mark, Lung neoplasm is separated Candidate region and false positive candidate region, when inputting the FPR models, second training module is specifically used for：

According to the fourth aspect of the embodiment of the present application, a kind of Lung neoplasm detection device is provided, including：

Image collection module, for obtaining the CT gray level images to be detected of continuous level；

Image input module, for the CT gray level images to be detected to be input in Lung neoplasm detection model；It is described Lung neoplasm detection model is including at least the candidate generator based on 2D Faster R-CNN networks and based on 3D CNN networks FPR models；The Lung neoplasm detection model is being trained by the training method as described in any of the above item；

First detection module, for being detected using the candidate generator to the CT gray level images to be detected, It generates candidate region and obtains the judgement to candidate region, be respectively background area, Lung neoplasm candidate region and false positive candidate Region；

Second detection module, for being screened using the FPR models to the Lung neoplasm candidate region, during screening, Following iteration is carried out to each candidate region：

According to the 5th of the embodiment of the present application the aspect, a kind of non-transitorycomputer readable storage medium is provided, when described When instruction in storage medium is performed by the processor of terminal so that terminal is able to carry out a kind of Lung neoplasm detection model training side Method, the Lung neoplasm detection model is including at least the candidate generator based on 2D Faster R-CNN networks and based on 3D CNN The FPR models of network；The described method includes：

According to the 6th of the embodiment of the present application the aspect, a kind of non-transitorycomputer readable storage medium is provided, when described When instruction in storage medium is performed by the processor of terminal so that terminal is able to carry out a kind of pulmonary nodule detection method, described Method includes：

Obtain the CT gray level images to be detected of continuous level；

According to the 7th of the embodiment of the present application the aspect, a kind of Lung neoplasm detection model training device, the Lung neoplasm are provided Detection model includes at least the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks； The training device includes：Processor；For storing the memory of processor-executable instruction；Wherein, the processor by with It is set to：

According to the eighth aspect of the embodiment of the present application, a kind of Lung neoplasm detection device is provided, including：Processor；For depositing Store up the memory of processor-executable instruction；Wherein, the processor is configured as：

Obtain the CT gray level images to be detected of continuous level；

The technical solution that embodiments herein provides can include the following benefits：Lung neoplasm detection model at least wraps The candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks are included, wherein, utilize candidate Maker carries out the dense graph picture that generation is stacked by the CT gray level images of continuous level background, Lung neoplasm candidate region and vacation sun Property the classification of candidate region three-dimensional, then classified using FPR models to Lung neoplasm candidate region, obtain Lung neoplasm and vacation sun Property, since false positive is individually divided into one kind by candidate generator, reduce the quantity for the false positive being input in FPR models, from And reduce the quantity of the false positive in the Lung neoplasm that FPR models obtain, so as to improve the sensitivity of detection.Again due to also logical It crosses FPR models and returns out translation vector, it accordingly, can be by the position of the Lung neoplasm of prediction to the position of actual Lung neoplasm during detection It is mobile so that testing result is more matched with actual Lung neoplasm, improves detection sensitivity, further reduces the number of false positive Amount.Further, since candidate generator is based on 2D Faster R-CNN networks, and what is inputted is to concentrate multi-Slice CT gray-scale map The dense graph picture of the information of picture is not only beneficial to improve sensitivity, but also can ensure computational efficiency simultaneously.

It should be appreciated that above general description and following detailed description are only exemplary and explanatory, not The application can be limited.

Description of the drawings

Attached drawing herein is merged in specification and forms the part of this specification, shows the implementation for meeting the application Example, and for explaining the principle of the application together with specification.

Fig. 1 is the flow chart according to a kind of training method of Lung neoplasm model shown in an exemplary embodiment.

Fig. 2 is the flow chart according to a kind of pulmonary nodule detection method shown in an exemplary embodiment.

Fig. 3 is the block diagram according to a kind of training device of Lung neoplasm model shown in an exemplary embodiment.

Fig. 4 is the block diagram according to a kind of Lung neoplasm detection device shown in an exemplary embodiment.

Specific embodiment

Here exemplary embodiment will be illustrated in detail, example is illustrated in the accompanying drawings.Following description is related to During attached drawing, unless otherwise indicated, the same numbers in different attached drawings represent the same or similar element.Following exemplary embodiment Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended The example of the consistent apparatus and method of some aspects be described in detail in claims, the application.

According to the embodiment of the present application in a first aspect, providing a kind of Lung neoplasm detection model training method, Lung neoplasm detects Model includes at least the candidate generator based on 2D Faster R-CNN networks and the reduction false positive based on 3D CNN networks (False positive Reduction, FPR) model；As shown in Figure 1, Lung neoplasm detection model training method, includes at least Following steps：

Step 110 stacks CT gray level images, wherein, by the M being continuously stacked layer CT gray level images according to default plan Slightly stack one layer of dense graph picture of generation；The value of M is positive integer.

Step 120, the mark by actual Lung neoplasm present in dense graph picture, dense graph picture, are input to candidate generator, Convergence training, output candidate region and the judgement to candidate region are carried out, is respectively background area, Lung neoplasm candidate region and vacation Positive candidate region；Wherein, above-mentioned mark includes position and the diameter of Lung neoplasm.

Generally, the region extracted in model is identified with rectangular shaped rim.Wherein, the position of Lung neoplasm can be by identifying lung knot Save the coordinate representation of diagonal two pixels of the rectangular shaped rim of region.In implementation, actual Lung neoplasm region is with just Square mark, the also commonly referred to as equal length of each edge, diameter.Therefore, the mark of actual Lung neoplasm includes the position of Lung neoplasm It puts and diameter.

Wherein, background area, Lung neoplasm candidate region and false positive candidate region are the Lung neoplasm candidate regions according to prediction The rectangle in domain is classified with identifying the square registration of actual Lung neoplasm.

Step 130 compares the Lung neoplasm candidate region of candidate generator output with mark, separates Lung neoplasm candidate region With false positive candidate region, FPR models are inputted, carry out convergence training, classification exports Lung neoplasm and false positive and according to classification The vector regression that the place-centric of the Lung neoplasm of output is moved to the place-centric of actual Lung neoplasm goes out translation vector, to detect The prediction of Shi Jinhang translation vectors, according to the translation vector of prediction, by the place-centric of the Lung neoplasm of prediction to actual Lung neoplasm Place-centric movement.

Wherein, during comparison, if the rectangle of lung knot candidate region and the registration of the square of actual Lung neoplasm are higher than threshold Value, then be categorized as Lung neoplasm candidate region by the candidate region, less than threshold value, then the candidate region be categorized as false positive candidate Region.

Wherein, the place-centric of Lung neoplasm can be the center for the rectangular shaped rim for identifying Lung neoplasm region.

Wherein, return translation vector refer to the set of parameter for finding model, can Accurate Prediction go out classification output The vector that the place-centric movement of Lung neoplasm is moved to the place-centric of actual Lung neoplasm.

Since the appearance of false positive should more or less have similitude, inventor considers that a false sun can be built The classification of property, itself is gone to select by Lung neoplasm detection model, in this way, when detecting, just without doctor again in substantial amounts of image False positive is searched for, and has unified the criterion to false positive.

Again since most candidate generator tends to detect several pixels away from actual Lung neoplasm position, cause doctor Think visually without attracting discovery, particularly when only several pixels are wide in itself for Lung neoplasm, more difficult discovery.For This, inventor considers the place-centric of the Lung neoplasm of moving projection, is moved to the place-centric of actual Lung neoplasm.

Based on this, in the present embodiment, Lung neoplasm detection model includes at least the time based on 2D Faster R-CNN networks Maker and the FPR models based on 3D CNN networks are selected, wherein, using candidate generator to the CT gray level images by continuous level The dense graph picture for stacking generation carries out background, Lung neoplasm candidate region and the classification of false positive candidate region three-dimensional, then utilizes FPR Model classifies to Lung neoplasm candidate region, obtains Lung neoplasm and false positive, since candidate generator individually divides false positive For one kind, reduce the quantity for the false positive being input in FPR models, so as to reduce in the Lung neoplasm that FPR models obtain The quantity of false positive, so as to improve the sensitivity of detection.Again due to also returning out translation vector by FPR models, accordingly, inspection During survey can by position from the position of the Lung neoplasm of prediction to actual Lung neoplasm move so that testing result with actual Lung neoplasm more Add matching, improve detection sensitivity, further reduce the quantity of false positive.Further, since candidate generator is based on 2D Faster R-CNN networks, and input be the information for concentrating multi-Slice CT gray level image dense graph picture, not only be beneficial to improve Sensitivity, and can ensure computational efficiency simultaneously.

It is preferred that in above-mentioned steps 110, the M being continuously stacked layer CT gray level images are stacked into generation one according to preset strategy Layer dense graph picture, there are many specific implementations.One of which specific implementation can be：Calculate the M layers CT being continuously stacked The average value of the gray value of the corresponding pixel points of gray level image, by the average value of the gray value of corresponding pixel points, as dense graph The gray value of the corresponding pixel points of picture.In this way, the sensitivity of detection can be further improved.

Thoracic CT scan provides the 3D views of chest by stacking axis level.It is assumed that CT gray level images are in x-axis, y-axis The plane of composition, multi-Slice CT gray level image are stacked along z-axis.So, the pixel of (x0, y0) position in a gray level images and b ashes The pixel for spending (x0, y0) position in image is corresponding pixel points.

It is preferred that candidate generator has R passages, G passages and channel B；Correspondingly, in above-mentioned steps 120, by dense graph Picture is input to candidate generator, can be including specific implementation：

By 3 layers of continuous dense graph picture, the R passages, G passages and channel B of candidate generator are separately input to.

It is assumed that the value of M is 3.It is possible to be, by z-4 layers, z-3 layers, z-2 layers of the gray level image heap along z-axis It is folded to obtain one layer of dense graph picture, it is input to the R passages of candidate generator；Z-1 layers, z layers, z+1 layers of gray level image are stacked To one layer of dense graph picture, the G passages of candidate generator are input to；Z+2 layers, z+3 layers, z+4 layers of gray level image are stacked to obtain One layer of dense graph picture is input to the channel B of candidate generator.

In CT gray level images, most of Lung neoplasm is smaller, if generating network by the candidate for striding larger, is carried finally It is invisible in the characteristic pattern taken, therefore, it is not detected, in order to avoid this problem, it is preferred that by 3 layers of continuous dense graph picture, It is separately input to before the R passages, G passages and channel B of candidate generator, this method, further including can be by compact image magnification Preset multiple.So, it is ensured that Lung neoplasm is detected, and improves detection sensitivity.

In above-mentioned steps 120, by the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, candidate is input to Maker carries out convergence training, output candidate region and the judgement to candidate region, is respectively background area, Lung neoplasm candidate Region and false positive candidate region, there are many specific implementations.One of which concrete implementation mode can be：By dense graph The mark of actual Lung neoplasm present in picture, dense graph picture, is input to candidate generator, carries out the convergence training of first stage, Classification output background area and Lung neoplasm candidate region；Collect the lung that candidate generator carries out the convergence training output of first stage False positive candidate region in nodule candidate region, and obtain the position of false positive candidate region；By dense graph picture, dense graph picture Present in the mark of actual Lung neoplasm, the position of false positive candidate region, be input to candidate generator, carry out second stage Convergence training, output background area, Lung neoplasm candidate region and false positive candidate region.In this way, two are carried out to candidate generator The training in a stage in the training of first stage, first carries out background area and Lung neoplasm candidate region two to classification, Ran Houshou False positive in the Lung neoplasm candidate region that the training of collection first stage obtains gets the position of these false positives, as The input of the training of second stage accordingly, then carries out candidate generator the training of second stage, the false positive for output of classifying Candidate region and Lung neoplasm candidate region are more accurate, beneficial to the quantity for the false positive for reducing detection.

It is preferred that 2D Faster R-CNN networks include RPN sub-networks and 2D Fast R-CNN sub-networks；Accordingly , by the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, candidate generator is input to, carries out convergence training, Specific implementation can be：Dense graph picture is input to RPN sub-networks, carries out convergence training, exports candidate region and candidate Region includes the probability score of Lung neoplasm；Probability score is more than to the candidate region of predetermined probabilities scoring threshold value, is input to 2D Fast R-CNN sub-networks carry out convergence training.Wherein, candidate region is judged as Lung neoplasm by 2D Fast R-CNN sub-networks Candidate region, background area and false positive region, also further make Lung neoplasm candidate region the recurrence on border, make its border It is more accurate.

The candidate region of candidate generator output for 2D candidate regions, FPR models be based on 3D CNN networks, therefore, When the candidate region of candidate generator is input to FPR models, it is necessary to carry out 3D conversions, based on this, it is preferred that candidate is given birth to Grow up to be a useful person output Lung neoplasm candidate region with mark compare, separate Lung neoplasm candidate region with and false positive candidate region, input FPR models, specific implementation can be：The Lung neoplasm candidate region of candidate generator output with mark is compared, separates lung Nodule candidate region and false positive candidate region；For Lung neoplasm candidate region：One candidate region is converted into a 3D to stand Cube is input to FPR models；Or by continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value Domain merges, and obtains the candidate region of a merging, is input to FPR models；For false positive candidate region：By a candidate region A 3D cube is converted into, is input to FPR models；It or will be continuously distributed and the distance between adjacent less than pre-determined distance threshold Multiple candidate regions of value merge, and obtain the candidate region of a merging, are input to FPR models.

Wherein, it can extend on the border of candidate region along z-axis a candidate region to be converted into a 3D cube Direction extends a certain amount.Larger value of the candidate region in the length of x-axis or y-axis is can be set as along the elongation of z-axis.

In the present embodiment, it is contemplated that candidate generator may generate multiple candidate regions in a zonule, to improve Multiple neighbouring candidate regions are merged into a candidate region, are then input in FPR models by treatment effeciency.

Based on same inventive concept, according to the second aspect of the embodiment of the present application, a kind of pulmonary nodule detection method is provided, As shown in Fig. 2, including at least following steps：

Step 210, the CT gray level images to be detected for obtaining continuous level；

CT gray level images to be detected are input in Lung neoplasm detection model by step 220；Lung neoplasm detection model is at least Including the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks；Lung neoplasm detects Model is being trained by the training method as described in any of the above embodiment；

Step 230 is detected CT gray level images to be detected using candidate generator, generates candidate region and obtains Judgement to candidate region is respectively background area, Lung neoplasm candidate region and false positive candidate region；

Step 240 screens Lung neoplasm candidate region using FPR models, and during screening, each candidate region is carried out Following iteration：

It is proceeded as follows according to the place-centric of this candidate region：According to the position of this candidate region, to be detected A cube is obtained in CT gray level images, FPR models is input to, obtains the translation vector of Lung neoplasm and false positive and prediction Amount；

According to the translation vector of prediction, the place-centric of this mobile candidate region；According to the new position of this candidate region Center repeats to operate, until the size of the translation vector of prediction is less than pre-set dimension threshold value or the number operated More than preset times threshold value.

Cube model therein refers to 3 D stereo, can be x, y, z axis the equal 3 D stereo of length value or The 3 D stereo that the length value of x, y, z axis is not completely equivalent.

Wherein, according to the position of this candidate region, a cube is obtained from CT gray level images to be detected, it can be with It is directly to take the cube of a fixed size.

In the present embodiment, Lung neoplasm detection model includes at least the candidate generator based on 2D Faster R-CNN networks With the FPR models based on 3D CNN networks, wherein, using candidate generator to by continuous level CT gray level images stack generation Dense graph picture carry out background, Lung neoplasm candidate region and false positive candidate region three-dimensional classification, then using FPR models to lung Nodule candidate region and false positive candidate region are classified, and obtain Lung neoplasm and false positive, since candidate generator will be false positive Property is individually divided into one kind, reduces the quantity for the false positive being input in FPR models, so as to reduce the lung that FPR models obtain The quantity of false positive in tubercle, so as to improve the sensitivity of detection.Again due to also returning out translation vector by FPR models Amount accordingly, can move the position of the Lung neoplasm of prediction to the position of actual Lung neoplasm during detection so that testing result is with reality Border Lung neoplasm more matches, and improves detection sensitivity, further reduces the quantity of false positive.Further, since candidate generates Device be based on 2D Faster R-CNN networks, and input be the information for concentrating multi-Slice CT gray level image dense graph picture, Not only it is beneficial to improve sensitivity, but also can ensures computational efficiency simultaneously.

Below by taking specific application scenarios as an example, to a kind of Lung neoplasm detection model training method of some embodiments offer It is explained in more detail with pulmonary nodule detection method.

In the present embodiment, the distance between continuous level of CT scanner scanning is between 1mm to 5mm scopes, each In level, the sampled distance between continuous x and y measurements is 0.7mm.In order to ensure the uniformity of all 3D renderings, it is necessary to weight It is newly sampled, ensures that the spacing on three directions is identical.For example, it is 1mm that sampling interval, which is,.

The Lung neoplasm detection model of the present embodiment includes candidate generator and base based on 2D Faster R-CNN networks In the FPR models of 3D CNN networks.Wherein, 2D Faster R-CNN networks include RPN sub-networks and 2D Fast R-CNN Sub-network.Candidate generator has R passages, G passages and channel B.

Based on this, pre-training process is as follows：

Step 1: the average value of the gray value of the corresponding pixel points for the M layer CT gray level images being continuously stacked is calculated, by correspondence The average value of the gray value of pixel, the gray value of the corresponding pixel points as dense graph picture.

In the present embodiment, the value of M is 3, the best results reached.

Step 2: the training of first stage is carried out to candidate generator：

By 3 layers of compact image magnification preset multiple be input to candidate generator RPN sub-networks and will be in dense graph picture The mark of existing actual Lung neoplasm is input to the RPN sub-networks of candidate generator, carries out convergence training, export candidate region and Candidate region includes the probability score of Lung neoplasm.In the step, it is preferred that preset multiple is 5.

Probability score is more than to the candidate region of predetermined probabilities scoring threshold value, is input to the progress of 2DFastR-CNN sub-networks Convergence training.In the step, candidate region is judged as Lung neoplasm candidate region, background area by 2D Fast R-CNN sub-networks Domain, and the border of accurate Lung neoplasm candidate region.

Step 3: when collecting the training to the candidate generator progress first stage, the false positive in Lung neoplasm candidate region, And obtain the position of false positive candidate region.

Step 4: the training of second stage is carried out to candidate generator：

By 3 layers of compact image magnification preset multiple be input to candidate generator RPN sub-networks and will be in dense graph picture The position for the false positive candidate region that the mark of existing actual Lung neoplasm, step 3 are collected is input to the RPN of candidate generator Sub-network carries out convergence training, exports the probability score that candidate region and candidate region include Lung neoplasm.It is preferred that default times Number is 5.

By probability score be more than predetermined probabilities scoring threshold value candidate region, be input to 2D Fast R-CNN sub-networks into Row convergence training.In the step, candidate region is judged as Lung neoplasm candidate region, background area by 2D Fast R-CNN sub-networks Domain and false positive candidate region, and the border of accurate Lung neoplasm candidate region.

Step 5: FPR models are trained：

The Lung neoplasm candidate region that candidate generator in step 4 exports and mark are compared, separate Lung neoplasm candidate region With false positive candidate region；For Lung neoplasm candidate region：One candidate region is converted into a 3D cube, is input to FPR models；Or merge continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, it obtains The candidate region of one merging, is input to FPR models；For false positive candidate region：One candidate region is converted into one 3D cubes are input to FPR models；Or by continuously distributed and the distance between adjacent multiple times less than pre-determined distance threshold value Favored area merges, and obtains the candidate region of a merging, is input to FPR models.Convergence training is carried out to FPR models, exports lung The place-centric of tubercle and false positive and the Lung neoplasm exported according to classification returns translation vector into the position of actual Lung neoplasm Amount.

Wherein, FPR models have added losses layer, for the Lung neoplasm that reduces the translation vector of prediction to the greatest extent Yu will predict The place-centric of candidate region is moved to the gap of the true translation vector of the place-centric of actual Lung neoplasm.True translation vector It can be in training stage, the vector moved by calculating the place-centric of Lung neoplasm of detection to the place-centric of actual Lung neoplasm It obtains.

It is based on after more than training process is trained Lung neoplasm detection model, is detected process：

Step 1: obtain the CT gray level images to be detected of continuous level.

Step 2: CT gray level images to be detected are input in trained Lung neoplasm detection model.

Step 3: CT gray level images to be detected are detected using candidate generator, generate candidate region and are obtained Judgement to candidate region is respectively background area, Lung neoplasm candidate region and false positive candidate region.

Step 4: screened using FPR models to Lung neoplasm candidate region, during screening, each candidate region is carried out Following iteration：

Wherein, pre-set dimension threshold value can be 1 pixel.Preset times threshold value can be 5.

The detection of the present embodiment is sensitiveer, and the reliability higher of Lung neoplasm, the quantity of false positive are less in testing result.

Based on same design, according to the third aspect of the embodiment of the present application, a kind of Lung neoplasm detection model training is provided Device, Lung neoplasm detection model is including at least the candidate generator based on 2D Faster R-CNN networks and based on 3D CNN nets The FPR models of network；Training device, as shown in figure 3, including：

Image stack module 301, for being stacked to CT gray level images, wherein, by the M being continuously stacked layer CT gray-scale maps As stacking one layer of dense graph picture of generation according to preset strategy；The value of M is positive integer；

First training module 302, for by the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, input To candidate generator, convergence training, output candidate region and the judgement to candidate region are carried out, is respectively background area, lung knot Save candidate region and false positive candidate region；Wherein, mark includes position and the diameter of Lung neoplasm；

Second training module 303, Lung neoplasm candidate region and mark for candidate generator to be exported compare, and separate lung Nodule candidate region and false positive candidate region, input FPR models, carry out convergence training, classification output Lung neoplasm and false positive, And translation is gone out according to the vector regression that the place-centric of the Lung neoplasm of classification output is moved to the place-centric of actual Lung neoplasm Vector carries out the prediction of translation vector during to detect, according to the translation vector of prediction, by the place-centric of the Lung neoplasm of prediction It is moved to the place-centric of actual Lung neoplasm.

It is preferred that the first training module, is specifically used for：

By the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, candidate generator is input to, carries out first The convergence training in stage, classification output background area and Lung neoplasm candidate region；

The false positive collected in the Lung neoplasm candidate region for the convergence training output that candidate generator carries out the first stage is waited Favored area, and obtain the position of false positive candidate region；

By the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, the position of false positive candidate region, input To candidate generator, the convergence training of second stage, output background area, Lung neoplasm candidate region and false positive candidate regions are carried out Domain.

It is preferred that 2D Faster R-CNN networks include RPN sub-networks and 2D Fast R-CNN sub-networks；

By the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, candidate generator is input to, is restrained Training, including：

Dense graph picture is input to RPN sub-networks, carries out convergence training, candidate region and candidate region is exported and includes lung knot The probability score of section；

By probability score be more than predetermined probabilities scoring threshold value candidate region, be input to 2D Fast R-CNN sub-networks into Row convergence training.

It is preferred that image stack module, is specifically used for：

It is preferred that candidate generator has R passages, G passages and channel B；

By dense graph picture, when being input to candidate generator, the second training module is specifically used for：

It is preferred that by 3 layers of continuous dense graph picture, be separately input to candidate generator R passages, G passages and channel B it Before, the second training module is additionally operable to compact image magnification preset multiple.

It is preferred that the Lung neoplasm candidate region of candidate generator output is compared with mark, Lung neoplasm candidate region is separated With false positive candidate region, when inputting FPR models, the second training module is specifically used for：

For Lung neoplasm candidate region：One candidate region is converted into a 3D cube, is input to FPR models；Or Person merges continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, obtains merging Candidate region is input to FPR models；

For false positive candidate region：One candidate region is converted into a 3D cube, is input to FPR models；Or Person merges continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, obtains merging Candidate region is input to FPR models.

According to the fourth aspect of the embodiment of the present application, a kind of Lung neoplasm detection device is provided, as shown in figure 4, including：

Image collection module 401, for obtaining the CT gray level images to be detected of continuous level；

Image input module 402, for CT gray level images to be detected to be input in Lung neoplasm detection model；Lung neoplasm Detection model includes at least the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks； Lung neoplasm detection model is being trained by the training method as described in any of the above embodiment；

First detection module 403 for being detected using candidate generator to CT gray level images to be detected, generates time Favored area simultaneously obtains the judgement to candidate region, is respectively background area, Lung neoplasm candidate region and false positive candidate region；

Second detection module 404, for being screened using FPR models to Lung neoplasm candidate region, during screening, to each Candidate region carries out following iteration：

According to the 5th of the embodiment of the present application the aspect, a kind of non-transitorycomputer readable storage medium is provided, works as storage When instruction in medium is performed by the processor of terminal so that terminal is able to carry out a kind of Lung neoplasm detection model training method, Lung neoplasm detection model is including at least the candidate generator based on 2D Faster R-CNN networks and based on 3D CNN networks FPR models；Method includes：

By the mark of actual Lung neoplasm present in dense graph picture, dense graph picture, candidate generator is input to, is restrained Training, output candidate region and the judgement to candidate region, are respectively background area, Lung neoplasm candidate region and false positive candidate Region；Wherein, mark includes position and the diameter of Lung neoplasm；

The Lung neoplasm candidate region of candidate generator output with mark is compared, separates Lung neoplasm candidate region and false positive Candidate region inputs FPR models, carries out convergence training, classification output Lung neoplasm and false positive and the lung according to classification output The vector regression that the place-centric of tubercle is moved to the place-centric of actual Lung neoplasm goes out translation vector, and to detect when is put down The prediction of vector is moved, according to the translation vector of prediction, by the place-centric of the Lung neoplasm of prediction into the position of actual Lung neoplasm The heart moves.

According to the 6th of the embodiment of the present application the aspect, a kind of non-transitorycomputer readable storage medium is provided, works as storage When instruction in medium is performed by the processor of terminal so that terminal is able to carry out a kind of pulmonary nodule detection method, and method includes：

Obtain the CT gray level images to be detected of continuous level；

CT gray level images to be detected are input in Lung neoplasm detection model；Lung neoplasm detection model is included at least and is based on The candidate generator of 2D Faster R-CNN networks and the FPR models based on 3D CNN networks；Lung neoplasm detection model be by Training method as described in any of the above embodiment is trained；

CT gray level images to be detected are detected using candidate generator, generate candidate region and are obtained to candidate regions The judgement in domain is respectively background area, Lung neoplasm candidate region and false positive candidate region；

Lung neoplasm candidate region is screened using FPR models, during screening, is changed as follows to each candidate region Generation：

According to the 7th of the embodiment of the present application the aspect, a kind of Lung neoplasm detection model training device, Lung neoplasm detection are provided Model includes at least the candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks；Training Device includes：Processor；For storing the memory of processor-executable instruction；Wherein, processor is configured as：

According to the eighth aspect of the embodiment of the present application, a kind of Lung neoplasm detection device is provided, including：Processor；For depositing Store up the memory of processor-executable instruction；Wherein, processor is configured as：

Obtain the CT gray level images to be detected of continuous level；

CT gray level images to be detected are input in Lung neoplasm detection model；Lung neoplasm detection model is included at least and is based on The candidate generator of 2D Faster R-CNN networks and the FPR models based on 3D CNN networks；Lung neoplasm detection model be by Training method such as any of the above embodiment is trained；

On the device in above-described embodiment, wherein modules perform the concrete mode of operation in related this method Embodiment in be described in detail, explanation will be not set forth in detail herein.

It is understood that same or similar part can mutually refer in the various embodiments described above, in some embodiments Unspecified content may refer to the same or similar content in other embodiment.

It should be noted that in the description of the present application, term " first ", " second " etc. are only used for description purpose, without It is understood that indicate or imply relative importance.In addition, in the description of the present application, unless otherwise indicated, the meaning of " multiple " Refer at least two.

Any process described otherwise above or method description are construed as in flow chart or herein, represent to include Module, segment or the portion of the code of the executable instruction of one or more the step of being used to implement specific logical function or process Point, and the scope of the preferred embodiment of the application includes other realization, wherein can not press shown or discuss suitable Sequence, including according to involved function by it is basic simultaneously in the way of or in the opposite order, carry out perform function, this should be by the application Embodiment person of ordinary skill in the field understood.

It should be appreciated that each several part of the application can be realized with hardware, software, firmware or combination thereof.Above-mentioned In embodiment, software that multiple steps or method can in memory and by suitable instruction execution system be performed with storage Or firmware is realized.If for example, with hardware come realize in another embodiment, can be under well known in the art Any one of row technology or their combination are realized：With for the logic gates to data-signal realization logic function Discrete logic, have suitable combinational logic gate circuit application-specific integrated circuit, programmable gate array (PGA), scene Programmable gate array (FPGA) etc..

Those skilled in the art are appreciated that realize all or part of step that above-described embodiment method carries Suddenly it is that relevant hardware can be instructed to complete by program, the program can be stored in a kind of computer-readable storage medium In matter, the program upon execution, one or a combination set of the step of including embodiment of the method.

In addition, each functional unit in each embodiment of the application can be integrated in a processing module, it can also That unit is individually physically present, can also two or more units be integrated in a module.Above-mentioned integrated mould The form that hardware had both may be employed in block is realized, can also be realized in the form of software function module.The integrated module is such as Fruit is realized in the form of software function module and is independent production marketing or in use, can also be stored in a computer In read/write memory medium.

Storage medium mentioned above can be read-only memory, disk or CD etc..

In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description Point is contained at least one embodiment or example of the application.In the present specification, schematic expression of the above terms is not Centainly refer to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be any One or more embodiments or example in combine in an appropriate manner.

Although embodiments herein has been shown and described above, it is to be understood that above-described embodiment is example Property, it is impossible to the limitation to the application is interpreted as, those of ordinary skill in the art within the scope of application can be to above-mentioned Embodiment is changed, changes, replacing and modification.

Claims

1. a kind of Lung neoplasm detection model training method, which is characterized in that the Lung neoplasm detection model is included at least based on 2D The candidate generator of Faster R-CNN networks and the FPR models based on 3D CNN networks；The Lung neoplasm detection model training Method, including：

CT gray level images are stacked, wherein, the M being continuously stacked layer CT gray level images are stacked into generation one according to preset strategy Layer dense graph picture；The value of M is positive integer；

By the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, the candidate generator is input to, Convergence training, output candidate region and the judgement to candidate region are carried out, is respectively background area, Lung neoplasm candidate region and vacation Positive candidate region；Wherein, it is described to mark position and the diameter for including Lung neoplasm；

The Lung neoplasm candidate region that the candidate generator is exported is compared with the mark, separates Lung neoplasm candidate region and vacation Positive candidate region inputs the FPR models, carries out convergence training, and classification exports Lung neoplasm and false positive and according to classification The vector regression that the place-centric of the Lung neoplasm of output is moved to the place-centric of actual Lung neoplasm goes out translation vector, to detect The prediction of Shi Jinhang translation vectors, according to the translation vector of prediction, by the place-centric of the Lung neoplasm of prediction to actual Lung neoplasm Place-centric movement.

2. according to the method described in claim 1, it is characterized in that, described will deposit in the dense graph picture, the dense graph picture Actual Lung neoplasm mark, be input to the candidate generator, carry out convergence training, output candidate region with to candidate regions The judgement in domain, respectively background area, Lung neoplasm candidate region and false positive candidate region, including：

By the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, the candidate generator is input to, Carry out the convergence training of first stage, classification output background area and Lung neoplasm candidate region；

The false positive collected in the Lung neoplasm candidate region for the convergence training output that the candidate generator carries out the first stage is waited Favored area, and obtain the position of false positive candidate region；

By the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, the position of false positive candidate region, The candidate generator is input to, carries out the convergence training of second stage, output background area, Lung neoplasm candidate region and vacation sun Property candidate region.

3. according to the method described in claim 1, it is characterized in that, the 2D Faster R-CNN networks include RPN sub-networks With 2D Fast R-CNN sub-networks；

The mark by actual Lung neoplasm present in the dense graph picture, the dense graph picture is input to candidate's life It grows up to be a useful person, carries out convergence training, including：

The dense graph picture is input to the RPN sub-networks, carries out convergence training, candidate region is exported and candidate region includes The probability score of Lung neoplasm；

By probability score be more than predetermined probabilities scoring threshold value candidate region, be input to the 2D Fast R-CNN sub-networks into Row convergence training.

4. according to the method described in claim 1, it is characterized in that, it is described by the M being continuously stacked layer CT gray level images according to pre- If strategy stacks one layer of dense graph picture of generation, including：

The average value of the gray value of the corresponding pixel points for the M layer CT gray level images being continuously stacked is calculated, by the ash of corresponding pixel points The average value of angle value, the gray value of the corresponding pixel points as dense graph picture.

5. according to the method described in claim 4, it is characterized in that, there is the candidate generator R passages, G passages and B to lead to Road；

6. according to the method described in claim 5, it is characterized in that, described by 3 layers of continuous dense graph picture, input respectively To before the R passages, G passages and channel B of the candidate generator, this method further includes and presets the compact image magnification Multiple.

7. the according to the method described in claim 1, it is characterized in that, Lung neoplasm candidate region that candidate generator is exported It is compared with the mark, separates Lung neoplasm candidate region and false positive candidate region, input FPR models, including：

For Lung neoplasm candidate region：One candidate region is converted into a 3D cube, is input to the FPR models；Or Person merges continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, obtains merging Candidate region is input to the FPR models；

For false positive candidate region：One candidate region is converted into a 3D cube, is input to the FPR models；Or Person merges continuously distributed and the distance between adjacent multiple candidate regions less than pre-determined distance threshold value, obtains merging Candidate region is input to the FPR models.

8. a kind of pulmonary nodule detection method, which is characterized in that including：

Obtain the CT gray level images to be detected of continuous level；

The CT gray level images to be detected are input in Lung neoplasm detection model；The Lung neoplasm detection model includes at least Candidate generator based on 2D Faster R-CNN networks and the FPR models based on 3D CNN networks；The Lung neoplasm detects mould Type trains to obtain by such as claim 1~7 any one of them training method；

The CT gray level images to be detected are detected using the candidate generator, generate candidate region and are obtained to waiting The judgement of favored area is respectively background area, Lung neoplasm candidate region and false positive candidate region；

The Lung neoplasm candidate region is screened using the FPR models, during screening, each candidate region is carried out as follows Iteration：

It is proceeded as follows according to the place-centric of this candidate region：According to the position of this candidate region, from CT ashes to be detected It spends and a cube is obtained in image, be input to the FPR models, obtain the translation vector of Lung neoplasm and false positive and prediction Amount；

According to the translation vector of prediction, the place-centric of this mobile candidate region；According to the new place-centric of this candidate region, Repeat the operation, until the size of the translation vector of prediction is less than pre-set dimension threshold value or carries out time of the operation Number is more than preset times threshold value.

9. a kind of Lung neoplasm detection model training device, which is characterized in that the Lung neoplasm detection model is included at least based on 2D The candidate generator of Faster R-CNN networks and the FPR models based on 3D CNN networks；The training device, including：

Image stack module, for being stacked to CT gray level images, wherein, by the M being continuously stacked layer CT gray level images according to Preset strategy stacks one layer of dense graph picture of generation；The value of M is positive integer；

First training module, for by the mark of actual Lung neoplasm present in the dense graph picture, the dense graph picture, input To the candidate generator, carry out convergence training, output candidate region and the judgement to candidate region, be respectively background area, Lung neoplasm candidate region and false positive candidate region；Wherein, it is described to mark position and the diameter for including Lung neoplasm；

Second training module, Lung neoplasm candidate region and the mark for the candidate generator to be exported compare, separate Lung neoplasm candidate region and false positive candidate region, input the FPR models, carry out convergence training, classification output Lung neoplasm and False positive and the vector regression moved according to the place-centric of the Lung neoplasm of classification output to the place-centric of actual Lung neoplasm Go out translation vector, the prediction of translation vector is carried out during to detect, according to the translation vector of prediction, by the position of the Lung neoplasm of prediction Center is put to move to the place-centric of actual Lung neoplasm.

10. a kind of Lung neoplasm detection device, which is characterized in that including：

Image input module, for the CT gray level images to be detected to be input in Lung neoplasm detection model；The lung knot It saves detection model and includes at least the candidate generator based on 2D Faster R-CNN networks and the FPR moulds based on 3D CNN networks Type；The Lung neoplasm detection model trains to obtain by such as claim 1~7 any one of them training method；

First detection module for being detected using the candidate generator to the CT gray level images to be detected, is generated Candidate region simultaneously obtains the judgement to candidate region, is respectively background area, Lung neoplasm candidate region and false positive candidate region；

Second detection module, for being screened using the FPR models to the Lung neoplasm candidate region, during screening, to every A candidate region carries out following iteration：