CN112085028B - Tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision - Google Patents
Tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision Download PDFInfo
- Publication number
- CN112085028B CN112085028B CN202010894993.7A CN202010894993A CN112085028B CN 112085028 B CN112085028 B CN 112085028B CN 202010894993 A CN202010894993 A CN 202010894993A CN 112085028 B CN112085028 B CN 112085028B
- Authority
- CN
- China
- Prior art keywords
- feature map
- tooth
- feature
- disturbance
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000000605 extraction Methods 0.000 claims abstract description 58
- 238000012549 training Methods 0.000 claims abstract description 13
- 230000004927 fusion Effects 0.000 claims description 17
- 238000010586 diagram Methods 0.000 claims description 14
- 238000005070 sampling Methods 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 10
- 238000011176 pooling Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 239000011521 glass Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 210000000988 bone and bone Anatomy 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 208000025157 Oral disease Diseases 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 208000030194 mouth disease Diseases 0.000 description 2
- 230000003239 periodontal effect Effects 0.000 description 2
- 208000018035 Dental disease Diseases 0.000 description 1
- 208000014151 Stomatognathic disease Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 210000000214 mouth Anatomy 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30036—Dental; Teeth
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a feature map disturbance and boundary supervision-based tooth full-view image semantic segmentation method, which comprises the steps of after a tooth full-view image is obtained, sharpening the tooth full-view image to obtain a tooth full-view image with a clearer tooth boundary, and then carrying out feature extraction on a disturbance feature map extraction network after training of the tooth full-view image to obtain a deep disturbance feature map; and finally, respectively inputting the deep disturbance feature map into a mask network and a boundary network which are completed by training, and obtaining a tooth region segmentation result and a tooth contour segmentation result. The invention greatly enhances the generalization capability of the network, and enables the trained model to still obtain a reasonable segmentation result by utilizing part of common features in special situations when the model encounters the special situations.
Description
Technical Field
The invention relates to the field of medical image processing, in particular to a tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision.
Background
The shortage of oral medical resources in China is mainly manifested by serious shortage of oral doctors, unbalanced regional development and insufficient development kinetic energy of domestic oral medical appliances and equipment. The national oral industry trend report in 2019 shows that the World Health Organization (WHO) is responsible for dentists: the recommended value for population ratio is 1:5000, which ratio rises to 1:2000 for developed countries. The population proportion of the dentists in China is less than 1:8000 and is far lower than the level and WHO recommended values in other countries. With rapid oral medical development in the eastern developed areas of the north america, the physician's possession has increased significantly, like the ratio of dentists to residents in the beijing urban area is about 1:2000, and the developed countries are almost as much, but only 1:8000 to suburban areas, and 1:20000 or 1:30000 to the west. This imbalance in regional development is a significant problem facing our country. In recent years, although the number of oral medical practitioners per ten thousands of people is slightly increased, the medical treatment requirements of the people on oral medical services are still far from being met. Except for the difference in the number of stomatologists, the overall constitution of our stomatologists is lower than the academic level in quality, and it is counted that the number of the oral doctors in China is about 45% of the number of the academic persons and the academic persons above in 2015, and most of the oral doctors are concentrated in public medical institutions or high-end medical institutions in large and medium cities, and the number of people who do not receive regular oral education or only receive primary oral medical education (Chinese proprietary level) for the oral and related medical industries is quite a part.
In addition, even in public oral medical institutions with high public confidence, because the patient volume far exceeds the normal load, doctors often only can diagnose and treat the complaint teeth of the patient, and the problem of non-complaint oral cavity of the patient is often ignored and treatment is delayed or omitted. On the other hand, due to the difference of doctor level, some oral diseases are easy to be missed or misdiagnosed. Therefore, if the panoramic film can be interpreted in advance by means of the artificial intelligence (Artificial Intelligence, AI) technology and a preliminary diagnosis report can be automatically provided, the efficiency and the accuracy of oral disease diagnosis can be improved, and misdiagnosis caused by missing diagnosis can be reduced. The segmentation of the teeth in the full view is the basis of all dental disease detection.
Patent title, panoramic film permanent tooth recognition method and device based on deep learning, application number of CN109949319A, application date of 2019-3-12; the patent describes a panoramic film permanent tooth recognition method and device based on deep learning, wherein an alveolar bone line segmentation model is utilized to obtain an alveolar bone line segmentation result, periodontal area image blocks are cut out from an original full-view film according to the alveolar bone line segmentation result, and finally the periodontal area image blocks are input into a permanent tooth segmentation model based on the deep learning to obtain a permanent tooth segmentation result and mark tooth position numbers.
Patent title, depth profile perception based tooth segmentation method, device and computer equipment, application number is CN110473243A, and application date is 2019-8-9; the patent describes a tooth segmentation method, a device and computer equipment based on depth contour perception, wherein an original mask is subjected to morphological treatment to extract a contour mask and is thickened, the thickened contour mask is used as supervision information, a preprocessed original tooth image is subjected to a full convolution network, a loss function is minimized, and the full convolution network is trained to obtain a contour prediction probability map. And then fusing the preprocessed tooth image and the contour prediction probability map, and obtaining a tooth segmentation result map after the fusion by using a U-shaped depth contour perception network taking an original mask as supervision information.
In the prior art, in the aspects of enhancing the generalization capability of a model and utilizing boundary information, a targeted strategy cannot be used for enhancing the generalization capability of the model, and the boundary information is not paid attention to enough, so that the boundary information and mask information are not considered in the same position, and the extracted features have insufficient universality and poor segmentation result.
Disclosure of Invention
The purpose of the application is to provide a tooth full-view film semantic segmentation method based on feature map disturbance and boundary supervision, which solves the problem that the prior art cannot utilize part of common features in special cases to conduct the tooth full-view film semantic segmentation method when the special cases are met.
In order to achieve the above purpose, the technical scheme of the application is as follows:
a tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision comprises the following steps:
obtaining a tooth full view picture, and sharpening the tooth full view picture to obtain a tooth full view picture I with clearer tooth boundaries;
inputting the tooth full-scene picture I into a disturbance feature map extraction network after training to obtain a deep disturbance feature map F deep ；
Map F of deep disturbance deep And respectively inputting the tooth region segmentation result and the tooth contour segmentation result into a mask network and a boundary network after training is completed.
Further, the obtaining the tooth full view picture, the sharpening operation is performed on the tooth full view picture, and the tooth full view picture I with clearer tooth boundary is obtained, including:
input original dental full view film I original And (3) sharpening filtering operation is carried out on all the dental full-view sheets by adopting a filter, and the filter kernel is 3*3 to obtain the sharpened dental full-view sheet I.
Further, the tooth full-scene image I is input into a disturbance feature image extraction network after training to perform feature extraction, so as to obtain a deep disturbance feature image F deep Comprising:
step 2.1, inputting the tooth full-scene film I into a simple feature extraction module with the convolution kernel size of 3*3 to obtain an output feature map F 1 Its dimension is C 1 ×H 1 ×W 1 ；
Step 2.2, feature map F 1 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 2 Its dimension is C 2 ×H 2 ×W 2 ；
Step 2.3, feature map F 2 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 3 Its dimension is C 3 ×H 3 ×W 3 ；
Step 2.4, feature map F 3 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 4 Its dimension is C 4 ×H 4 ×W 4 ；
Step 2.5, feature map F 4 The pooled disturbance characteristic is input into a disturbance characteristic extraction module with the convolution kernel size of 3*3 to obtain a deep disturbance characteristic diagram F deep Its dimension is C 5 ×H 5 ×W 5 。
Further, the depth disturbance characteristic map F deep Respectively inputting the tooth region segmentation result and the tooth contour segmentation result into a mask network and a boundary network which are trained, and obtaining the tooth region segmentation result and the tooth contour segmentation result, wherein the tooth region segmentation result and the tooth contour segmentation result comprise,
step 3.1, in the mask network, depth disturbance feature map F deep Up-sampling and F 4 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 5 +C 4 Inputting the simple feature extraction module to obtain a feature map UP 4 Its dimension is C 4 ×H 4 ×W 4 ；
Step 3.2, feature map UP 4 Up-sampling and F 3 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 4 +C 3 Inputting the simple feature extraction module to obtain a feature map UP 3 Its dimension is C 3 ×H 3 ×W 3 ；
Step 3.3, feature map UP 3 Up-sampling and F 2 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 3 +C 2 Inputting the simple feature extraction module to obtain a feature map UP 2 Its dimension is C 2 ×H 2 ×W 2 ；
Step 3.4, feature map UP 2 Up-sampling and F 1 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 2 +C 1 Inputting the simple feature extraction module to obtain a feature map UP 1 Its dimension is C 1 ×H 1 ×W 1 ；
Step 3.5, feature map UP 1 Inputting 1*1 convolution block to obtain a characteristic map UP 0 Dimension 32 XH 1 ×W 1 Wherein 32 represents 32 different teeth, UP 0 Each channel of (2) is activated by the following formula to obtain UP 0 The probability that each pixel point belongs to the tooth area is multiplied by 255 to obtain the segmentation of the final 32 teethResults;
and step 3.6, the operations of the steps 3.1 to 3.5 are adopted in the boundary network, and finally, the tooth profile segmentation result is output.
Further, the simple feature extraction module comprises two groups of convolution layers with the convolution kernel size of 3*3, a batch normalization layer and an activation layer which are connected in series.
Further, the disturbance feature extraction module comprises two groups of convolution layers with the convolution kernel size of 3*3 connected in series, a feature disturbance operation layer, a batch normalization layer and an activation layer.
Further, the channel fusion module is configured to combine the up-sampled lower-layer feature map and the current-layer feature map according to channels, and output a feature map with unchanged size and changed channel number into the sum of the lower-layer feature map and the current-layer feature map.
Further, the characteristic disturbance operation realizes disturbance on the characteristic diagram by using the following formula;
wherein x is i To input a feature map, f (x i ) Andrespectively representing the characteristic diagram before disturbance and after disturbance, m i Consisting of 0 and 1, following a Bernoulli distribution, ε i For controlling the disturbance amplitude, the parameter value is automatically optimized in training, and the matrix is expressed by corresponding multiplication of each point.
According to the tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision, on one hand, the feature map is disturbed by the disturbance feature extraction module in the feature extraction process, so that partial feature information can be lost in the disturbed feature map, and the neural network obtains segmentation results by learning how to use the partial feature-lost feature map, so that the generalization capability of the network is greatly enhanced, and the reasonable segmentation results can be obtained by using the partial features which are common in the special case when the special case is encountered. On the other hand, due to the introduction of the boundary network, the characteristics of the boundary of the segmented region can be directly learned through the boundary network, the boundary of the segmented region can be more easily found, and the segmentation effect on the conditions of difference in the classes and similarity between the classes is improved.
Drawings
FIG. 1 is a flow chart of a method for semantic segmentation of a dental full-scene based on feature map perturbation and boundary supervision;
FIG. 2 is a schematic diagram of a network structure according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a simple feature extraction module structure of the present application;
FIG. 4 is a schematic diagram of a disturbance feature extraction module structure of the present application;
fig. 5 is a schematic structural diagram of a channel fusion module of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, a method for semantic segmentation of a dental full-view film based on feature map perturbation and boundary supervision is provided, comprising:
step S1, obtaining a tooth full-view film, and sharpening the tooth full-view film to obtain a tooth full-view film I with clearer tooth boundaries.
This application carries out necessary preliminary treatment to the whole sight glass of tooth that acquires, acquire the whole sight glass of tooth, pass through sharpening operation with the whole sight glass of tooth, obtain the whole sight glass I of tooth that the tooth boundary is clearer, include:
input original dental full view film I original Sharpening filtering operation is carried out on all the scenic spots of each tooth by adopting a filterThe filter kernel is 3*3, which yields a sharpened dental full view I.
It should be noted that, the present application may also directly use the original dental panorama to perform subsequent processing without performing sharpening operation. The filter kernel may also be set to 5*5, or 7*7, as desired.
S2, inputting the tooth full-scene image I into a disturbance feature image extraction network after training to perform feature extraction to obtain a deep disturbance feature image F deep 。
The application describes that the feature extraction is carried out on the tooth full-scene film I to obtain a deep disturbance feature map F deep Comprising:
step 2.1, inputting the tooth full-scene film I into a simple feature extraction module with the convolution kernel size of 3*3 to obtain an output feature map F 1 Its dimension is C 1 ×H 1 ×W 1 ；
Step 2.2, feature map F 1 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 2 Its dimension is C 2 ×H 2 ×W 2 ；
Step 2.3, feature map F 2 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 3 Its dimension is C 3 ×H 3 ×W 3 ；
Step 2.4, feature map F 3 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 4 Its dimension is C 4 ×H 4 ×W 4 ；
Step 2.5, feature map F 4 The pooled disturbance characteristic is input into a disturbance characteristic extraction module with the convolution kernel size of 3*3 to obtain a deep disturbance characteristic diagram F deep Its dimension is C 5 ×H 5 ×W 5 。
As shown in fig. 2, the disturbance map extraction network of the present application includes a simple feature extraction module (CBR) and a four-layer disturbance feature extraction module (CDBR). In other embodiments, the disturbance feature map extraction network may also use other scrambling structures, such as a structure of three layers of simple feature extraction modules and two layers of disturbance feature extraction modules in sequence.
Compared with the undisturbed feature map, the disturbed deep feature map has more common features, and is beneficial to improving the generalization capability of the network.
The simple feature extraction module, as shown in fig. 3, includes two sets of serially connected convolution layers (conv 3×3) with convolution kernel size 3*3, a batch normalization layer (BN), and an activation layer (ReLU).
Firstly, calculating an input feature map through a convolution layer, then carrying out batch normalization and ReLU activation layer processing, then through a second group of convolution layers, batch normalization and ReLU activation layers, and finally outputting the processed feature map.
The disturbance feature extraction module, as shown in fig. 4, includes two sets of serially connected convolution layers (conv 3×3) with convolution kernel size 3*3, a feature disturbance, a batch normalization layer (BN), and an activation layer (ReLU).
Firstly, calculating an input feature map through a convolution layer, then, disturbing the feature map through feature disturbance operation, then, carrying out batch normalization and ReLU activation layer processing, then, through a second group of convolution layers, feature disturbance, batch normalization and ReLU activation layers, and finally, outputting the processed feature map.
The characteristic disturbance operation realizes the disturbance of the characteristic diagram by using the following formula;
wherein x is i To input a feature map, f (x i ) Andrespectively representing the characteristic diagram before disturbance and after disturbance, m i Consisting of 0 and 1, following a Bernoulli distribution, ε i For controlling the disturbance amplitude, the parameter value is automatically optimized in training, and the matrix is expressed by corresponding multiplication of each point. i represents the i-th layer of the network, x i A feature map representing the i-th layer input.
It should be noted that, the convolution kernel sizes of the simple feature extraction module and the disturbance feature extraction module in the present application may be set to be 5*5 or 7*7 as required.
S3, deep disturbance characteristic diagram F deep And respectively inputting the tooth region segmentation result and the tooth contour segmentation result into a mask network and a boundary network after training is completed.
The deep disturbance characteristic map F deep Respectively inputting the tooth region segmentation result and the tooth contour segmentation result into a mask network and a boundary network which are trained, and obtaining the tooth region segmentation result and the tooth contour segmentation result, wherein the tooth region segmentation result and the tooth contour segmentation result comprise,
step 3.1, in the mask network, depth disturbance feature map F deep Up-sampling and F 4 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 5 +C 4 Inputting the simple feature extraction module to obtain a feature map UP 4 Its dimension is C 4 ×H 4 ×W 4 ；
Step 3.2, feature map UP 4 Up-sampling and F 3 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 4 +C 3 Inputting the simple feature extraction module to obtain a feature map UP 3 Its dimension is C 3 ×H 3 ×W 3 ；
Step 3.3, feature map UP 3 Up-sampling and F 2 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 3 +C 2 Inputting the simple feature extraction module to obtain a feature map UP 2 Its dimension is C 2 ×H 2 ×W 2 ；
Step 3.4, feature map UP 2 Up-sampling and F 1 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 2 +C 1 Inputting the simple feature extraction module to obtain a feature map UP 1 Its dimension is C 1 ×H 1 ×W 1 ；
Step (a)3.5 feature map UP 1 Inputting 1*1 convolution block to obtain a characteristic map UP 0 Dimension 32 XH 1 ×W 1 Wherein 32 represents 32 different teeth, UP 0 Each channel of (2) is activated by the following formula to obtain UP 0 Multiplying the probability that each pixel point belongs to the tooth area by 255 to obtain a segmentation result of the final 32 teeth;
and step 3.6, the operations of the steps 3.1 to 3.5 are adopted in the boundary network, and finally, the tooth profile segmentation result is output.
Wherein sigmoid is an activation function and e is a constant.
In this application, the channel fusion module (Copy), as shown in fig. 5, is configured to combine the upsampled lower layer feature map and the current layer feature map according to channels, and output a feature map with unchanged size and changed channel number into the sum of the lower layer feature map and the current layer feature map.
Similarly, the convolution kernel size of the simple feature extraction module in this embodiment is 3*3, and may be 5*5 or 7*7 as needed.
It should be noted that, in the present application, C is the number of channels, H is the height of the picture, W is the width of the picture, and the subscript of the letter indicates the sequence number, so as to distinguish the dimensions of different feature graphs.
According to the method and the device, the characteristics of the boundary of the segmented region can be directly learned through the boundary network, the boundary of the segmented region can be found more easily, and the segmentation effect on the conditions of difference in the classes and similarity between the classes is improved. For partial images, there may be a large difference in image characteristics between two parts within the same semantic region, which are easily identified as two types of semantic regions, which are referred to as intra-class variability. Similarly, there is a greater similarity between image features in different semantic regions of a part of an image, so that the two parts can be easily identified as a type of semantic region, which is called as an inter-type similarity. By learning the boundary information, the correct semantic boundary can be better found, and the segmentation effect when images with intra-class differences and inter-class similarity are encountered can be well improved.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (6)
1. The tooth full-view picture semantic segmentation method based on feature map disturbance and boundary supervision is characterized by comprising the following steps of:
obtaining a tooth full view picture, and sharpening the tooth full view picture to obtain a tooth full view picture I with clearer tooth boundaries;
inputting the tooth full-scene picture I into a disturbance feature map extraction network after training to perform feature extraction to obtain a deep disturbance feature map F deep ；
Map F of deep disturbance deep Respectively inputting the tooth region segmentation result and the tooth contour segmentation result into a mask network and a boundary network after training is completed;
the tooth full-scene image I is input into a disturbance feature image extraction network after training to perform feature extraction, so as to obtain a deep disturbance feature image F deep Comprising:
step 2.1, inputting the tooth full-scene film I into a simple feature extraction module with the convolution kernel size of 3*3 to obtain an output feature map F 1 Its dimension is C 1 ×H 1 ×W 1 ；
Step 2.2, feature map F 1 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 2 Its dimension is C 2 ×H 2 ×W 2 ；
Step 2.3, feature mapF 2 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 3 Its dimension is C 3 ×H 3 ×W 3 ；
Step 2.4, feature map F 3 After pooling, inputting the obtained product to a disturbance feature extraction module with a convolution kernel of 3*3 to obtain a feature map F 4 Its dimension is C 4 ×H 4 ×W 4 ；
Step 2.5, feature map F 4 The pooled disturbance characteristic is input into a disturbance characteristic extraction module with the convolution kernel size of 3*3 to obtain a deep disturbance characteristic diagram F deep Its dimension is C 5 ×H 5 ×W 5 ；
Wherein the deep disturbance feature map F deep Respectively inputting the tooth region segmentation result and the tooth contour segmentation result into a mask network and a boundary network which are trained, and obtaining the tooth region segmentation result and the tooth contour segmentation result, wherein the tooth region segmentation result and the tooth contour segmentation result comprise,
step 3.1, in the mask network, depth disturbance feature map F deep Up-sampling and F 4 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 5 +C 4 Inputting the simple feature extraction module to obtain a feature map UP 4 Its dimension is C 4 ×H 4 ×W 4 ；
Step 3.2, feature map UP 4 Up-sampling and F 3 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 4 +C 3 Inputting the simple feature extraction module to obtain a feature map UP 3 Its dimension is C 3 ×H 3 ×W 3 ；
Step 3.3, feature map UP 3 Up-sampling and F 2 Inputting the feature images into a channel fusion module together to obtain a feature image after channel combination, wherein the number of channels is C 3 +C 2 Inputting the simple feature extraction module to obtain a feature map UP 2 Its dimension is C 2 ×H 2 ×W 2 ；
Step 3.4, feature map UP 2 Up-sampling and F 1 Together input channelsThe fusion module is used for obtaining a characteristic diagram after being combined according to channels, wherein the number of the channels is C 2 +C 1 Inputting the simple feature extraction module to obtain a feature map UP 1 Its dimension is C 1 ×H 1 ×W 1 ；
Step 3.5, feature map UP 1 Inputting 1*1 convolution block to obtain a characteristic map UP 0 Dimension 32 XH 1 ×W 1 Wherein 32 represents 32 different teeth, UP 0 Each channel of (2) is activated by the following formula to obtain UP 0 Multiplying the probability that each pixel point belongs to the tooth area by 255 to obtain a segmentation result of the final 32 teeth;
and step 3.6, the operations of the steps 3.1 to 3.5 are adopted in the boundary network, and finally, the tooth profile segmentation result is output.
2. The method for semantic segmentation of dental full-view sheets based on feature map perturbation and boundary supervision according to claim 1, wherein the step of obtaining the dental full-view sheets, and the step of obtaining dental full-view sheets I with clearer tooth boundaries by sharpening the dental full-view sheets comprises the following steps:
input original dental full view film I original And (3) sharpening filtering operation is carried out on all the dental full-view sheets by adopting a filter, and the filter kernel is 3*3 to obtain the sharpened dental full-view sheet I.
3. The feature map perturbation and boundary supervision-based tooth full-scene semantic segmentation method of claim 1, wherein the simple feature extraction module comprises two sets of serially connected convolution layers with convolution kernel size 3*3, a batch normalization layer and an activation layer.
4. The feature map perturbation and boundary supervision-based tooth full-scene semantic segmentation method of claim 1, wherein the perturbation feature extraction module comprises two groups of convolution layers with the convolution kernel size of 3*3, feature map perturbation operation, batch normalization layer and activation layer which are connected in series.
5. The method for semantic segmentation of dental full-view sheets based on feature map perturbation and boundary supervision according to claim 1, wherein the channel fusion module is used for combining the up-sampled lower-layer feature map and the current-layer feature map according to channels, and outputting feature maps with unchanged sizes and channel numbers which are the sum of the lower-layer feature map and the current-layer feature map.
6. The feature map perturbation and boundary supervision-based tooth full-scene semantic segmentation method according to claim 4, wherein the feature map perturbation operation realizes perturbation of a feature map by using the following formula;
wherein x is i To input a feature map, f (x i ) Andrespectively representing the characteristic diagram before disturbance and after disturbance, m i Consisting of 0 and 1, following a Bernoulli distribution, ε i For controlling the disturbance amplitude, the parameter value is automatically optimized in training, and the matrix is expressed by corresponding multiplication of each point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010894993.7A CN112085028B (en) | 2020-08-31 | 2020-08-31 | Tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010894993.7A CN112085028B (en) | 2020-08-31 | 2020-08-31 | Tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112085028A CN112085028A (en) | 2020-12-15 |
| CN112085028B true CN112085028B (en) | 2024-03-12 |
Family
ID=73731245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010894993.7A Active CN112085028B (en) | 2020-08-31 | 2020-08-31 | Tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112085028B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112750111B (en) * | 2021-01-14 | 2024-02-06 | 浙江工业大学 | Disease identification and segmentation method in tooth full-view film |
| CN114004831B (en) * | 2021-12-24 | 2022-04-08 | 杭州柳叶刀机器人有限公司 | Method for assisting implant replacement based on deep learning and auxiliary intelligent system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109087327A (en) * | 2018-07-13 | 2018-12-25 | 天津大学 | A kind of thyroid nodule ultrasonic image division method cascading full convolutional neural networks |
| CN109829900A (en) * | 2019-01-18 | 2019-05-31 | 创新奇智(北京)科技有限公司 | A kind of steel coil end-face defect inspection method based on deep learning |
| CN110473243A (en) * | 2019-08-09 | 2019-11-19 | 重庆邮电大学 | Tooth dividing method, device and computer equipment based on depth profile perception |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11645746B2 (en) * | 2018-11-28 | 2023-05-09 | Orca Dental AI Ltd. | Dental image segmentation and registration with machine learning |
-
2020
- 2020-08-31 CN CN202010894993.7A patent/CN112085028B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109087327A (en) * | 2018-07-13 | 2018-12-25 | 天津大学 | A kind of thyroid nodule ultrasonic image division method cascading full convolutional neural networks |
| CN109829900A (en) * | 2019-01-18 | 2019-05-31 | 创新奇智(北京)科技有限公司 | A kind of steel coil end-face defect inspection method based on deep learning |
| CN110473243A (en) * | 2019-08-09 | 2019-11-19 | 重庆邮电大学 | Tooth dividing method, device and computer equipment based on depth profile perception |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112085028A (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112085028B (en) | Tooth full-scene semantic segmentation method based on feature map disturbance and boundary supervision | |
| CN112750111B (en) | Disease identification and segmentation method in tooth full-view film | |
| CN110675462B (en) | Gray image colorization method based on convolutional neural network | |
| CN108492271A (en) | A kind of automated graphics enhancing system and method for fusion multi-scale information | |
| CN113221945B (en) | Dental caries identification method based on oral panoramic film and dual attention module | |
| CN111696084B (en) | Cell image segmentation method, device, electronic equipment and readable storage medium | |
| Hou et al. | Teeth U-Net: A segmentation model of dental panoramic X-ray images for context semantics and contrast enhancement | |
| CN112837278B (en) | Tooth full-scene caries identification method based on depth boundary supervision | |
| CN109544451A (en) | A kind of image super-resolution rebuilding method and system based on gradual iterative backprojection | |
| CN114332123A (en) | Automatic caries grading method and system based on panoramic film | |
| CN113344950A (en) | CBCT image tooth segmentation method combining deep learning with point cloud semantics | |
| CN116630621A (en) | Image segmentation method integrating multi-scale features | |
| CN113344867A (en) | Periodontitis absorption degree identification method based on near-middle and far-middle key points | |
| CN115796306B (en) | Training of permanent tooth maturity grading model and permanent tooth maturity grading method | |
| CN114743251B (en) | Drama character facial expression recognition method based on shared integrated convolutional neural network | |
| CN115482384A (en) | Visible light OCT image retina layer segmentation method and system | |
| CN116205925A (en) | Tooth occlusion wing tooth caries segmentation method based on improved U-Net network | |
| CN115760875A (en) | Full-field medical picture region segmentation method based on self-supervision learning | |
| CN113705613A (en) | X-ray sheet distal radius fracture classification method based on spatial position guidance | |
| CN114360034A (en) | Method, system and equipment for detecting deeply forged human face based on triplet network | |
| CN109446944B (en) | Visual semantic-structured analytic method of sign language | |
| Reeshav et al. | Sign language recognition using convolutional neural network | |
| CN116402682B (en) | Image reconstruction method and system based on differential value dense residual super-resolution | |
| CN1508756A (en) | Sensitive image identifying method based on body local and shape information | |
| Krishnaveni | Classification of Dental Disease Through Various CNN Techniques and Performance Analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |