KR20190090664A - Dental area extraction system and method - Google Patents

Abstract

The present invention for achieving the above object relates to a dental area extraction system. The dental area extraction system includes: a database which stores a plurality of 3D dental CT images and a first dental area image obtained by separating a portion corresponding to a dental area from each of the 3D dental CT images; a first artificial neural network system which determines a first weight for the presence of teeth by learning based on the 3D dental CT image and the first dental area image provided from the database, and estimating the presence of teeth from the 3D dental CT image input from the outside; and a second artificial neural network system which determines a second weight of a dental region by learning based on the 3D dental CT image and the first dental area image provided from the database, estimating a dental area from the 3D dental CT image input from the outside, and extracting the dental area. It is possible to automatically separate and acquire the dental area by deep learning.

Description

Dental area extraction system and method {DENTAL AREA EXTRACTION SYSTEM AND METHOD}

The present invention relates to a tooth region extraction system and method, and more particularly to a tooth region extraction system and method for automatically separating and precisely acquiring a tooth region using deep learning in a three-dimensional computed tomography (hereinafter, referred to as CT) image. It is about.

Generally, dentists need to know the exact shape of the patient's teeth and surrounding tissue structures for patient care and surgical designs such as crowns and implants. To this end, dental practitioners take 3D CT images of the patient's mouth to check the shape and size of surrounding tissues. For more precise diagnosis, only the area of the tooth is separated from the 3D CT image to determine the patient's dental condition. I want to look.

In this case, when the voxel of the dental CT image is quantified, the characteristics of the same tissue are not uniform, and noise is included due to the characteristic of transmitting the inside of the body of the CT image. Therefore, it is difficult to precisely separate the teeth and the surrounding tissue from the CT image through the conventional image processing method.

In addition, a method mainly used for separating tooth regions from a CT image is a method of manually designating a region corresponding to a tooth region and automatically grouping regions having similar properties by an algorithm. However, this method takes a lot of time for the operator to obtain a precise result, it is difficult to precisely separate the tooth area. Therefore, there is a need for a system and method that allows simpler and more precise separation of dental regions from CT images.

The present invention relates to a tooth region extraction system and method, and more particularly, to accurately separate and acquire a tooth region using deep learning in a 3D CT image.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention, which are not mentioned above, can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

In order to achieve the above object, the present invention provides a tooth region extraction system comprising: a first tooth region image obtained by separating a portion corresponding to a tooth region from a plurality of three-dimensional teeth CT images and respective three-dimensional teeth CT images; A first weight for the presence or absence of a tooth is determined through a stored database and learning based on the 3D tooth CT image and the first tooth region image provided from the database, and the tooth is determined from an external 3D tooth CT image. The second artificial neural network system for estimating the presence or absence and a second weight for the tooth region are determined through learning based on the 3D tooth CT image and the first tooth region image provided from the database, Second artificial neural network system extracting tooth region by estimating tooth region from dimensional tooth CT image It includes.

The present invention, while learning the artificial neural network for a large number of three-dimensional CT image, by repeatedly updating the weight of the artificial neural network until the desired tooth region is obtained, the effect of more precise separation acquisition of the tooth region in the three-dimensional CT image There is.

In addition, the artificial neural network structure used for deep learning in the present invention has the effect of improving the accuracy of the estimation of the tooth region by supplementing the original data is lost.

1 is a block diagram showing a tooth region extraction system according to an embodiment of the present invention.
2 is a view showing a Recurrent Neural Network (RNN) method used in a tooth region extraction system according to an embodiment of the present invention.
3 is a view showing a connection structure of the artificial neural network used in the tooth region extraction system according to an embodiment of the present invention.
4 is a view showing the structure of the first artificial neural network system in the tooth region extraction system according to an embodiment of the present invention.
5 is a view showing the structure of the second artificial neural network system in the tooth region extraction system according to an embodiment of the present invention.
6 is a view showing the presence of teeth and the estimation result of the tooth region in the tooth region extraction system according to an embodiment of the present invention.
7 is a view showing an image output from the image output unit of the tooth region extraction system according to an embodiment of the present invention.
8 is a flowchart illustrating a tooth region extraction method according to an embodiment of the present invention.
9 is a flowchart illustrating steps S200 and S300 in the tooth region extraction method according to an embodiment of the present invention.

The foregoing objects, features, and advantages will become more apparent from the following examples taken in conjunction with the accompanying drawings.

It is to be understood that the specific structure or functional description is illustrative only for the purpose of describing an embodiment according to the concept of the present invention and that the embodiments according to the concept of the present invention may be embodied in various forms, Should not be construed as limited to these.

The embodiments according to the concept of the present invention can make various changes and have various forms, so that specific embodiments are illustrated in the drawings and described in detail in the specification of the present application. However, it should be understood that the embodiments according to the concept of the present invention are not intended to limit the present invention to specific modes of operation, but include all changes, equivalents and alternatives included in the spirit and scope of the present invention.

Whenever an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but there may be other elements in between It should be understood. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions for describing the relationship between components, such as "between" and "immediately between" or "adjacent to" and "directly adjacent to", should be interpreted as well.

The terminology used in the specification of the present application is used only to describe a specific embodiment and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. It is to be understood that the terms such as " comprises "or" having "in this specification are intended to specify the presence of stated features, integers, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Unless defined otherwise, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements.

1 is a block diagram showing a tooth region extraction system according to an embodiment of the present invention. In FIG. 1, the tooth region extraction system includes a database 100, an image input unit 200, a first artificial neural network system 300, a second artificial neural network system 400, and an image output unit 500.

The database 100 stores a plurality of 3D tooth CT images and a tooth region image obtained by separating a portion corresponding to a tooth region from each 3D tooth CT image.

The image input unit 200 receives a 3D dental CT image from the outside and performs a preprocessing process on the CT image. Here, the preprocessing process refers to normalization of a field of view and a sampling interval difference that are different for each image. That is, by normalizing a plurality of two-dimensional CT slices that can be configured differently for each three-dimensional CT image, it is a process for facilitating internal calculation of CT slices in an artificial neural network.

The first artificial neural network system (300) is a tooth through the learning based on the tooth region image obtained by separating a portion corresponding to the tooth region from a plurality of three-dimensional teeth CT images and each of the three-dimensional teeth CT images provided from the database, A weight for presence or absence is determined, and the presence or absence of teeth is estimated from an external 3D dental CT image. Here, the artificial neural network refers to a hierarchical structure of a plurality of layers modeled similarly to a human recognition method. The weight refers to the strength connecting the layers in the artificial neural network, and by adjusting the weight repeatedly, the artificial neural network may be learned. Hereinafter, a tooth region image acquired by separating a portion corresponding to a tooth region from a 3D tooth CT image by a user by hand or the like is referred to as a first tooth region image.

The second artificial neural network system 400 determines weights for tooth regions through learning based on a plurality of three-dimensional tooth CT images provided from a database and first tooth region images for each three-dimensional tooth CT images. The tooth region is extracted by estimating the tooth region from the 3D dental CT image input from the outside.

The image output unit 500 outputs a binary mask in which the tooth region is separated based on the estimated tooth region.

2 is a view showing a Recurrent Neural Network (RNN) method used in the tooth region extraction system according to an embodiment of the present invention, Figure 3 is an artificial neural network used in the tooth region extraction system according to an embodiment of the present invention Is a diagram showing a connection structure of a. Referring to FIG. 2, in the present invention, the first artificial neural network system 300 and the second artificial neural network system 400 utilize deep learning technology employing a Recurrent Neural Network (RNN) scheme. The RNN method estimates output data (e.g., mask image) for data input from the outside (e.g., a CT slice) through an artificial neural network, and then inputs the input data and the estimated output data to the artificial neural network again. By repeating the above operation, the output data is more accurately estimated. Here, the CT slice refers to the two-dimensional teeth image constituting the three-dimensional teeth CT image, the mask image refers to the two-dimensional image constituting the mask image obtained by separating the portion corresponding to the tooth region in the three-dimensional teeth CT image do.

In addition, in the present invention, the artificial neural network structure preferably uses FCN (Fully Convolutional Network) technique. The FCN technique reduces the size of the input image through the convolution layers several times, extracts the features within the image, and then classifies the images through the fully-connected layer. Say technique.

Referring to FIG. 3, in the present invention, the first artificial neural network system 300 and the second artificial neural network system 400 may include a first neural network, a second neural network, and a third neural network. Each neural network may be connected to each other, and a dotted arrow indicates a data flow when the first and second neural networks are connected.

The first neural network is a neural network that allows a CT slice to pass through a plurality of convolutional layers and a pulling layer in consideration of position information of a CT image. In this process, the CT slices are compressed to extract features. Here, the convolutional layer is a layer for extracting features from the input data, and the pooling layer is a layer for compressing data by subsampling the features extracted through the convolutional layer. In this case, the pooling layer may be a max pulling layer that compresses data by extracting a maximum value among features extracted from the convolutional layer.

In the second neural network, after the concatenation process of the data provided from the first neural network with the CT slice, the CT slice concatenated with the data provided from the first neural network passes through the convolutional layer and the up-conv layer. This is a neural network that extracts image data from which a specific region is estimated through a sigmoid function. In this process, the resolution is restored as the CT slice is enlarged in stages. In this case, the joining is to combine two matrices side by side in a specific axial direction to form one matrix, and the upconve layer is a layer for upsampling, ie, expanding the reduced matrix again, and applying convolution operations in reverse. The sigmoid function is a discrete classification function for finding the minimum point.

The third neural network is a multi-layer perceptron (MLP) composed of a plurality of fully-connected layers, and is a neural network that grasps the meaning of an image in a CT image through an entirely connected neural network connection structure.

4 is a view showing the structure of the first artificial neural network system 300 in the tooth region extraction system according to an embodiment of the present invention, Figure 5 is a second in the tooth region extraction system according to an embodiment of the present invention The structure of the artificial neural network system 400 is shown. 6 is a view showing the presence of the tooth and the estimation result of the tooth region in the tooth region extraction system according to an embodiment of the present invention, Figure 7 is an image of the tooth region extraction system according to an embodiment of the present invention The image output from the output unit 500 is a view.

1 to 7, the operation of the tooth region separation system will be described in detail. The operations in each artificial neural network are performed sequentially for each of the plurality of CT slices.

The first artificial neural network system 300 proceeds in the forward direction, compresses the input 3D dental CT image to distinguish the presence of teeth, and converts the compressed image into a lined vector to estimate the presence or absence of teeth. Create Specifically, the first artificial neural network system 300 reduces the size of the CT slice input through the first neural network so as to distinguish the presence of teeth, and flattens the reduced result into a lined vector, 3 Neural network to estimate the presence of teeth. At this time, the final result is expressed as a two-dimensional vector. If the output value is not present through the softmax function, the zeroth index of the vector appears as 1, and if the tooth exists, the first index is represented as 1. . Here, the softmax function is a function that normalizes all inputs to a value between 0 and 1, and outputs them so that the sum of the outputs is 1. When the vector data of the K dimension is z, the softmax function is represented by Equation 1 below.

The second artificial neural network system 400 proceeds in the forward direction, and estimates the tooth center region in the CT image, and the input data are previously inputted data for the presence of the CT slice and the presence of the teeth of the first artificial neural network system 300, and previously inputted. The data on which the estimation of the tooth center region in the CT slice has been made may be the joined data. The second artificial neural network system 400 uses an artificial neural network to which the first neural network and the second neural network are connected. The first neural network projects the input data in a size that can distinguish regions having tooth characteristics, and the second neural network groups the regions having the same characteristics by combining features distinguished through the first neural network. do. Accordingly, a mask image in which a region in which the tooth is assumed to exist is represented by 1 and a region in which the tooth is estimated to be absent is represented by 0 is output. Here, the grouping using the second neural network may be a semantic segmentation technique in which regions having the same feature extracted from the image are grouped with the same label. In addition, referring to the data flow of FIG. 2, a method in which the first neural network and the second neural network are connected may be a u-net method, wherein the u-net method is an image in the first neural network. When the size is downsampled, the output data is connected to the splicing step in the upsampling process of the image size in the second neural network. That is, as the layer deepens in the artificial neural network structure, the information of the original data lost is fed to the deep layer to improve the accuracy of the estimation and to facilitate the learning of the artificial neural network.

In addition, referring to FIG. 6, the estimation result of the presence or absence of a tooth is represented by a graph on the right, and the X axis is a probability value having a value of 1 for CT slices with teeth and 0 for CT slices without teeth, and Y. The axis is the number of CT slices. The arch-shaped point image shown below the CT image is an estimated image of the tooth region, and the point image is displayed on the CT slice only in the region where the X axis is 1 in the right graph.

The second artificial neural network system 400 re-progresses in the forward direction and estimates the tooth region more precisely. The input data may be data in which a CT slice and a mask image and a mask image of a previously input CT slice are bonded. The second artificial neural network system 400 joins the input CT image with the position information about the tooth region estimated through re-progression, so that the tooth region is represented by 1 and the tooth region is represented by 0. Outputs

Thereafter, the second artificial neural network system 400 separates the tooth region from the output mask image. In this case, the image processing is separated using a labeling algorithm and a watershed algorithm, which are well known to those skilled in the art, and thus a detailed operation thereof will be omitted.

At the time of learning each artificial neural network system, the artificial neural network system is processed in the opposite direction to the estimated direction of each artificial neural network system, and the artificial neural network is trained in a direction that minimizes the difference between the desired result (Ground Truth) and the predicted image estimated by the artificial neural network system. In the present invention, when learning about artificial neural networks, gradient descent may be applied, and batch normalization may be used to solve a local minima problem. Here, the gradient descent method is a method of finding the lowest slope from the current position and moving to that direction to find the lowest point. The problem of the local optimal point is a problem that does not go in the direction intended by the user in the process of finding the lowest point through the artificial neural network, but in a different direction. Batch normalization is used to normalize the output value or activation value of the function. In this method, the distribution of data is normalized for each layer of the neural network.

Specifically, the first artificial neural network system 300 proceeds in the reverse direction, and through the learning based on a plurality of three-dimensional teeth CT images provided from the database and the first tooth region image for each three-dimensional teeth CT image, The first weight for presence or absence is determined. After receiving the 3D tooth CT image and the first tooth region image stored in the database, the second tooth region image is obtained by separating the portion corresponding to the tooth region from the 3D tooth CT image and then obtaining the first tooth region image. By comparing the second tooth region image, the first weight is repeatedly updated until a desired result is obtained by the user. Here, the second tooth region image refers to an image obtained by separating and acquiring a portion corresponding to the tooth region in the 3D tooth CT image through an artificial neural network system.

The second artificial neural network system 400 proceeds in the reverse direction and learns about the tooth region through learning based on a plurality of three-dimensional tooth CT images provided from a database and a first tooth region image of each three-dimensional tooth CT image. Determine a second weight. After receiving the 3D tooth CT image and the first tooth region image stored in the database, the second tooth region image is obtained by separating the portion corresponding to the tooth region from the 3D tooth CT image and then obtaining the first tooth region image. By comparing the second tooth region image with the second tooth region image, the second weight value is repeatedly updated until a desired result is obtained by the user.

Referring to FIG. 6, the image output unit 500 may output a binary mask in which all teeth are precisely separated, and may perform post-filtering smoothing to smooth the boundary. Smooth filtering post-processing uses algorithms well known to those skilled in the art, and thus, detailed descriptions thereof will be omitted.

8 is a flowchart illustrating a tooth region extraction method according to an embodiment of the present invention. In FIG. 8, a tooth region extraction method from a 3D dental CT image will be described with reference to the description of the tooth region extraction system described above.

First, in step S100, images necessary for learning the artificial neural network are stored in a database. Here, the images necessary for learning the artificial neural network mean a first tooth region image obtained by separating a portion corresponding to a tooth region from a plurality of 3D tooth CT images and respective 3D tooth CT images.

In step S200, the weight is determined by learning an artificial neural network based on the 3D dental CT image and the first tooth region image provided from the database.

In operation S300, a part corresponding to the tooth region is separately acquired from the 3D dental CT image input from the outside by using the artificial neural network whose weight is determined.

In operation S400, the separated tooth region is output as a binary mask.

9 is a flowchart illustrating steps S200 and S300 in the tooth region extraction method according to an embodiment of the present invention.

As shown in FIG. 9, in step S210, the first artificial neural network system is trained on the presence of teeth from a plurality of three-dimensional dental CT images stored in a database to determine a first weight of the first artificial neural network system. .

In step S310, using the first artificial neural network system to compress the input three-dimensional teeth CT image to distinguish the presence of the teeth, and to generate data for estimating the presence of teeth by changing the compressed image into a lined vector do.

In operation S220, the second artificial neural network system is trained on the tooth center region of the three-dimensional CT image from the plurality of three-dimensional dental CT images stored in the database to determine a second weight of the second artificial neural network system.

In operation S320, a second artificial neural network system generating data, that is, a mask image, in which regions having the same feature values are grouped in the input 3D dental CT image so as to distinguish a tooth center region is used.

In operation S230, the second artificial neural network system updates the second weight by comparing the first tooth region image and the second tooth region image stored in the database.

In operation S330, the mask image is bonded to the input 3D dental CT image, and the entire tooth region is estimated from the 3D dental CT image bonded using the second artificial neural network system. Separate the relevant parts.

Although the invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and those skilled in the art will appreciate that various modifications and equivalent other embodiments are possible therefrom. . Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (16)

In the tooth area extraction system,
A database storing a first tooth region image obtained by separating a portion corresponding to a tooth region from a plurality of 3D tooth CT images and each 3D tooth CT image;
An artificial neural network system extracting a tooth region by estimating the presence or absence of a tooth from a 3D tooth CT image input from the outside based on the 3D tooth CT image and the first tooth region image provided from the database;
An image output unit for outputting the extracted tooth region as a binary mask
Tooth area extraction system.
The method of claim 1,
The artificial neural network system,
The first weight for the presence of the tooth is determined by learning based on the 3D tooth CT image and the first tooth region image provided from the database, and the tooth is present from the 3D tooth CT image input from the outside. A first artificial neural network system for estimating the presence and absence of
The second weight of the tooth region is determined by learning based on the 3D tooth CT image and the first tooth region image provided from the database, and the tooth region is determined from the 3D tooth CT image input from the outside. Including a second artificial neural network system to extract
Tooth area extraction system.
3. The method of claim 2,
At least one of the first or second artificial neural network system,
After receiving the 3D tooth CT image and the first tooth region image stored in the database, and obtaining a second tooth region image by separating a portion corresponding to the tooth region from the 3D tooth CT image, the first tooth region Updating at least one of the first or second weights by comparing an image with a second tooth region image
Tooth area extraction system.
The method of claim 3, wherein
The first artificial neural network system,
Compressing the input three-dimensional teeth CT image to distinguish the presence of the teeth, and converts the compressed image into a lined vector to generate data to estimate the presence of teeth
Tooth area extraction system.
The method of claim 3, wherein
The second artificial neural network system,
In order to distinguish the central tooth region, the data having the same feature values are grouped in the input 3D dental CT image.
The grouped data is bonded to the 3D tooth CT image, the entire tooth region is estimated from the bonded 3D tooth CT image and the grouped data, and a portion corresponding to the tooth region is separated from the 3D tooth CT image. doing
Tooth area extraction system.
5. The method of claim 4,
The data for estimating the presence or absence of teeth,
Represented as a two-dimensional vector, the tooth has a one-hot vector in which the first index is activated to 1, and if the tooth is not present, the zero index is activated to 1.
Tooth area extraction system.
The method of claim 5, wherein
The grouped data is,
The area where teeth are assumed to be present is represented by 1, and the area where the tooth is not present is represented by 0, which is mask image data.
Tooth area extraction system.
The method of claim 5, wherein
The estimated entire area of the tooth is,
The area where the tooth is present is 1 and the area where the tooth is not is 0.
Tooth area extraction system.
In the tooth region extraction method,
Storing a first tooth region image obtained by separating a portion corresponding to a tooth region from a plurality of 3D tooth CT images and each 3D tooth CT image in a database;
Determining a weight by learning an artificial neural network based on the 3D dental CT image and the first tooth region image provided from the database; and
Using the weighted artificial neural network, separating and acquiring a portion corresponding to a tooth region from an external 3D dental CT image
Tooth area extraction method.
The method of claim 9,
Determining the weight,
After receiving the 3D tooth CT image and the first tooth region image stored in the database, and obtaining a second tooth region image by separating a portion corresponding to the tooth region from the 3D tooth CT image, the first tooth region And updating the weight by comparing the image with the second tooth region image.
Tooth area extraction method.
11. The method of claim 10,
Separately acquiring the tooth region,
Using a first artificial neural network system to compress the input 3D dental CT image to distinguish the presence of teeth and to generate data estimating the presence of teeth by changing the compressed image into a lined vector;
Using a second artificial neural network system to generate data in which regions having the same feature values are grouped in the input 3D dental CT image so as to distinguish a central tooth region; And
Bonding the grouped data to the 3D dental CT image, estimating the entire tooth region from the bonded 3D dental CT image and the grouped data using the second artificial neural network system, and the 3D dental CT image Comprising the step of obtaining a portion corresponding to the tooth region in the
Tooth area extraction method.
The method of claim 11,
The data for estimating the presence or absence of teeth,
Represented as a two-dimensional vector, the tooth has a one-hot vector in which the first index is activated to 1, and if the tooth is not present, the zero index is activated to 1.
Tooth area extraction method.
The method of claim 11,
The grouped data is,
The area where teeth are assumed to be present is represented by 1, and the area where the tooth is not present is represented by 0, which is mask image data.
Tooth area extraction method.
The method of claim 11,
The estimated entire area of the tooth is,
The area where the tooth is present is 1 and the area where the tooth is not is 0.
Tooth area extraction method.
The method of claim 11,
Determining the weight,
Learning the presence of teeth from the plurality of three-dimensional dental CT images stored in the database to determine the first weight of the first artificial neural network system;
Learning the second artificial neural network system from the plurality of three-dimensional dental CT images stored in the database on a tooth center region in a three-dimensional CT image to determine a second weight of the second artificial neural network system; And
And updating the second artificial neural network system by comparing the first tooth region image and the second tooth region image stored in the database with a second weight.
Tooth area extraction method.
The method of claim 9,
Outputting the separated tooth region as a binary mask;
Tooth area extraction method.

Images