KR20220097697A - Method and system of medical image segmentation using fusion deep neural network model - Google Patents

Abstract

The present invention relates to a method and a system for dividing a medical image by using a fusion deep-neural network model and, more specifically, to a method and a system for dividing a medical image by using a fusion deep-neural network model, capable of more accurately dividing an image by fusing shape characteristic information of the image extracted from a convolutional neural network-based model with brightness information of the image extracted from a Gaussian mixture model. In accordance with one embodiment of the present invention, the system for dividing a medical image by using a fusion deep-neural network model, includes: an input part receiving medical image data; a shape characteristic part extracting a shape characteristic value of an image through a convolutional neural network-based U-net from the medical image data; a brightness characteristic part extracting a brightness characteristic value of the image through a Gaussian mixture model from the medical image data; a combination part combining the shape characteristic value with the brightness characteristic value through a channel-wise, and fusing the values through a two-dimensional neural network; and a dividing part encoding or decoding the medical image data in which the shape characteristic value and the brightness characteristic value are fused, multiple times to divide the medical image data.

Description

Method and system of medical image segmentation using fusion deep neural network model

The present invention relates to a medical image segmentation method and system using a convergence deep neural network model, and more specifically, the fusion of shape characteristic information of an image extracted from a convolutional neural network-based model and brightness information of an image extracted from a Gaussian mixture model. It relates to a medical image segmentation method and system capable of more accurately segmenting an image through

Recently, in the development of an automatic diagnosis assistance system through medical image reading, the problem of segmentation of a specific lesion or organ in various CT or MR images is being treated as an essential problem in determining the patient's current condition or predicting the prognosis.

As artificial intelligence develops, easier and more accurate diagnosis is attempted through the assistance of artificial intelligence, which has various advantages. In the medical field, artificial intelligence can play a role as an auxiliary means for diagnosis rather than final diagnosis.

As a prior art in this regard, Korean Patent Application Laid-Open No. 2015-0098119 relates to a system and method for removing false-positive lesion candidates in medical images. Elimination of benign lesion candidates is suggested.

However, there is no suggestion of a technique for automatically classifying medical images through machine learning, and for accurately segmenting parts necessary for diagnosis in medical images.

In the present invention, by fusing the shape characteristic information and the brightness characteristic information of the image to segment the parts necessary for diagnosis in the medical image, the time taken to the final diagnosis is reduced to increase the efficiency of disease diagnosis using the medical image and to reduce the possibility of misdiagnosis. An image segmentation method and system are presented.

KR 10-2015-0098119

In order to solve the above-mentioned problems, the present invention provides a system for more accurately segmenting a medical image through the fusion of shape characteristic information of an image extracted from a convolutional neural network-based model and brightness characteristic information of an image extracted from a Gaussian mixture model. and to provide a method.

As an embodiment of the present invention, a medical image segmentation system using a fusion deep neural network model is provided.

A medical image segmentation system using a convergence deep neural network model according to an embodiment of the present invention extracts shape feature values of an image through an input unit receiving medical image data and a convolutional neural network-based U-net from the medical image data a shape feature, a brightness feature for extracting a brightness feature value of an image from the medical image data through a Gaussian mixture model, and combining the shape feature value and the brightness feature value through channel-wise, 2 It may include a combining unit that converges through a dimensional convolutional neural network, and a division unit that divides the medical image data by encoding or decoding the medical image data in which the shape feature value and the brightness feature value are fused multiple times.

In the medical image segmentation system using the convergence deep neural network model according to an embodiment of the present invention, the input unit includes a preprocessing unit that converts the medical image data into a combination of a plurality of 2D images when the medical image data is a 3D image may include more.

In the medical image segmentation system using the convergence deep neural network model according to an embodiment of the present invention, the shape feature may extract a shape feature value by encoding or decoding the medical image data a plurality of times.

In the medical image segmentation system using a convergence deep neural network model according to an embodiment of the present invention, the brightness feature unit extracts a plurality of Gaussian distributions for brightness information of an image from the medical image data, and calculates the plurality of Gaussian distributions. A mixed probabilistic model may be generated using the EM algorithm, the mixed probabilistic model may be trained using an EM algorithm (Expectation-maximixation algorithm), and parameters of the mixed probabilistic model may be estimated.

As an embodiment of the present invention, a medical image segmentation method using a fusion deep neural network model is provided.

A medical image segmentation method using a convergence deep neural network model according to an embodiment of the present invention includes an image input step of receiving medical image data, and a shape feature value of an image through a convolutional neural network-based U-net from the medical image data. A feature extraction step of extracting . The method may include combining features of fusion and segmenting the medical image data by encoding or decoding the medical image data in which the shape feature value and the brightness feature value are fused multiple times to divide the medical image data.

In the medical image segmentation method using a fused deep neural network model according to an embodiment of the present invention, the image input step includes converting the medical image data into a combination of a plurality of 2D images when the medical image data is a 3D image. It may further include a pretreatment step.

In the medical image segmentation method using the convergence deep neural network model according to an embodiment of the present invention, the feature extraction step further includes a shape feature extraction step of extracting shape feature values by encoding or decoding the medical image data a plurality of times. can do.

In the medical image segmentation method using a fused deep neural network model according to an embodiment of the present invention, the feature extraction step includes extracting a plurality of Gaussian distributions for brightness information of an image from the medical image data, the Gaussian distribution extraction step, the A probability model generation step of generating a mixed probability model using a plurality of Gaussian distributions and a parameter estimation step of learning the mixed probability model using an EM algorithm (Expectation-maximixation algorithm) and estimating the parameters of the mixed probability model may include more.

As an embodiment of the present invention, a computer-readable recording medium in which a program for implementing the above-described method is recorded is provided.

According to the present invention, since not only the morphological features of the image but also the brightness information can be considered together, it is possible to accurately segment a medical image, and there is an advantage that the time taken for diagnosis can be shortened by analysis using a deep neural network.

In addition, the present invention is advantageous in that it is not limited to a specific field and can be applied to various fields of diagnosing a disease using a medical image.

The effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 is a flowchart of a medical image segmentation method using a fused deep neural network model according to an embodiment of the present invention.
2 is an exemplary diagram of the structure of a fusion deep neural network according to an embodiment of the present invention.
3 is a flowchart of a feature extraction step of a medical image segmentation method using a fused deep neural network model according to an embodiment of the present invention.
4 is a block diagram of a medical image segmentation system using a fusion deep neural network model according to an embodiment of the present invention.
5 is an exemplary diagram showing the performance of a medical image segmentation system and method using a fusion deep neural network model according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. . In addition, throughout the specification, when a part is "connected" with another part, this includes not only the case of "directly connected" but also the case of "connecting with another element in the middle".

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart of a medical image segmentation method using a fused deep neural network model according to an embodiment of the present invention, and FIG. 2 is an exemplary diagram of the structure of a fused deep neural network according to an embodiment of the present invention.

Referring to FIG. 1 , the medical image segmentation method using the fused deep neural network model according to an embodiment of the present invention includes an image input step 100 , a feature extraction step S200 , a feature combining step S300 , and an image segmentation step (S400) may be included.

The image input step 100 receives medical image data, and images obtained from diagnostic devices such as X-ray, ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), and positive emission tomography (PET). can be received and recorded.

According to an embodiment, the image input step 100 of the medical image segmentation method using the fused deep neural network model may further include a pre-processing step S110.

In the pre-processing step S110, when the medical image data is a 3D image, the medical image data may be converted into a combination of a plurality of 2D images. In addition, noise removal, non-homogeneity correction, contrast standardization, and patch-based segmentation can be performed on medical image data.

In the feature extraction step (S200), the shape feature value of the image is extracted from the medical image data through the U-net based on the convolutional neural network, and the brightness feature value of the image is extracted through the Gaussian mixture model.

The shape feature value is a combination of pixel values including spatial information of medical image data, and is extracted by the type of disease to be diagnosed and the structure of the deep neural network used.

The brightness feature value refers to brightness information of medical image data, that is, information about a bright part and a dark part, and may be expressed as a numerical value of one pixel.

U-net is a model based on Fully-Convolutional Network of End-to-End method proposed for the purpose of image segmentation, and is defined as U-net because its network configuration is similar to the letter U. In addition, it is characterized in that the network for obtaining the overall context information of the image and the network for accurate localization are configured in a symmetrical form.

The Gaussian mixture model is a model that can express a complex-shaped probability distribution as a mixture of a plurality of single Gaussian distributions, and can express data distribution in more detail. In this case, the number of single Gaussian distributions is arbitrarily set by the user.

According to an embodiment, the feature extraction step ( S200 ) of the medical image segmentation method using the fusion deep neural network model may further include the shape feature extraction step ( S210 ).

The shape feature extraction step S210 extracts shape feature values by encoding or decoding the medical image data a plurality of times.

In the feature combining step ( S300 ), the shape feature value and the brightness feature value are combined through channel-wise and fused through a two-dimensional convolutional neural network.

The image segmentation step (S400) divides the medical image data by encoding or decoding the medical image data in which the shape feature value and the brightness feature value are fused multiple times.

In this case, the criterion for dividing the medical image through the encoding-decoding process will vary depending on the type of disease to be diagnosed, and will be set by a medical staff or a user.

The convergence deep neural network model according to an embodiment may be implemented as a model architecture having a structure as shown in FIG. 2 .

3 is a flowchart of a feature extraction step (S200) of a medical image segmentation method using a fused deep neural network model according to an embodiment of the present invention.

Referring to FIG. 3 , the feature extraction step (S200) of the medical image segmentation method using the fused deep neural network model according to an embodiment of the present invention includes the Gaussian distribution extraction step (S221), the probability model generation step (S222), and parameter estimation. Step S223 may be further included.

In the Gaussian distribution extraction step S221, a plurality of Gaussian distributions for image brightness information are extracted from the medical image data. Prior to extraction of the Gaussian distribution, pre-training of the Gaussian mixture model is performed using medical image data.

In the probabilistic model generation step S222, a mixed probabilistic model is generated using a plurality of Gaussian distributions. In this case, a mixed probability model is generated using the average, variance, and weight of a plurality of Gaussian distributions.

In the parameter estimation step S223, the mixed probabilistic model is learned by using an EM algorithm (Expectation-maximixation algorithm), and parameters of the mixed probabilistic model are estimated. The parameters are extracted through [Equation 1] below, and include brightness feature values of medical image data.

는 평균,

는 분산,

는 가중치이다.)(where K is the number of Gaussians,

is the average,

is distributed,

is the weight.)

According to an embodiment, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium.

In addition, the data used in the above-described method may be recorded in a computer-readable medium through various means. A recording medium for recording an executable computer program or code for performing various methods of the present invention should not be construed as including temporary objects such as carrier waves or signals. The computer-readable medium may include a storage medium such as a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.) and an optically readable match (eg, a CD-ROM, a DVD, etc.).

4 is a block diagram of a medical image segmentation system using a fusion deep neural network model according to an embodiment of the present invention.

Referring to FIG. 4 , the medical image segmentation system using the fused deep neural network model according to an embodiment of the present invention includes an input unit 100 , a shape feature 200 , a brightness feature 300 , and a combiner 400 . and a division unit 500 .

The input unit 100 receives medical image data, and is a device capable of receiving and storing a CT or MR image.

The shape feature 200 extracts shape feature values of an image through a convolutional neural network-based U-net from the medical image data received by the input unit 100 , and may be configured as a U-net block. In addition, shape feature values may be extracted by encoding or decoding medical image data a plurality of times.

The brightness feature unit 300 extracts the brightness feature value of the image from the medical image data through a Gaussian mixture model,

In addition, the brightness feature unit 300 extracts a plurality of Gaussian distributions for the brightness information of the image from the medical image data, generates a mixed probability model using the plurality of Gaussian distributions, and performs an EM algorithm (Expectation-maximixation algorithm) It is used to learn a mixed probabilistic model and estimate parameters of the mixed probabilistic model.

The combining unit 400 combines the shape feature value and the brightness feature value through channel-wise, and converges through a two-dimensional convolutional neural network.

The dividing unit 500 divides the medical image data by encoding or decoding the medical image data in which the shape feature value and the brightness feature value are fused multiple times. In this case, the criterion for dividing the medical image through the encoding-decoding process will vary depending on the type of disease to be diagnosed, and will be set by a medical staff or a user.

The pre-processing unit 100 may further include a pre-processing unit that converts the medical image data into a combination of a plurality of 2D images when the medical image data is a 3D image.

According to an embodiment, the medical image segmentation system using the convergence deep neural network model is a device capable of performing a function to be described later, and may include, for example, a server computer, a personal computer, and the like. As an embodiment, the medical image segmentation system using the fused deep neural network model may further include one or more processors and/or one or more memories.

The one or more memories may store various data. Data stored in the memory is data acquired, collected, or used by at least one component of a medical image segmentation system using a fusion deep neural network model, and may include software. Memory may also include volatile and/or non-volatile memory.

The one or more memories may store instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation.

One or more processors may drive software to control at least one component of a medical image segmentation system using a fused deep neural network model connected to the processor. In addition, the processor may perform various operations, processing, data generation, processing, etc. related to the present disclosure. In addition, the processor may load data or the like from or store to the memory.

5 is an exemplary diagram showing the performance of a medical image segmentation system and method using a fusion deep neural network model according to an embodiment of the present invention.

Referring to FIG. 5 , it can be seen a qualitative evaluation of the convergence deep neural network model according to an embodiment of the present invention. 2 and 3 were used for the Gaussian number K of the Gaussian mixture model.

(a) of FIG. 5 is input image data, (b) is a control that indicates the part to be segmented in the medical image data, (c) is a medical image segmented using a U-net model (d) is a medical image segmented using a Gaussian mixture model with a Gaussian number of 2, and (e) is a medical image segmented using a Gaussian mixture model with a Gaussian number of 3.

Quantitative evaluation of the qualitative evaluation of FIG. 5 is shown in [Table 1] below. In the quantitative evaluation, the similarity of FIGS. 5(b) and 5(c), (d), and (e) was calculated and expressed as a similarity index (Dice Score). The similarity index may have a value from 0 to 1, and the closer to 1, the more similar.

Model U-net K = 2 K = 3 Dice score 0.712 0.723 0.715

According to [Table 1], the similarity coefficient (Dice score) of the U-net model of the medical image segmentation system and method using the fused deep neural network model according to an embodiment of the present invention is 0.712, and the number of Gaussians (K) is 2 It can be seen that the similarity coefficient of the proposed model is 0.723 in the case of , and the similarity coefficient of the proposed model is 0.715 when the number of Gaussians (K) is 3. And, those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included in the scope of the present invention. do.

100: input unit
110: preprocessor
200: shape feature
300: brightness feature
400: coupling part
500: division

Claims (9)

an input unit for receiving medical image data;
a shape feature for extracting shape feature values of an image from the medical image data through a convolutional neural network-based U-net;
a brightness feature for extracting a brightness feature value of an image from the medical image data through a Gaussian mixture model;
a combining unit for combining the shape feature value and the brightness feature value through a channel-wise method and fusion through a two-dimensional convolutional neural network; and
A medical image segmentation system using a convergence deep neural network model, comprising a segmentation unit for dividing the medical image data by encoding or decoding the medical image data in which the shape feature value and the brightness feature value are fused multiple times.
The method of claim 1,
The input unit,
When the medical image data is a 3D image, the medical image segmentation system using a convergence deep neural network model further comprising a preprocessor for converting the medical image data into a combination of a plurality of 2D images.
The method of claim 1,
The shape characteristic is,
A medical image segmentation system using a fusion deep neural network model, which extracts shape feature values by encoding or decoding the medical image data multiple times.
The method of claim 1,
The brightness feature is
Extracting a plurality of Gaussian distributions for image brightness information from the medical image data, generating a mixed probability model using the plurality of Gaussian distributions, and using the EM algorithm (Expectation-maximixation algorithm) to obtain the mixed probability model A medical image segmentation system using a fusion deep neural network model that learns and estimates the parameters of the mixed probabilistic model.
an image input step of receiving medical image data;
a feature extraction step of extracting a shape feature value of an image through a convolutional neural network-based U-net from the medical image data, and extracting a brightness feature value of the image through a Gaussian mixture model;
a feature combining step of combining the shape feature value and the brightness feature value through channel-wise and fusion through a two-dimensional convolutional neural network; and
A medical image segmentation method using a fusion deep neural network model, comprising an image segmentation step of segmenting the medical image data by encoding or decoding the medical image data in which the shape feature value and the brightness feature value are fused multiple times.
6. The method of claim 5,
In the video input step,
When the medical image data is a 3D image, the medical image segmentation method using a convergence deep neural network model further comprising a pre-processing step of converting the medical image data into a combination of a plurality of 2D images.
6. The method of claim 5,
The feature extraction step is
A medical image segmentation method using a convergence deep neural network model, further comprising a shape feature extraction step of extracting shape feature values by encoding or decoding the medical image data a plurality of times.
6. The method of claim 5,
The feature extraction step is
a Gaussian distribution extraction step of extracting a plurality of Gaussian distributions for image brightness information from the medical image data;
a probabilistic model generating step of generating a mixed probabilistic model using the plurality of Gaussian distributions; and
A medical image segmentation method using a convergence deep neural network model, further comprising a parameter estimation step of learning the mixed probabilistic model using an EM algorithm (Expectation-maximixation algorithm) and estimating the parameters of the mixed probabilistic model.
A computer-readable recording medium in which a program for implementing the method according to any one of claims 5 to 8 is recorded.

Images