KR20220072693A - Brain Neural Network Structure Image Processing System, Brain Neural Network Structure Image Processing Method, and a computer-readable storage medium - Google Patents

Abstract

The present invention relates to a method for processing an image of a neural network structure, and to a system, method, and computer-readable medium for image processing so that a neural network structure is distinguished by using deep learning for an input brain image image. An image processing system of a neural network structure according to an embodiment of the present invention includes: an input unit to which a neural network image image is input by interworking with a neural network image photographing apparatus; An actual label map storage unit and a deep neural network are provided in which an actual measurement label map, which is an actual measurement image of a cranial neural network for use as a control in learning of a cranial neural network, is provided by structure, and the cranial neural network image image input to the input unit is divided by cranial neural network structure. It generates a prediction label map, and includes an image processing unit for image processing while learning by applying a loss function to reduce the deviation from the actual measurement label map for each structure stored in the actual measurement label map storage unit for each structure prediction label map. can

Description

A brain neural network structure image processing system, a brain neural network structure image processing method, and a computer-readable medium recording a computer program for providing the same storage medium}

The present invention relates to an image processing system, method, and computer-readable medium of a neural network structure through deep learning, and in particular, a loss function used for learning to reduce the deviation between a predicted image and an actual image in the process of performing deep learning. It relates to an image processing system, a method, and a computer-readable medium of a neural network structure that allow image processing closer to the actual image based on the adaptive loss function.

As a unit of the nervous system, a neuron that transmits stimuli and excitation generally represents a highly polarized structure of several dendrites and a single axon from the cell body, and the form of these neurons is It is used as an important data for understanding various neuronal properties, including synaptic integration and neuronal excitability. For example, several neurodegenerative diseases such as Alzheimer's disease or Parkinson's disease have been observed through abnormalities in neuronal morphology.

On the other hand, imaging technology through machine learning has developed a lot, and large-scale image data sets based on automation and accurate analysis tools have appeared, and these data sets are being used considerably in the field of biomedical imaging.

At this time, most of the algorithms used rely on supervised learning using clean labels for learning, that is, correct answer labels. It is difficult to actually create an accurate pixel-level actual label from a neuron image with the deep learning algorithm that has been used in the biomedical image area due to the structure that is included and intersects each other.

For this reason, recently, Mean Absolute Error (MAE), Symmetric Cross Entropy Learning (SL), Reverse Cross Entropy (RCE), Normalized Cross Entropy (NCE) ), Active and Passive Loss (APL), and Noise Robust Dice (NR-Dice), which are strong loss functions, were used for error labels (Noisy labels). It is not possible to represent a complex form as a result close to the actual measurement.

The present invention is proposed to solve the above problems, and in image processing for labels with a lot of space in units of pixels and complex in a neural network, the labels are classified according to the structure of the neural network and learned for each label, but the average By mixing Mean Squared Error (MSE) and Adaptive Mean Squared Error (ADMSE), an adaptive loss function that uses spatially various weights to prevent incorrect label learning, it can be applied to more ground-level images. It enables close image processing, and furthermore, it is possible to process images close to the actual image of the neural network by applying the Structure Prior (STPR) loss function, which is a loss function that reduces the loss in merging after learning between each label is completed. It is an object to provide a system, a method and a computer-readable medium.

An image processing system of a cranial neural network structure according to an embodiment of the present invention for solving the above problems includes: an input unit to which a cranial neural network image image is input by interworking with a cranial neural network image photographing apparatus; An actual label map storage unit and a deep neural network are provided in which an actual measurement label map, which is an actual measurement image of a neural network for use as a control for learning of a neural network, is stored for each structure. It generates a prediction label map, and includes an image processing unit that processes the image while learning by applying a loss function to reduce the deviation from the actual measurement label map for each structure stored in the actual measurement label map storage unit for each structure prediction label map. can

Here, the structure of the cranial neural network may include a background of the cranial neural network structure image, a neuronal cell body, dendrites, and axons.

In addition, the application of the loss function applies a mean squared error (MSE) loss function to the prediction label map for the background of the neural network structure image and the neuronal cell body, and predicts the dendrites and axons. The label map may be formed to apply an adaptive mean squared error (ADMSE) loss function in which a weight generated by determining a prediction value in units of pixels to the mean squared error loss function is reflected.

In addition, the image processing unit further applies a structure prior (STPR) loss function that reduces the loss between the predicted label maps divided for each structure of the neural network, and the STPR loss function includes prior information of the neural network structure. Based on this, it can be formed to reduce the loss between the predicted label maps for each structure of the neural network by using the MSE loss function.

Meanwhile, the method for processing an image of a neural network structure according to an embodiment of the present invention includes: inputting a neural network image image to an input unit; transmitting the input neural network image image to an image processing unit; A step of generating a predicted label map for each structure of the neural network by passing the neural network image transmitted to the image processing unit through the deep neural network, and the generated prediction label map for each structure of the neural network by the image processing unit is stored in an actual measurement label map storage unit. It may be configured to include a step of estimating a loss function for each structure by comparing it with a pre-stored actual label map for each structure of the neural network, and learning through the estimated loss function.

Here, the application of the loss function applies a mean squared error (MSE) loss function to the prediction label map for the neural network structure image background and the neuronal cell body, and predicts the dendrites and axons. The label map may be configured to apply an adaptive mean squared error (ADMSE) loss function in which a weight generated by determining a prediction value in units of pixels to the mean squared error loss function is reflected.

In addition, the application of the loss function may be configured to further apply a structure prior (STPR) loss function that reduces the loss between the predicted label maps divided for each structure of the neural network.

The neural network structure image processing system, method, and computer-readable medium according to an embodiment of the present invention, with reference to FIGS. 5 to 7 , as a result, state-of-the-art noise in various segmentation quality metrics by using ADMSE and STPR loss functions It can be confirmed that it outperforms the robust loss function, and in particular, the ClDice score is the best, confirming that it is more effective for the neural network segmentation problem.

1 is a block diagram of a neural network structure image processing system according to an embodiment of the present invention.
2 is a schematic diagram showing a process of a neural network structure image processing method according to an embodiment of the present invention.
3 is a flowchart of an image processing method of a neural network structure according to an embodiment of the present invention.
4 is a diagram illustrating in detail image processing changes in the course of the neural network structure image processing method according to an embodiment of the present invention.
Here, (a) is the input image, (b) is a partial label map, (c) is the label map to which the ADMSE loss function is applied, (d) is the label map to which the STPR loss function is applied, and (e) is the final result label map. .
5A to 5B are diagrams showing comparison results of training images for the neural network structure image processing method using the ADMSE and STPR loss functions and the neural network structure image processing method using other loss functions according to an embodiment of the present invention. .
Here, (a) is an input image, (b) is a partial label map, (c) is a case of applying the MSE loss function, (d) is a case of applying the NCE loss function, (e) is a case of applying the RCE loss function, (f) ) is an NR-Dice loss function application case, (g) is an ADMSE and STPR loss function application case according to an embodiment of the present invention
6A to 6B are diagrams showing comparison results of test images for the neural network structure image processing method according to an embodiment of the present invention using ADMSE and STPR loss functions and the neural network structure image processing method using other loss functions. .
Here, (a) is the input image, (b) is the full label map, (c) is the case of applying the MSE loss function, (d) is the case of applying the NCE loss function, (e) is the case of applying the RCE loss function, (f) is an NR-Dice loss function application case, (g) is an ADMSE and STPR loss function application case according to an embodiment of the present invention
7 shows the IoU, precision, recall, F1 score, and ClDice metric results for the neural network structure image processing method using a different loss function from the neural network structure image processing method according to an embodiment of the present invention using ADMSE and STPR loss functions. This is a comparison table.

Hereinafter, the description of the present invention with reference to the drawings is not limited to specific embodiments, and various modifications may be made and various embodiments may be provided. In addition, it should be understood that the contents described below include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention.

In the following description, terms such as 1st, 2nd, etc. are terms used to describe various components, meanings are not limited thereto, and are used only for the purpose of distinguishing one component from other components.

Like reference numbers used throughout this specification refer to like elements.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as "comprises", "comprises" or "have" described below are intended to designate the existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. It should be construed as not precluding the possibility of addition or existence of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, a neural network structure image processing system, a method, and a computer-readable medium according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a neural network structure image processing system according to an embodiment of the present invention.

Referring to FIG. 1 , a neural network structure image processing system according to an embodiment of the present invention may include an input unit 1 , a measurement label map storage unit 2 , and an image processing unit 3 .

Specifically, the input unit 1 is a place where a neural network image is input, and may be linked with the neural network image photographing apparatus 5 . Here, the neural network imaging apparatus 5 is not limited to any specific form, and various imaging apparatuses capable of imaging a neural network, such as CT or MRI, may be configured.

The actual measurement label map storage unit 2 may store the measurement label map to be used for learning in the image processing unit 3 to be described later. Here, the actual measurement label map is an actual measurement image of the neural network, and as the image processing unit 3 generates the predicted label map for each structure, it may be stored for each structure of the neural network so as to correspond to each predicted label map. The actual label map stored for each structure of the neural network in this way is used as a control for learning the predicted label map.

The image processing unit 3 divides the neural network image input into the input unit 1 according to the structure of the neural network and generates a predicted label map. For this purpose, the image processing unit 3 includes a deep neural network 20 can be

More specifically, the neural network image image input to the input unit 1 may be transmitted to the deep neural network 20 provided in the image processing unit 3 to perform deep learning, and in the deep neural network 20 through deep learning By dividing the neural network into structures, it is possible to generate a predictive label map for each structure.

At this time, the structure of the cranial neural network is a structure including the background of the structural image of the cranial neural network, the neuronal cell body, dendrites, and axons. can do.

In other words, the predicted label map for the neural network structure image background, the predicted label map for the neuron body, the predicted label map for the dendrites, and the predicted label map for the axon can be generated, respectively.

However, the present invention is not necessarily limited thereto, and the neural network structure may be divided into other structures in addition to the above four parts, and may be divided into a more simplified or more subdivided structure than the above structure.

In addition, the image processing unit 3 applies a loss function to reduce the deviation from the actual measurement label map stored in the actual measurement label map storage unit 2 in the predicted label map generated for each structure of the neural network, so as to process the image while learning. can be

More specifically, when the image processing unit 3 first generates the predicted label map for each neural network structure, it compares it with the actual measurement label map stored in the actual measurement label map storage unit 2, and through this process, the loss function is estimated can do. At this time, when the loss function is estimated, the predicted label map generated again becomes closer to the actual measurement label map by applying the loss function by backpropagating it.

If the above process is repeated, the predicted label map generated by the image processing unit 3 is generated as close as possible to the actual measurement label map to be obtained, and through this learning, stronger neural network classification can be made.

Here, the loss function used may include a mean squared error (MSE) loss function and an adaptive mean squared error (ADMSE) loss function, which may be used differently depending on the structure of the neural network. can

Specifically, the MSE loss function can be applied to the prediction label map for the background and neuronal body of the neural network structure image with relatively clear distinction and few gaps between the structures.

In addition, the ADMSE loss function is a loss function in which the weight generated by determining the prediction value in pixel units is reflected in the MSE loss function, and can be applied to the prediction label map for the dendrites and axons that are not clearly distinguished and are entangled relatively.

This, ADMSE loss function is formed to suppress learning in the pixel by considering it as labeled when the predicted value appears higher in unlabeled pixels of the prediction label map for dendrites or axons. can be

일 수 있으며, ADMSE 손실함수에 대한 설명은 후술하는 뇌신경망 구조 영상처리 방법에서 보다 구체적으로 설명하기로 한다.The weight applied here is

, and the description of the ADMSE loss function will be described in more detail in the neural network structure image processing method to be described later.

는

번째 클래스의 예측 레이블 맵인

의

번째 픽셀 값을 의미하며,

는 손실함수에 적용되는 가중치의 정도를 제어하는 사용자 정의 매개 변수이고,

는

를 수정하기 위한 수렴 전의 학습 초기 단계에서 안정적인 수렴을 보장하기 위한 손실 함수에 추가된 상수 값이다.here,

Is

The predicted label map of the th class

of

means the second pixel value,

is a user-defined parameter that controls the degree of weight applied to the loss function,

Is

It is a constant value added to the loss function to ensure stable convergence at the initial stage of learning before convergence to correct for .

Meanwhile, the image processing unit 3 may use the STPR loss function in addition to the MSE loss function and the ADMSE loss function described above.

The STPR loss function is a loss function used to reduce the loss between predicted label maps generated when the neural network structure imaging system of the present invention generates a predicted label map for each neural network structure.

Here, the STPR loss function can be formed to reduce the loss between the predicted label maps for each structure to the MSE loss function based on the prior information of the neural network structure. can be avoided to represent a more accurate cranial neural network structure.

Such a description of the STPR loss function will also be described in more detail in the neural network structure image processing method to be described later.

The neural network structure image processing system according to the embodiment of the present invention described above applies the MSE loss function, the ADMSE loss function, and the STPR loss function while performing prediction and learning by dividing by the neural network structure, each generated by the neural network structure. By learning the predicted label map to reduce the deviation from the actual label, on the other hand, by reducing the loss between the predicted label maps for each segmented neural network structure, accurate image processing is possible compared to the neural network structure image processing system that applies other loss functions. can

2 is a schematic diagram showing a process of a neural network structure image processing method according to an embodiment of the present invention, FIG. 3 is a flowchart of a neural network structure image processing method according to an embodiment of the present invention, and FIG. 4 is an embodiment of the present invention It is a diagram showing in detail the image processing change in the process of the image processing method of the neural network structure according to FIG. Here, (a) is the input image, (b) is a partial label map, (c) is a label map to which the ADMSE loss function is applied, (d) is a label map to which the STPR loss function is applied, and (e) is the final result label map. .

로부터 쌍을 이루는 랜덤 변수 (X, Y)를 사용하여

로 매개 변수화된 심층 신경망

에, 샘플 이미지 X ~

(여기서,

는 관찰된 이미지의 분포)를 입력하여 뇌신경망 구조를 나타내는 모든 클래스에 대한 응답 맵 Y ~

(여기서,

는 뇌신경망 구조 레이블의 분포)를 예측하도록 영상처리 하는 방법에 관한 것이다. 2 to 4 , as an image processing method using deep learning, a neural network structure image processing method according to an embodiment of the present invention, distribution

Using paired random variables (X, Y) from

A deep neural network parameterized by

In, sample image X~

(here,

is the response map Y for all classes representing the neural network structure by inputting the distribution of the observed images.

(here,

relates to an image processing method to predict the distribution of the neural network structure label).

In this process, the present invention predicts the response map closest to the actual image while learning using a mean squared error (MSE) loss function and an adaptive mean squared error (ADMSE) loss function. image processing can be performed to

Here, the adaptive mean squared error (ADMSE) loss function is a loss function obtained by applying a weight generated by determining the predicted value in units of pixels to the mean squared error (MSE) loss function. A more detailed description will be given later. do it with

Looking more specifically at the present invention, in the method for processing an image of a neural network structure according to an embodiment of the present invention, a step (S10) of inputting a neural network image to the input unit 1, the inputted image of a neural network image is an image A step of transmitting to the processing unit 3 (S20), a step of generating a predicted label map for each structure of the cranial neural network by passing the cranial neural network image image transmitted to the image processing unit 3 through the deep neural network 20 (S30) and The loss function is estimated for each structure by comparing the predicted label map for each neural network structure generated in the image processing unit 3 with the actually measured label map for each neural network structure stored in advance in the measured label map storage unit 2, and the estimated It may be configured to include a step (S40) of learning through the loss function.

That is, the present invention may be configured to generate a prediction label map for each structure of the neural network based on the neural network image input to the deep neural network in the image processing process through deep learning.

Here, since the structure of the cranial neural network includes the background, the cell body, the dendrite, and the axon of the image of the cranial neural network, prediction for each of the four structures It is configured to be able to create a label map.

In this case, an MSE loss function or an ADMSE loss function may be applied to each prediction label map, respectively.

Specifically, the MSE loss function is used for the prediction label map of the neural cell body and the background of the neural network structure image, where the structure is relatively clear. It can be used for the prediction label map for

를 레이블 인덱스 세트로 정의하면

으로 정의되고, 비교적 구조가 명확한 뇌신경망 구조 이미지의 배경과 신경세포체 몸체에 대한 예측 레이블 맵 세트는

, 구조가 명확하지 않은 수상돌기 및 축삭돌기에 대한 예측 레이블 맵 세트는

,

로 정의될 수 있는 것이다.That is, mark each prediction label map sequentially as 0, 1, 2, 3,

If we define as a set of label indices,

The set of predicted label maps for the background and neuronal cell body of the neural network structure image, which is defined as

, a set of predictive label maps for dendrites and axons with unclear structures

,

can be defined as

However, this is only a preferred example and is not necessarily limited, and the definition of a set for the prediction label map may vary according to a set criterion. For example, the nerve cell body may be combined with dendrites or axons, and the background of the neural network structure image and dendrites or axons may be combined with each other.

를 생성할 수 있다.On the other hand, as described above, when a prediction label map is given for each neural network structure, that is, a partial prediction label map cannot be treated as a background image for all unlabeled pixels. Because of this, the background image is a label for the background image by applying intensity-based thresholding to the input image.

can create

The ADMSE loss function, which is a loss function applied to predictive label maps such as complex dendrites or axons, will be discussed below.

First, the MSE loss function (or mean square error), which is the basis of the ADMSE loss function, is widely used to minimize the distance between the predicted label map and the measured label map, and the formula is as follows [Equation 1].

[Equation 1]

는

로 매개 변수화된 심층 신경망이며, X는 입력되는 이미지이고, Y는 실측 레이블 맵이다.here,

Is

It is a deep neural network parameterized by , where X is the input image, and Y is the ground truth label map.

의 의미에서 큰 오류에 효과적으로 패널티를 주지만 본 발명이 해결하려는 문제에서와 같이 훈련 데이터의 복잡성을 이루는 노이즈(noise)에 대해서는 탄력적이지 않다. The MSE loss as above is

Although it effectively penalizes a large error in the sense of , it is not resilient to noise constituting the complexity of the training data as in the problem to be solved by the present invention.

For this reason, in the present invention, using the ADMSE loss function to which an adaptive weight for controlling pixel-level backpropagation is applied to the MSE loss function, the learning of the prediction label map for structurally complex dendrites and axons can be proceeded. .

를

번째 클래스의 레이블 맵이고,

를

의

번째 픽셀 값이라 하면, 픽셀별 적응 가중치 함수

(·,·)를 사용하는

는 하기 [수학식 2]와 같이 정의될 수 있다.Specifically,

cast

is the label map of the th class,

cast

of

If the th pixel value is the pixel-specific adaptive weight function

using (·,·)

may be defined as in [Equation 2] below.

[Equation 2]

At this time, the ADMSE loss function is an unlabeled pixel in the prediction label map for structurally complex dendrites and axons as shown in Equation 3 below, and when the predicted value is high, since it is a non-labeled pixel, during back propagation By assigning a weight, learning in the corresponding pixel can be suppressed.

[Equation 3]

는 손실함수에 적용되는 가중치의 정도를 제어하는 사용자 정의 매개 변수이고,

는

를 수정하기 위한 수렴 전의 학습 초기 단계에서 안정적인 수렴을 보장하기 위한 손실 함수에 추가된 상수 값이다.here

is a user-defined parameter that controls the degree of weight applied to the loss function,

Is

It is a constant value added to the loss function to ensure stable convergence at the initial stage of learning before convergence to correct for .

That is, if there is no label in the pixel as described above, it is learned using the ADMSE loss function to which a low weight is applied according to the prediction value, and if there is a label, it is learned using the MSE loss function, so that an image can be processed accurately even with a complex label.

On the other hand, the present invention further comprises the step (S45) of applying a structure prior (STPR) loss function that reduces the loss between each prediction label map after the neural network image image is generated as a prediction label map for each structure. can be

Here, the step (S45) of applying the pre-structural loss function may preferably proceed simultaneously with the step S40, but may be applied before the step S40 or after the step S40.

That is, in the neural network structure image processing method according to an embodiment of the present invention, in addition to the ADMSE loss function that reduces the loss between the predicted label map and the actual label map, the structure prior (STPR) loss that reduces the loss between the predicted label maps for each structure Functions can be used to more accurately image the neural network structure.

To this end, in the present invention, the prior information of the neural structure can be stored first, and the prediction label map for each structure of the neural network is set as the first class prediction label map that is the basis of the neural network structure, that is, the background of the image of the neural network structure. ) can be predicted and extracted from the prediction label map for

)으로부터, 두번째(

)와, 세번째 또는 네번째(

)를 예측하여 추출하는 것이다.That is, the first class prediction label map (

) from the second (

) and the third or fourth (

) to predict and extract.

At this time, as shown in the figure, the loss between maps can be reduced through the STPR loss function, which reduces the loss by using the MSE loss function between the predicted label maps, which is based on the pre-stored neural structure information. can reduce

This applies to the condition that only one correct class should be assigned for a given pixel location, for example, since an axon and a dendrite cannot belong to a pixel at the same time. have.

In other words, the STPR loss function can be constrained so that the pixel is not classified as a neuronal body or an axon when the probability that the pixel is a background or dendrite is predicted to be high. We can learn to normalize between predicted values so that they do not overlap each other while generating full labels for each.

Such an STPR loss function can be defined by Equation 4 below, and the present invention utilizes the above-described ADMSE loss function and STPR loss function and performs high-accuracy prediction to perform image processing as close to the actual image as possible. have.

[Equation 4]

는 샘플이미지 이고,

이다.here,

is the sample image,

to be.

…

중 하나만 가질 수 있도록 정의되어, 각 픽셀은 정확히 하나의 레이블만을 가질 수 있음을 의미한다.In Equation 4, one pixel cannot be a dendrite and an axon at the same time.

…

It is defined to have only one of these, meaning that each pixel can have exactly one label.

In addition, the present invention can be implemented by storing computer-readable codes in a computer-readable storage medium. The computer-readable storage medium includes any type of storage device in which data readable by a computer system is stored.

The computer-readable code is configured to perform the steps of implementing the image processing method according to the present invention when the image is processed by the image processing unit 3 from the input unit 1 from the computer-readable storage medium. do. The computer readable code may be implemented in various programming languages. And functional programs, codes, and code segments for implementing the present invention can be easily programmed by those skilled in the art to which the present invention pertains.

Examples of the computer-readable storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and includes implementation in the form of a carrier wave (eg, transmission over the Internet). In addition, the computer readable storage medium may be distributed in networked computer systems, so that the computer readable code may be stored and executed in a distributed manner.

Hereinafter, the following experimental examples are presented to explain in more detail the brain neural network structure image processing system and method of the present invention, but the following experimental examples are only illustrative to help the understanding of the present invention, It is not limited to the content.

[Experimental example]

[Experimental conditions]

The neuron image data used in this experiment consisted of paired training data with partial annotations from a total of 23 single cells and the sum of three fully labeled test sets. In addition, each neuron image has a size of 1024 X 1024 which is divided into small portions of 128 X 128 size, and the completely empty background portion is discarded and data augmentation is applied using rotation and flipping. did.

Here, since the number of pixel labels in dendrites was about five times greater than the number of pixel labels in axons, oversampling was applied to the labels of axons to prevent class imbalance problems.

The network proposed in this experiment is based on a conventional U-Net using a ResNet-34 backbone encoder, and the model is trained using an Adam Optimizer with an initial learning rate of 3e-3 and a random batch size of 300, followed by cosine annealing ( using the cosine annealing method, which was gradually reduced to a learning rate of 3e-4.

를 생성하였으며, 손실함수에 적용되는 가중치의 정도를 제어하는 사용자 정의 매개 변수인

는 0.01을 구조적으로 복잡한 레이블에 적용하고,

를 수정하기 위한 수렴 전의 학습 초기 단계에서 안정적인 수렴을 보장하기 위한 손실 함수에 추가된 상수 값인

는 0.03을 적용하여, 초기 학습에서 수렴시켰으며, 10 겹의 교차 검증을 통해 실험을 수행하였다.Also, for the input image, we set the threshold value to 0.3 so that the background image

is a user-defined parameter that controls the degree of weight applied to the loss function.

applies 0.01 to structurally complex labels,

A constant value added to the loss function to ensure stable convergence at the initial stage of learning before convergence to correct

was converged in the initial learning by applying 0.03, and the experiment was performed through 10-fold cross-validation.

[Experimental evaluation]

To prove the efficacy of partial labels, the neural network structure was compared with the MSE loss function, NCE loss function, APL loss function, RCE loss function, and NR-Dice loss function by setting the image processing method as a control group, respectively.

To quantitatively evaluate the performance of each method, we used three error metrics commonly used in image segmentation: IoU (Intersection over Union), F1 score, and ClDice.

[Results and Discussion]

The comparison results for the image processing method of the neural network structure according to the application of various loss functions are shown in FIGS. 5 and 6 . Figure 5 shows the results of the training image, Figure 6 shows the results of the test image.

As can be seen in FIGS. 5 and 6 , the MSE loss function produced incomplete results, and the other loss functions showed slight improvement compared to the MSE loss function, but did not produce accurate results as in the image processing method of the present invention.

In addition, other loss functions did not learn about axons during learning due to partial labels, but the loss functions used in the present invention correctly reconstructed most of the axons. This is thought to be because other loss functions act as negative labels for unlabeled pixels.

Table 1 shown in FIG. 7 is a table showing the segmentation accuracy measured using the IoU, Precision, Recall, F1 score, and ClDice metrics compared with other controls. The last three rows of this table show the study results of the loss function proposed in the present invention.

From these results, it can be seen that axons show extremely low recall values for MSE and noise-robust loss functions than other classes, with 0.0954 and 0.1676.

However, it can be seen that the ADMSE and STPR loss functions are more resilient to label noise and achieve a maximum recall of 0.5731.

In addition, the image processing method according to an embodiment of the present invention performs much better than other methods in which the ClDice metric for evaluating the linearity of a structure is superior to other methods, and through this, loss functions having different ADMSE and STPR loss functions applied according to an embodiment of the present invention It shows that it is a better fit for neural network segmentation.

As a result, it can be confirmed that the neural network structure image processing system and method according to an embodiment of the present invention using the ADMSE and STPR loss functions outperform the state-of-the-art noise robust loss function in various segmentation quality metrics, and in particular, the ClDice score It was the best, and it was confirmed that it was more effective in the problem of splitting the neural network.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can practice the present invention in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Accordingly, the embodiments described above are illustrative in all respects and not restrictive.

1: input
2: Actual measurement label map storage unit
3: image processing unit
5: video recording device
10: input image
20: Deep Neural Network
30: Prediction Label Map
30a: Background Prediction Label Map
30b: Neuronal body prediction label map
30c: dendrite prediction label map
30d: Axon prediction label map
40: ground truth label map
40a: background ground truth label map
40b: Neuron cell body actual measurement label map
40c: dendrite measurement label map
40d: axon measurement label map
A1 : MSE loss function
A2 : ADMSE loss function
A3: SPTR loss function
C: neuronal cell body
D: dendrite
A: axon

Claims (10)

an input unit that is interlocked with a neural network imaging apparatus to input a neural network image;
An actual measurement label map storage unit in which an actual measurement label map, which is an actual measurement image of a neural network for use as a control for learning of a neural network, is stored for each structure; and
A deep neural network is provided to generate a predicted label map by dividing the neural network image image input to the input unit for each structure of the neural network, and the predicted label map for each structure is stored in the actual label map storage unit for each structure. A neural network structure image processing system comprising an image processing unit that processes an image while learning by applying a loss function to reduce deviation.
The method of claim 1,
The structure of the cranial neural network is,
A cranial neural network structure image processing system including a background of the neural network structure image, a nerve cell body, dendrites, and axons.
3. The method of claim 2,
The application of the loss function is,
A mean squared error (MSE) loss function is applied to the predicted label map for the neural network structure image background and the neuronal body body,
In the prediction label map for the dendrites and axons, the adaptive mean squared error (ADMSE) loss function in which the weight generated by determining the prediction value in units of pixels is reflected in the mean squared error loss function is applied. Brain neural network structure image processing system, characterized in that.
The method of claim 1,
The image processing unit,
The neural network structure image processing system, characterized in that further applying a structure prior (STPR) loss function that reduces the loss between the predicted label maps divided for each structure of the neural network.
5. The method of claim 4,
The STPR loss function is
A neural network structure image processing system, characterized in that it reduces the loss between the predicted label maps for each structure of the neural network based on the prior information of the neural network structure by using the MSE loss function.
A cranial neural network image image is input to the input unit;
transmitting the input neural network image image to an image processing unit;
generating a predicted label map for each structure of the neural network by passing the neural network image transmitted to the image processing unit through the deep neural network; and
By comparing the predicted label map for each structure of the neural network generated in the image processing unit with the actual label map for each structure of the neural network stored in the actual label map storage unit, the loss function is estimated for each structure, and the estimated loss function is applied to learn A brain neural network structure image processing method comprising the step of.
7. The method of claim 6,
The structure of the cranial neural network is,
A neural network structure image processing method comprising the background of the image of the neural network structure, a neuron cell body, dendrites, and axons;
8. The method of claim 7,
The application of the loss function is,
A mean squared error (MSE) loss function is applied to the predicted label map for the neural network structure image background and the neuronal body body,
In the prediction label map for the dendrites and axons, the adaptive mean squared error (ADMSE) loss function in which the weight generated by determining the prediction value in units of pixels is reflected in the mean squared error loss function is applied. Brain neural network structure image processing method, characterized in that.
8. The method of claim 7,
The application of the loss function is,
A neural network structure image processing method, characterized in that further applying a structure prior (STPR) loss function that reduces the loss between the predicted label maps divided for each structure of the neural network.
A computer-readable recording medium on which a computer program is recorded, which provides a method for processing an image of a neural network structure according to any one of claims 1 to 9.

Images