KR102180569B1 - Apparatus for Enhancing Contrast and Resolution in a Grating Interferometer by Machine Learning - Google Patents

Abstract

The present invention relates to an apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning to improve both the sensitivity and spatial resolution of an image in a lattice interferometer through machine learning. A lattice interferometer image acquisition unit that obtains a high-resolution image and a high-sensitivity image by linearly moving the position of the sample; a numerical phantom generator that generates a numerical phantom for performing machine learning; from the input data by performing an operation process of a convolutional neural network Convolutional layer generator for extracting features; ReLu (Rectified linear unit) activation function applied to the output value of the convolution operation to perform smoothly iterative machine learning; The forward propagation and backward propagation processes repeatedly It includes a CNN iterative machine learning unit for modifying the convolution operator while performing; an image matching output unit for matching and outputting features extracted by the iterative machine learning of the CNN iteration machine learning unit.

Description

Device for enhancing sensitivity and resolution in a grating interferometer through machine learning {Apparatus for Enhancing Contrast and Resolution in a Grating Interferometer by Machine Learning}

The present invention relates to a lattice interferometer, and specifically, an apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning to improve both the sensitivity and spatial resolution of an image in the lattice interferometer through machine learning. It is about.

X-rays and neutrons, which are a type of radiation, are widely used in non-destructive testing because they can see the inside of an object without damaging it.

The same principle is used in that X-rays and neutrons transmit through an object and react to the nuclei of electrons and atoms, respectively, but are imaged and inspected through the energy difference that decreases through the reaction.

In general, radiographic images used in non-destructive testing are absorption imgaing and show the absoprtion contrast according to the inherent linear attenuation coefficient of an object.

Radiation imaging has developed through various applications as science advances.

Among them, there are phase contrast imaging and dark field imaging using a grating interferometer applying the principle of interference to radiation.

As shown in FIG. 1, a general X-ray grating interferometer uses three gratings: a ray remote grating G0, a phase grating G1, and an absorption grating G2.

Grids made with fine patterns at intervals of several µm have their respective roles to acquire phase information of the subject.

The ray remote grid G0 improves the spatial coherence of X-rays, and the X-rays transmitted through the G0 grid pass through the phase grid G1 and generate an interference pattern due to the wave nature of the X-rays.

The intensity of the interference pattern changes while having a specific period. If the absorption grating G2 is placed at a specific location according to the period and the phase stepping process of the grating is additionally performed, the phase information △Φ of the subject can be obtained from a general detector. I can.

Since the ΔΦ value obtained in this way is more sensitive than the linear attenuation coefficient μ obtained in the conventional X-ray image, the contrast between substances can be significantly improved.

In order to obtain an image with high resolution (high sensitivity) in such an X-ray grating interferometer, the location of the grating and the subject is very important.

에 따른 영상의 민감도 관계를 나타낸 것이다.2 is the location of the subject

It shows the relationship of the sensitivity of the image according to.

As shown in FIG. 2, when the subject is positioned immediately in front of or behind the phase grid G1, an image with the highest sensitivity can be obtained, and the sensitivity decreases linearly as the object moves away from the phase grid G1.

는 격자의 패턴주기, 도금깊이 및 엑스선 에너지 등에 의해 결정되기 때문에 피사체의 위치에 따른 공간해상도를 추가적으로 고려할 수 없다. Where, distance

Since is determined by the grid pattern period, plating depth, and X-ray energy, spatial resolution according to the location of the subject cannot be additionally considered.

3A and 3B show the relationship between spatial resolution according to the position of the subject.

As the subject is positioned closer to the X-ray source, the magnification increases but the sharpness decreases. As the subject increases, the magnification decreases but the sharpness increases.

Through this relationship, the maximum resolution section of each system can be determined.

As such, since the factors that determine the sensitivity of the grating interferometer and the resolution of the X-ray image are independent of each other, it is virtually impossible to determine a physical section in which high sensitivity and high resolution of a subject can be obtained.

As one of the prior art, there is a technology related to a deep learning system using image patterning based on a convolutional neural network based on a convolutional neural network (CNN) that learns an input image and a pattern image. No. 10-2017-0096298)

The other is a technology related to a surface image processing method based on a convolutional neural network that performs convolutional filtering using a predefined spatial filter (Korean Patent No. 10-1804840).

Another option is to extract the lesion feature expression of each of the plurality of cross-sectional images according to the depth direction of the plurality of cross-sectional images by using a recurrent neural network based on the representation of the lesion spatial feature of each of the plurality of cross-sectional images. (Republic of Korea Patent Publication No. 10-2018-0021635)

However, with such conventional techniques, it is practically difficult to simultaneously obtain high sensitivity and high resolution of a subject in a grating interferometer.

Therefore, development of a new technology capable of improving both the sensitivity and spatial resolution of an image in a grating interferometer is required.

Republic of Korea Patent Publication No. 10-2017-0096298 Korean Patent Registration No. 10-1804840 Republic of Korea Patent Publication No. 10-2018-0021635

The present invention is to solve the problem of the image processing technology in the conventional lattice interferometer, and machine learning in which both the sensitivity and spatial resolution of the image in the lattice interferometer can be improved through machine learning. It is an object of the present invention to provide an apparatus for improving sensitivity and resolution in a grating interferometer.

The present invention uses a convolutional neural network (CNN) to extract image information with high sensitivity and high resolution into images at various locations of a subject in a grid interferometer system. An object thereof is to provide an apparatus for improving sensitivity and resolution.

The present invention provides an apparatus for improving the sensitivity and resolution of a phase difference image in a lattice interferometer through machine learning to provide a high-quality medical image by improving the sensitivity and resolution of a phase difference image using a convolutional neural network. have.

The present invention enables the sensitivity and resolution of phase difference images to be improved using a convolutional neural network to increase the applicability to other medical imaging systems beyond the X-ray and neutron imaging fields, and to pioneer a new medical device market. Its purpose is to provide a device for improving sensitivity and resolution in a grating interferometer through learning.

Objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The apparatus for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention to achieve this object is a grating interferometer image that obtains a high-resolution image and a high-sensitivity image by linearly moving the position of a sample from a symmetrical grating interferometer. Acquisition unit; Numerical phantom generation unit for generating a numerical phantom for performing machine learning; Convolutional layer generation unit for extracting features from input data by performing a computational process of a convolutional neural network; ReLu (rectified linear unit) activation function Activation function applying operation unit that can perform seamless iterative machine learning by applying to the output value of the convolution operation; CNN iterative machine learning unit that modifies the convolution operator while repeatedly performing forward propagation and backward propagation processes; CNN iterative machine learning And an image matching output unit for matching and outputting features extracted by negative repetition machine learning.

Here, the grating interferometer image acquisition unit acquires a pair of high-sensitivity and high-resolution images by installing a symmetrical grating interferometer to linearly move the location of the subject, and performs an image size rearrangement process to match different magnifications of each image. Characterized in that.

In addition, the numerical phantom generator measures the resolution, sensitivity, and image noise of the acquired phase difference image and implements an image of the same level as a numerical simulation, and generates a plurality of numerical phantoms for high accuracy. do.

In addition, the convolutional layer generator generates a filter, which is a convolution operation factor, in order to convolution the input data, and the width and height of the filter are smaller than the input data and the number of filters is 32 or more. It is characterized by being able to.

으로 이루어지고, 여기서, (C_o,R_o)는 출력데이터의 크기(C=column, R=row), (C_i,R_i)는 입력데이터의 크기, S는 스트라이드, P는 zero-padding, 그리고 (C_f,R_f)는 필터의 크기인 것을 특징으로 한다.And in the convolution operation of the input data and the filter in the convolutional layer generator, the result of the convolution operation is zero padding, the stride, and the generated filter size to keep the horizontal and vertical dimensions of the input image the same. And the size of the output data is,

Where (C _o ,R _o ) is the size of the output data (C=column, R=row), (C _i ,R _i ) is the size of the input data, S is the stride, and P is the zero-padding , And (C _f ,R _f ) are the size of the filter.

And in the convolution operation of the input data and the filter in the convolutional layer generator, zero padding is performed to solve the problem that edge information of the input data disappears, and the convolution operation is the movement interval when the filter is moved within the input data. It is characterized in that the size of the output data is set to the same size as the input value by setting is 1 (stride=1).

In addition, the activation function application operation unit is characterized in that the ReLu function is applied to an output value calculated from an operation of each convolutional layer, thereby reducing an error that may occur during the iterative machine learning performed thereafter.

And in the CNN iterative machine learning unit, forward propagation is to finally calculate the loss value of the cost function through the convolution operation between the input data generated by the simulation and the convolution operator, and the back propagation is the loss value. In order to minimize the value, weight and bias are corrected by partial differentiation of the convolution operator in the reverse direction of the forward propagation operation.

In addition, the CNN iterative machine learning unit generates a cost function that reduces the difference between the result of the convolution operation performed on the input image and the ground truth through the mean square method, and forward propagation in the direction in which the generated cost function decreases. And the backward propagation process is repeatedly performed.

The apparatus for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention has the following effects.

First, it is possible to improve both the sensitivity and spatial resolution of the image in the lattice interferometer through machine learning.

Second, by using a convolutional neural network (CNN), it is possible to extract image information with high sensitivity and high resolution from images at various locations of a subject in a grid interferometer system.

Third, high quality medical images can be provided by improving the sensitivity and resolution of the phase difference image using a convolutional neural network.

Fourth, the sensitivity and resolution of the phase difference image can be improved using a convolutional neural network to increase the applicability to other medical imaging systems beyond X-ray and neutron imaging fields, and to pioneer a new medical device market.

에 따른 영상의 민감도 관계를 나타낸 구성도
도 3a와 도 3b는 피사체의 위치에 따른 공간해상도의 관계를 나타낸 구성도
도 4는 엑스선 격자간섭계에서 민감도와 해상도의 관계를 나타낸 구성도
도 5는 본 발명에 따른 기계 학습을 통한 격자간섭계에서의 민감도 및 해상도 향상을 위한 장치의 구성도
도 6은 합성곱 신경망(CNN)의 동작 메커니즘을 나타낸 구성도
도 7은 합성곱 연산과정을 나타낸 구성도
도 8a와 도 8b는 반복 기계학습을 위한 forward propagation 및 backward propagation 연산과정을 나타낸 구성도
도 9는 본 발명에 따른 기계 학습을 통한 격자간섭계에서의 민감도 및 해상도 향상을 위한 방법을 나타낸 플로우 차트
도 10a와 도 10b는 본 발명에 따른 본 발명에 따른 기계 학습을 통한 격자간섭계에서의 민감도 및 해상도 향상을 위한 장치의 시뮬레이션 결과를 나타낸 구성도1 is a configuration diagram of a general X-ray grating interferometer
2 is the location of the subject

Diagram showing the relationship of sensitivity of images according to
3A and 3B are configuration diagrams showing the relationship between spatial resolution according to the location of a subject
4 is a block diagram showing a relationship between sensitivity and resolution in an X-ray grating interferometer
5 is a block diagram of an apparatus for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention
6 is a block diagram showing an operation mechanism of a convolutional neural network (CNN)
7 is a block diagram showing a convolutional operation process
8A and 8B are configuration diagrams showing a forward propagation and backward propagation calculation process for iterative machine learning
9 is a flow chart showing a method for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention.
10A and 10B are configuration diagrams showing simulation results of a device for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention according to the present invention.

Hereinafter, a preferred embodiment of an apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention will be described in detail.

Features and advantages of an apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention will become apparent through detailed description of each embodiment below.

2 and 4 are configuration diagrams showing a relationship between sensitivity and resolution in an X-ray grating interferometer, and FIG. 5 is a configuration diagram of an apparatus for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention.

The apparatus and method for improving the sensitivity and resolution in a grating interferometer through machine learning according to the present invention acquires both a high-sensitivity image and a high-resolution image as shown in FIG. 2 from a symmetric grating interferometer, and machine learning. Through this, both the reduced sensitivity and resolution can be improved.

Machine learning is a technology in which a person provides a learning model and a computer synthesizes the features of an image based on accumulated data to derive an answer.

In the present invention, a convolutional neural network (CNN), which is a kind of artificial neural network among several machine learning algorithms, can be applied, but is not limited thereto.

CNN is being studied in various image processing and computer vision fields such as understanding images and extracting high-level abstracted information from them or drawing pictures with new textures.

Based on the ability to extract abstract information, the grid interferometer system is intended to extract image information with high sensitivity and high resolution from images at various locations of the subject.

4 is a block diagram showing a relationship between sensitivity and resolution in an X-ray grating interferometer.

The operation mechanism of the convolutional neural network applied to the present invention is as shown in FIG. 5.

The input data used for machine learning is arranged in the form of a three-dimensional matrix (the number of images per image horizontal, vertical image, and subject position).

This 3D matrix performs a convolution operation with a convolution operator, and then extracts a nonlinear relationship between the result of the operation by applying a ReLU (rectifier linear unit) function.

In the convolutional process, 3D values corresponding to the number of images for each location of the subject are integrated to extract all high-sensitivity and high-resolution information, and the depth is set to 1.

In addition, a convolutional layer is created in a direction in which the horizontal and vertical dimensions of the image are kept constant.

A convolutional neural network is created by repeatedly performing a convolution operation in a convolutional layer using a numerical phantom implemented by simulation as input data.

The convolutional neural network generated through iterative machine learning extracts the features of the phase difference image for each position of the subject acquired from the lattice interferometer to generate a result image with high sensitivity and high resolution.

Specifically, the apparatus for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention includes a grating interferometer image acquisition unit 10 that acquires an image for each position of a subject from the grating interferometer, as shown in FIG. A numerical phantom generator 20 that additionally generates a numerical phantom similar to the actual experimental image, a convolutional layer generator 30 for constructing a convolutional neural network, and a ReLu (rectified linear unit) activation function are synthesized. CNN repetition to complete the convolutional neural network by repeatedly performing the forward propagation and backward propagation processes of the activation function application operation unit 40 that applies to the product operation result and performs efficient iterative learning, and the generated convolutional layer and activation function application unit It includes a machine learning unit 50 and an image matching output unit 60 for matching and outputting features extracted by the iterative machine learning of the CNN repetition machine learning unit 50.

Here, forward propagation is a convolution operation by constructing a convolutional neural network in a direction in which the size of the input image generated through numerical simulation is maintained, and the result value generated through several convolution operations. Are a process for calculating a loss value after a process of comparing through a reference value (ground-truth) and a mean squared error method.

In addition, back propagation is the correction of weights and biases, which are convolutional factors, by performing the operation direction in reverse to reduce the calculated error values. Ideal weights and biases are achieved by repeating the forward propagation and back propagation processes several times. It is to be able to reason.

In the apparatus for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention having such a configuration, a technique for inducing a high-sensitivity and high-resolution image by applying a machine learning algorithm to an X-ray grating interferometer is divided into stages. Specifically, it is as follows.

6 is a block diagram showing an operation mechanism of a convolutional neural network (CNN), and FIG. 7 is a block diagram showing a convolutional operation process.

First, the grating interferometer image acquisition unit 10 acquires high-resolution and low-sensitivity images in which the location of the subject is located close to the detector from the symmetrical grating interferometer. This is how to acquire an image.

The numerical phantom generator 20, which measures the sensitivity, image noise, and resolution information of each image from the acquired image, and implements this information as a numerical simulation, includes images and reference images according to sensitivity and resolution (the shape of the subject). It includes the process of creating a sample, ground-truth) that knows information and sensitivity information.

For more accurate machine learning, tens of thousands of samples of various types are generated.

In addition, the convolutional layer generation unit 30 generates several filters to convolution the input data, and the horizontal and vertical sizes of the filters generated in one convolutional layer are smaller than the input data. Make sure that the number of is 32 or more.

The number of generated convolutional layers is 3 or more, and the accuracy of machine learning increases as the number of generated convolutional layers increases.

7 schematically shows a process for a convolution operation between input data and a filter.

In general, since the operation of the convolutional neural network is to reduce the size of the input data, if zero padding of FIG. 7 is not performed, edge information of the input data disappears, so zero padding is performed.

In the convolution operation, when the filter is moved within the input data, the movement interval is set to 1 (stride = 1), and the size of the output data is adjusted to the same size as the input value.

The size of the output data can be calculated through Equation 1.

Here, (C _o ,R _o ) is the size of the output data (C=column, R=row), (C _i ,R _i ) is the size of the input data, S is the stride, P is the zero-padding, and (C _f , R _f ) is the size of the filter.

In the convolution operation for image matching, since the size of the input data must be kept the same during the operation process, appropriate zero padding, stride, and filter sizes are set through Equation 1.

In addition, the activation function application operation unit 40 performs a ReLu (rectified linear unit) activation function application operation. If the result value of the convolution operation is greater than 0 according to the definition of the Relu function, the result value is output as it is, and if it is less than 0, the result value is 0. Prints.

8A shows a forward propagation calculation equation for CNN iterative machine learning, and finally a loss value c is calculated.

8B shows a backward propagation calculation formula for CNN iterative machine learning, and shows a process of calculating a gradient by taking a partial derivative to a calculation formula of forward propagation in order to correct the weight, which is a convolution operation factor.

In addition, the CNN iterative machine learning unit 50 minimizes the loss value of the cost function calculated through the mean squared error method through repetitive forward propagation and backward propagation calculation processes.

The CNN neural network that can match the high-resolution and high-sensitivity images of the interfering system images is constructed through the CNN repetition machine learning unit 50, and the actual medical images acquired from the interfering system image acquisition unit 10 are applied to the CNN neural network You can acquire the high-resolution, high-sensitivity image you want to finally get.

A method for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention will be described in detail as follows.

9 is a flow chart showing a method for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention.

In the method for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention, a high-resolution image and a high-sensitivity image are obtained by linearly moving the position of a sample from a symmetrical grating interferometer as shown in FIG. 9 (S901).

A similar numerical phantom is generated through simulation by measuring the resolution, sensitivity, and image noise of the image through the phase difference image acquired in (S901). (S902)

In addition, a convolution layer is created by generating a filter composed of various weights and biases in order to convolution the input data.

During the convolution operation, the filter size, zero padding, and stride values are appropriately set so that the horizontal and vertical sizes of the input data are output identically (S903).

And by applying the ReLu function, the result of the convolution operation of the input data is activated (S904).

Forward propagation and backward propagation operations are repeatedly performed through the successive arrangement of multiple convolutional layers and ReLu activation functions to correct weight and bias, which are convolutional factors (S905).

Subsequently, the convolutional neural network is completed by repeating the convolutional operation, and the phase difference image obtained in (S901) is applied to the convolutional neural network, thereby finally outputting a high-sensitivity, high-resolution image (S906).

The apparatus and method for improving sensitivity and resolution in a grating interferometer through machine learning according to the present invention described above include a method of obtaining a high-sensitivity image and a high-resolution image by linearly moving the position of a subject from a symmetric grating interferometer, and here It includes supplementing the problem of resolution and sensitivity reduction occurring in the acquired image using a convolutional neural network (CNN).

Looking at the simulation results of the apparatus and method for improving the sensitivity and resolution in the grating interferometer through machine learning according to the present invention, the sensitivity and resolution of the phase difference image are improved using a convolutional neural network to produce high quality medical images. You can see that it can be provided.

10A to 10B are configuration diagrams showing simulation results of an apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning according to the present invention.

As described above, it will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention.

Therefore, the specified embodiments should be considered from a descriptive point of view rather than a limiting point of view, and the scope of the present invention is shown in the claims rather than the above description, and all differences within the scope equivalent thereto are included in the present invention. It will have to be interpreted.

10. Grating interferometer image acquisition unit 20. Numerical phantom generation unit
30. Convolutional layer generation unit 40. Activation function application operation unit
50. CNN iteration machine learning unit 60. Image matching output unit

Claims (9)

A grating interferometer image acquisition unit that obtains a high-resolution image and a high-sensitivity image by linearly moving the position of the sample from the symmetric grating interferometer;
A numerical phantom generator for generating a numerical phantom for performing machine learning;
A convolutional layer generator for extracting features from input data by performing an operation of the convolutional neural network;
An activation function application operation unit capable of performing seamless repetitive machine learning by applying a ReLu (Rectified linear unit) activation function to an output value of a convolution operation;
A CNN iterative machine learning unit that modifies a convolution operator while repeatedly performing forward propagation and backward propagation processes;
Including; an image matching output unit for matching and outputting the features extracted by the iterative machine learning of the CNN iteration machine learning unit,
The numerical phantom generator measures the resolution, sensitivity, and image noise of the acquired phase difference image to implement an image of the same level as a numerical simulation, and generates a plurality of numerical phantoms for high accuracy. A device for improving sensitivity and resolution in grating interferometers through machine learning. The method of claim 1, wherein the grid interferometer image acquisition unit,
Machine learning, characterized in that a symmetrical lattice interferometer is installed to linearly move the position of the subject to obtain a pair of high-sensitivity images and high-resolution images, and to rearrange image sizes to match different magnifications of each image. Device for improving sensitivity and resolution in grating interferometer through delete The method of claim 1, wherein the convolutional layer generation unit,
A machine characterized in that a filter, which is a convolution operation factor, is created to convolution of input data, and the width and height of the filter are smaller than the input data and the number of filters is 32 or more. A device for improving sensitivity and resolution in a grating interferometer through learning. The method of claim 1, wherein in the convolution operation of the input data and the filter in the convolutional layer generator,
In order for the result of the convolution operation to maintain the same horizontal and vertical size of the input image, zero padding, stride, and generated filter size are set,
The size of the output data is,

Is made of,
Here, (C _o ,R _o ) is the size of the output data (C=column, R=row), (C _i ,R _i ) is the size of the input data, S is the stride, P is the zero-padding, and (C _An apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning, characterized in that _f , R _f ) is the size of a filter. The method of claim 1, wherein in the convolution operation of the input data and the filter in the convolutional layer generator,
Perform zero padding to solve the problem of disappearing edge information of input data,
Convolution operation is characterized in that when the filter is moved within the input data, the movement interval is set to 1 (stride = 1) and the size of the output data is adjusted to the same size as the input value, and the sensitivity and resolution in a lattice interferometer through machine learning. Device for improvement. The method of claim 1, wherein the activation function application operation unit,
An apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning, characterized in that a ReLu function is applied to an output value calculated from an operation of each convolutional layer to reduce an error that may occur during subsequent iterative machine learning. The method of claim 1, wherein in the CNN iterative machine learning unit,
Forward propagation is the final calculation of the loss value of the cost function through the convolution operation between the input data generated by simulation and the convolution operation factor.
Back propagation is a device for improving sensitivity and resolution in a lattice interferometer through machine learning, characterized in that weight and bias are corrected by partial differentiation of a convolution operator in the reverse direction of a forward propagation operation to minimize loss. The method of claim 1, wherein the CNN repetition machine learning unit,
Create a cost function that reduces the difference between the result of the convolution operation performed from the input image and the ground truth through the mean square method,
An apparatus for improving sensitivity and resolution in a lattice interferometer through machine learning, characterized in that forward propagation and backward propagation processes are repeatedly performed in a direction in which the generated cost function is reduced.

Images