KR20190002957A - A method for segmentation of wall or lumen in intravascular ultrasound image with using convolution neural network - Google Patents

Abstract

The present invention relates to a method for segmentation of blood vessel walls and lumen in an intravascular ultrasound (IVUS) image, which comprises the following steps of: receiving image data; processing the image data by using a convolution neural network; and displaying a resultant image obtained in the processing step on a display device. Also, the present invention may provide an apparatus for segmentation of blood vessel walls and lumen in an IVUS image, comprising: at least one processing unit for executing computer-executable instructions; a graphic processing unit (GPU) integrated or separately included on the processing unit and one board; and a memory, wherein the CPU performs segmentation of blood vessel walls and lumen in the IVUS image through a convolution neural network.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method for forming a vessel wall and a lumen area in an intravascular ultrasound image using a convolutional neural network,

The present invention relates to a method for segmenting a lesion in an intravascular ultrasound (IVUS) image using a convolutional neural network. More specifically, the present invention relates to a method for precisely analyzing a lesion by segmenting a vessel wall and / or a lumen by using a convolutional neural network in an IVUS image.

Intravascular ultrasound (IVUS) is an invasive medical imaging technique. Ultrasound equipment connected to the end of a catheter can be used to determine the size and type of a vein. Therefore, current IVUS technique is very useful for the analysis of coronary artery lesions. However, in order to perform an accurate analysis from these IVUS images, it is necessary to clearly divide the vessel wall and lumen (lumen) in the image in order to analyze the lesion, which must be handled manually by a skilled medical image analysis expert so far. Therefore, the task of analyzing the lesion from the IVUS image is uneven due to individual differences such as proficiency of the performer.

Accordingly, there is a need for a more effective analysis method that is not affected by the individual differences of the performers performing the analysis work.

An object of the present invention is to provide an analysis method that is not affected by an individual who performs an analysis of an IVUS image.

According to an embodiment of the present invention, a method for analyzing an image through a convolutional neural network, which is one of artificial intelligence techniques, can be provided to solve the above-described problems.

According to another embodiment of the present invention, it is possible to provide an apparatus for segmenting an intravascular lesion using an IVUS image from a convolutional neural network.

The present invention, which is devised corresponding to the background art described above, can provide a method for efficiently and accurately identifying a lesion included in an image for a blood vessel. Particularly, this method is an effective method that is not affected by individual performances of performing identification and analysis.

FIG. 1 is a diagram illustrating an IVUS image obtained by segmenting a blood vessel wall according to the method of the present invention and comparing ground truth. FIG.
2 is a schematic diagram of an embodiment of a computer apparatus according to the present invention.

In this specification, various explanations are given in order to provide an understanding of the present invention. It will be apparent, however, that such embodiments may be practiced without these specific details. In other instances, well-known structures and devices are provided in block diagram form in order to facilitate describing the embodiments.

The terms "component," "module," system, "and the like, as used herein, refer to a computer-related entity, hardware, firmware, software, combination of software and hardware, or execution of software. For example, a component may be, but is not limited to, a process executing on a processor, a processor, an object, an executing thread, a program, and / or a computer. For example, both an application running on a computing device and a computing device may be a component. One or more components may reside within a processor and / or thread of execution, one component may be localized within one computer, or it may be distributed between two or more computers. Further, such components may execute from various computer readable media having various data structures stored therein. The components may be, for example, a signal (e.g., a local system, data from one component interacting with another component in a distributed system, and / or data over a network, such as the Internet, Lt; RTI ID = 0.0 > and / or < / RTI >

IVUS is a medical imaging method using a specially designed catheter with a small ultrasound probe attached to the distal end. The proximal end of the catheter is attached to a computerized ultrasound machine. Through this, it is a technique used for visualizing the inner wall of the blood vessel from inside the blood vessel by applying an ultrasonic technique such as a piezoelectric transducer or CMUT.

Coronary arteries are the most commonly applied IVUS in the heart. IVUS is commonly used in coronary arteries to determine the amount of atherosclerotic lesions accumulated at specific points in the epicardial coronary arteries. This is an apparatus for performing imaging directly from inside the blood vessel using ultrasonic waves, so that it is possible to provide a relatively accurate internal image of the blood vessel as compared with the case where the reliability of other non-invasive images (angiography, etc.) is insufficient.

However, even with the IVUS technique that provides this direct image, it is entirely manual to analyze the lesion from the provided image. Specifically, it is difficult to accurately divide the vessel wall and the lumen from the photographed image, and to analyze the size or type of the lesion present in the blood vessel, it depends entirely on the competence of a physician. As a result, Is inevitable.

On the other hand, deep-learning using artificial intelligence has recently been emerging. Deep learning, or in-depth learning, is defined as a set of machine learning algorithms that try to achieve a high level of abstraction through a combination of several non-linear transformation techniques. It is a field of machine learning that teaches computers how people think in a larger framework .

As a result of these efforts, various deep learning techniques have been applied to computer visualization, speech recognition, natural language processing, Voice / signal processing, and convolution neural networks.

Convolutional Neural Network (CNN) is a kind of multilayer perceptrons designed to use minimal preprocessing. CNN is composed of one or several layers of convolutional hierarchy and general artificial neural networks layered on top of it, and utilizes additional weighting and pooling layers. This structure allows CNN to fully utilize the input data of the two-dimensional structure. Compared to other deep-run structures, CNN offers good performance in both video and audio applications. CNN can also be trained through standard reverse delivery. CNN is easier to train than other feedforward artificial neural network techniques and has the advantage of using fewer parameters.

Machine learning can be a direct learning process that extracts knowledge from data, but it is common to learn from the stage of "data-feature-knowledge" through feature extraction, which is usually an intermediate step. For example, in order to recognize an object in a photograph, first a characteristic line or a characteristic color distribution is first extracted from a pixel value, and then a target object is determined based on this characteristic line. This intermediate expression step is called a feature map. The performance of machine learning depends largely on how well the features are extracted. In particular, convolution neural networks that extract and learn features in multiple stages are useful for image recognition.

Machine learning or data extraction using such a convolutional neural network can be further improved by the following techniques.

1. Data Augmentation

For machine learning, a large amount of data is required, which can lead to general estimation by learning about various cases. If the amount of data is small, only the learned data is estimated well, which leads to the problem of overfitting, which affects the performance of the artificial intelligence model. Medical images belong to a small number of data due to the nature of technology. It is therefore necessary to solve the problems caused by the lack of data, which can be supplemented by flipping or rotating the image.

2. Convolution filter

The convolutional neural network convolutes the values of the input image included in the size of the input image with a predetermined filter size and outputs the result. Expressing this as an equation, when the filter is W, the convolution result can be expressed as Wx + b (b is a bias). This is linear modeling, and the artificial intelligence of a deeper layer allows a more sophisticated classifier to be designed through the combination of linear and nonlinear functions.

The size of the image through convolution depends on the size of the filter and the stride on which the filter moves in the image. When the input image size is N × N and the filter size is F × F, the size of the image output through this filter is (NF) / stride +1, and the feature map of the next layer is affected by the size of the filter . Basically, there is a problem in that the size of the input image becomes smaller as the filter becomes rougher. In order to prevent this, it is necessary to amplify the input image.

3. Max pooling

Maximum pooling is the task of sampling a convolved feature map. For example, if a 4 × 4 feature map and a 2 × 2 filter are assumed, a 2 × 2 result layer is created by extracting the largest value among the values within the 2 × 2 filter. According to this technique, the feature map can be extracted from only the strongest features from the input image.

4. Softmax layer and one-hot encoding.

Softmax performs the task of changing the score result to a value between 0 and 1 where the total sum is 1. The exponent (exp) is taken for each score, then divided by the normalization constant so that the sum is 1.

At the end of the neural network, vector values for N classes corresponding to the class are calculated, and vectors corresponding to the number of classes have arbitrary values. When the result passes through the soft max layer, each result value is changed to a vector value between 0 and 1. Thereafter, the highest value is converted to 1 by the original hot encoding, and the remaining values are converted to 0, thereby extracting the most feature dominant result.

5. Cost function and gradient descent algorithm.

In order to learn the weight value of artificial intelligence, an objective function is defined. The objective function generally depends on the purpose but uses a cross entropy method. The known ground truth is defined using the objective function using a feature vector passing through the soft max layer. At this time, when the gradient descent method is used, the learning is performed in a direction in which the value of the objective function is minimized.

When using the gradient descent algorithm, if the number of learning epochs is provided too much, the neural network tends to produce results that are specific to specific input values and actual information. For example, Exercise problems can be solved accurately, but if other problems arise, the results are not as expected. To prevent this, we can solve the problems of overfitting by adding dropout layer, input noise, weight noise, and so on.

Accordingly, in the method of the present invention, the machine learning of the convolutional neural network can be performed through the following steps:

i) Receive learning sample data and neural network data

ii) Prepare neural network data as graphic data

iii) For each input sample

- Output calculation result

- Calculation of error for expected output

- Determination of objective function

- Reset Convolution Filter

- Repeat the above

iv) Enter the following sample - repeat

According to the method of the present invention, the vessel wall and the lumen can be precisely segmented and displayed from the IVUS image using the convolution neural network as described above. Referring to FIG. 1, it can be seen from the IVUS image that the result of the zoning using the convolutional neural network (display of the red circle of the center image) have.

According to the present invention, it is possible to provide an apparatus for segmenting an intravascular lesion using an IVUS image using a convolutional neural network. The apparatus is not intended to suggest any limitation as to the scope of the functionality or use of the invention, and the invention may be implemented in a variety of general purpose or special purpose computing environments.

Referring to FIG. 2, the localization apparatus of the present invention includes at least one processing unit, a GPU (graphics processing unit), and a memory. In Figure 2, this most basic configuration is included in the dashed line. The processing unit executes computer-executable instructions and may be a real processor or a virtual processor. In a multiprocessing system, multiple processing units execute computer executable instructions to improve processing capabilities. The memory can be any combination of volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.) or two memories. The memory stores the described encoder / decoder and software implementing an efficient transform coefficient encoding / decoding technique. The GPU can be integrated with the processing unit on one board or can be included separately.

The zoning device of the present invention may have additional features. For example, a repository, one or more input devices and one or more output devices, and one or more communication connections. Interconnection mechanisms, such as buses, controllers, or networks interconnect components of computing devices. Typically, operating system software provides an operating environment for other software running on the computing device and coordinates the operation of the components of the device.

The storage may be removable or non-removable and may include magnetic disks, magnetic tape or cassettes, CD-ROMs, CD-RWs, DVDs, or any other medium that can be used to store information and that can be accessed within computing devices do. The repository stores instructions for software to implement the described neural network processing results.

The input device may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device capable of providing input to a computing device. In the case of audio, the input device may be a similar device or sound card that accepts audio input in analog or digital form, or may be a CD-ROM recorder that provides audio samples to the computing device. The output device may be a display, a printer, a speaker, a CD-recorder, or another device that is provided with output from the computing device.

A communication connection enables communication with another computing entity via a communication medium. The communication medium carries information such as computer-executable instructions, compressed audio or video information, or other data into the modulated data signal. A modulated data signal is a signal having one or more of its characteristics set or changed in the same manner as encoding information in the signal. By way of example, and not limitation, communication media includes wireless technologies embodied in wireline or electrical, optical, RF, infrared, acoustic, or other carrier waves.

Digital media processing techniques in this document may be described in the general context of computer readable media. The computer readable medium is any available medium that can be accessed within the computing device. By way of example, and not limitation, computer readable media, along with computing devices, include memory, storage, communication media, and any combination of the foregoing.

Neural network learning techniques in this document may be described in the general context of computer executable instructions, which are included in a program module running on a computing device on a target virtual processor or on a target virtual processor. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or separated among the program modules as desired in various embodiments. The computer executable instructions for the program modules may be executed within a local computing device or a distributed computing device.

For purposes of explanation, the detailed description has described computer operations in computing devices using terms such as "determine", "determine", "generate", "adjust" and "apply". This term is a high-level abstraction for the actions performed by the computer and should not be confused with the actions performed by humans. The actual computer operation corresponding to this term is implementation dependent.

In view of the many possible variations of the technical idea described herein, it is intended that all embodiments falling within the scope of the following claims and their equivalents be claimed.

Claims (4)

A method for segmenting a vessel wall and a lumen in an intravascular ultrasound (IVUS)
Receiving image data;
Processing the image data using a convolution neural network; And
Displaying the resultant image obtained in the processing step on a display device
≪ / RTI > The method according to claim 1,
The processing of the image data may comprise combining one or more techniques selected from the group consisting of data enhancement, magnification of the input image size, Max pooling, soft max, original hot encoding, objective function setting, ≪ / RTI > An apparatus for segmenting a vessel wall and a lumen in an intravascular ultrasound (IVUS)
At least one processing unit for executing computer executable instructions;
A GPU (graphics processing unit) integrated or separately contained on one board with the processing unit; And memory
Lt; / RTI >
Wherein the GPU scales the vessel wall and lumen in the ultrasound image through a convolution neural network.
The method of claim 3,
At least one communication connection, at least one input and output device, and a storage.

Images