KR20230090378A - Bone metastasis detecting method using ct image and analysis apparatus - Google Patents

Abstract

A bone metastasis detection method using a CT image includes the steps of receiving a CT image of a subject by an analysis device, extracting a plurality of 2D images of different views from the CT image by the analysis device, and The method includes inputting a plurality of 2D images to a previously learned neural network model and determining, by the analysis device, whether or not bone metastasis of the subject is present based on a probability value output from the neural network model.

Description

Bone metastasis detection method and analysis device using CT image {BONE METASTASIS DETECTING METHOD USING CT IMAGE AND ANALYSIS APPARATUS}

The technique described below is CT It relates to a technique for automatically detecting bone metastases using images.

In the medical field, CT (Computed Tomography) images are commonly used for diagnosis and treatment. Bone metastasis is a state in which a primary cancer (lung cancer, breast cancer, prostate cancer, etc.) has metastasized to bone, and can be diagnosed using MRI or CT.

Korean Patent Publication No. 10-2021-0020629

Recently, artificial intelligence technology for detecting lesions by analyzing medical images such as CT has been actively researched. However, conventional techniques use a method of inputting an image of a specific direction from a CT image having a tertiary space characteristic to an artificial intelligence model or inputting 3D volume data to an artificial intelligence model at once. However, this method has limitations in that the detection performance is poor or the false positive probability is high compared to the method in which the medical staff comprehensively reads images from various directions to detect lesions.

The technique described below aims to provide a technique for detecting bone metastasis using an artificial intelligence model that comprehensively determines images in axial, coronal, and sagittal directions.

A bone metastasis detection method using a CT image includes the steps of receiving a CT image of a subject by an analysis device, extracting a plurality of 2D images of different views from the CT image by the analysis device, and The method includes inputting a plurality of 2D images to a previously learned neural network model and determining, by the analysis device, whether or not bone metastasis of the subject is present based on a probability value output from the neural network model.

An analysis device that receives a CT image and detects bone metastasis is an input device that receives a CT image of a subject and a neural network model that receives a plurality of 2D images from different views among the CT images and calculates a probability value of bone metastasis. A storage device for storing , and a plurality of 2D images of different viewpoints are extracted from the input CT image and inputted to the neural network model, and based on a probability value output by the neural network model, it is determined whether or not the subject has bone metastasis. It includes an arithmetic device that

The technique described below can minimize the probability of false positive or false negative by using images from various viewpoints at once to determine bone metastasis. The technology to be described below can accurately determine bone metastasis by setting a region to be analyzed intensively with attention for each of the images of various viewpoints. A diagnosis assisting means for a medical staff may be provided using the technology described below.

1 is an example of a bone metastasis detection system using a CT image.
2 is an example of a schematic process for bone metastasis detection.
3 is an example of a neural network model for bone metastasis detection.
4 is an example of an analysis device for detecting bone metastasis using a CT image.

Since the technology to be described below can have various changes and various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, and it should be understood to include all modifications, equivalents, or substitutes included in the spirit and scope of the technology described below.

Terms such as first, second, A, B, etc. may be used to describe various elements, but the elements are not limited by the above terms, and are merely used to distinguish one element from another. used only as For example, without departing from the scope of the technology described below, a first element may be referred to as a second element, and similarly, the second element may be referred to as a first element. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In terms used in this specification, singular expressions should be understood to include plural expressions unless clearly interpreted differently in context, and terms such as “comprising” refer to the described features, numbers, steps, operations, and components. , parts or combinations thereof, but it should be understood that it does not exclude the possibility of the presence or addition of one or more other features or numbers, step-action components, parts or combinations thereof.

Prior to a detailed description of the drawings, it is to be clarified that the classification of components in the present specification is merely a classification for each main function in charge of each component. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function. In addition, each component to be described below may additionally perform some or all of the functions of other components in addition to its main function, and some of the main functions of each component may be performed by other components. Of course, it may be dedicated and performed by .

In addition, in performing a method or method of operation, each process constituting the method may occur in a different order from the specified order unless a specific order is clearly described in context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

Hereinafter, it will be described that the analysis device detects bone metastases using CT using a certain learning model. The analysis device may be implemented with various devices capable of processing data. For example, the analysis device may be implemented as a PC, a server on a network, a smart device, a chipset in which a dedicated program is embedded, and the like.

There are many types of learning models, including decision trees, random forests, K-nearest neighbors (KNNs), Naive Bayes, support vector machines (SVMs), and artificial neural networks (ANNs).

ANN (Artificial Neural Network) is a statistical learning algorithm that mimics biological neural networks. Various neural network models are being studied. A deep learning network (DNN) can model complex non-linear relationships like a general artificial neural network. Various types of DNN models have been studied. For example, there are a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), a Generative Adversarial Network (GAN), and Relation Networks (RL). Hereinafter, it is assumed that the analysis device detects bone metastasis by analyzing CT, which is a medical image, using an artificial neural network model.

1 is an example of a bone metastasis detection system 100 using a CT image.

The CT equipment 110 generates a CT image of the subject's abdomen. The subject may be a patient suspected of having bone metastasis. An abdominal CT image generated by the CT equipment 110 may be stored in an Electronic Medical Record (EMR) 120. Although FIG. 1 is described based on abdominal CT, bone metastasis may be the head, arms, legs, etc., depending on the examination site.

In FIG. 1 , the user A may use the computer terminal 130 to detect bone metastases in the abdominal CT image of the subject. The computer terminal 130 may receive a CT image of the subject's abdomen from the CT equipment 110 or the EMR 120 through a wired or wireless network. In some cases, the computer terminal 130 may be a device physically connected to the CT equipment 110. The computer terminal 130 may simultaneously analyze axial plane data, sagittal plane data, and coronal plane data of the subject's abdominal CT image to detect bone metastasis in the subject. The computer terminal 130 may input axial plane data, sagittal plane data, and coronal plane data to a previously trained neural network model, and classify whether or not the subject has bone metastasis based on a value output from the neural network model. User A may check whether the subject has bone metastasis through the computer terminal 130 .

The server 140 may receive an abdominal CT image of the subject from the CT equipment 110 or the EMR 120 through a network. The server 140 may simultaneously analyze axial plane data, sagittal plane data, and coronal plane data of the subject's abdominal CT image to detect bone metastasis in the subject. The server 140 may input axial plane data, sagittal plane data, and coronal plane data to a previously trained neural network model, and classify whether or not the subject has bone metastasis based on a value output from the neural network model. The server 140 may transmit, as an analysis result, whether or not the subject has bone metastases to the user A's terminal. User A may check whether the subject has bone metastases through the user terminal.

The computer terminal 130 and/or the server 140 may store in the EMR 120 whether or not the subject has bone metastasis as an analysis result.

2 is an example of a schematic process 200 for bone metastasis detection. 2 is an example of a process in which an analysis device analyzes an input CT image of a subject using a learned neural network model.

The analysis device extracts axial plane data, sagittal plane data, and coronal plane data from the input CT image (210). A CT image is composed of consecutive slices in a certain axial direction. The analysis device may select at least one slice for each of the axial plane, the sagittal plane, and the coronal plane. Furthermore, the analysis device may select a plurality of slices for each of the axial, sagittal and coronal planes. The number of input slices for each of the axial plane, the sagittal plane, and the coronal plane may vary depending on the structure of the neural network model described later.

The analysis device may perform pre-processing on the input CT image (220). (i) The analysis device may adjust or crop the CT image to a certain size. (ii) Also, the analysis device can remove noise from the CT image. For example, the analysis device may remove noise by performing erosion and dilation on the image. The analysis device may remove noise by an opening technique (sequential performance of erosion and dilation) or a closing technique (sequential performance of dilation and erosion). The analysis device may perform the same pre-processing on each of the axial, sagittal, and coronal slices extracted from the CT image.

The analysis device may detect bone metastasis by inputting axial plane data, sagittal plane data, and coronal plane data into one neural network model (230). The neural network model receives axial plane data, sagittal plane data, and coronal plane data and calculates a probability value for whether or not the subject has bone metastases. The analysis device may determine whether or not the subject has bone metastasis based on the probability value output by the neural network model. For example, if the probability value is greater than or equal to a threshold value, the analysis device may determine bone metastasis. Alternatively, the analysis device may determine that a probability value between 0 and 1 is normal if it is close to 0, and bone metastasis if it is close to 1.

2 explains that the analysis device uses axial, sagittal, and coronal slices. Furthermore, the analysis device may extract or synthesize 2D images in a specific direction using 3D CT images. Accordingly, images of different viewpoints input to the neural network model may be images of various directions. However, for convenience of description below, it is assumed that the three different viewpoints are an axial plane, a sagittal plane, and a coronal plane.

3 is an example of a neural network model 300 for bone metastasis detection. 3 is an example of a process in which the neural network model 300 detects (infers) whether or not there is bone metastasis based on an input image. Therefore, it is assumed that the neural network model 300 has been trained using training data in advance. The process of FIG. 3 is a process performed by inputting the input CT image to the neural network model 300 by the analysis device.

3 shows a neural network model 300 with a CNN structure. The neural network model 300 includes a plurality of convolutional layers (input layers) that process images of different viewpoints and a fully convolutional layer (fully connected) that performs final classification by receiving features output from the plurality of convolutional layers. connected layer).

In the CT image, axial plane data, coronal plane data, and sagittal plane data are each input to different convolution layers.

The first convolution layer 311 receives axial plane data. The first convolution layer 311 may extract features from one slice. Furthermore. The first convolution layer 311 may be configured as an individual layer that processes one slice to simultaneously extract features from a plurality of slices.

The first attention layer 312 receives axial plane data. The first attention layer 312 generates a first attention map (attention map 1) by calculating information about a region that affects a detection result (whether there is bone metastasis) while extracting features from input data. The first attention map may be composed of regional weight information for features calculated by the first convolution layer 311 .

The first convolution layer 311 repeats the convolution operation and the pooling operation to finally calculate a first feature map (feature map 1) for the on-axis plane data. At this time, the first convolution layer 311 generates a first feature map by applying the first attention map.

The second convolution layer 321 receives coronal plane data. The second convolution layer 321 may extract features from one slice. Furthermore. The second convolution layer 321 may be composed of individual layers that process one slice to simultaneously extract features from a plurality of slices.

The second attention layer 322 receives coronal plane data. The second attention layer 322 generates a second attention map (attention map 2) by calculating information about a region that affects a detection result (whether there is bone metastasis) while extracting features from input data. The second attention map may be composed of regional weight information for features calculated by the second convolution layer 321 .

The second convolution layer 321 repeats the convolution operation and the pooling operation to finally calculate a second feature map (feature map 2) for the on-axis plane data. At this time, the second convolution layer 321 generates a second feature map by applying the second attention map.

The third convolution layer 331 receives axial plane data. The third convolution layer 331 may extract features from one slice. Furthermore. The third convolution layer 331 may be configured as an individual layer that processes one slice to simultaneously extract features from a plurality of slices.

The third attention layer 332 receives axial plane data. The third attention layer 332 generates a third attention map (attention map 3) by calculating information about a region that affects a detection result (bone metastasis) while extracting features from input data. The third attention map may be composed of regional weight information for features calculated by the third convolution layer 331 .

The third convolution layer 331 repeats the convolution operation and the pooling operation to finally calculate a third feature map (feature map 3) for the on-axis plane data. At this time, the third convolution layer 331 generates a third feature map by applying the third attention map.

The analysis device concatenates the first feature map, the second feature map, and the third feature map. At this time, at least one of the first convolution layer 311, the second convolution layer 321, and the third convolution layer 331 may generate feature maps for a plurality of slices. When a plurality of slices are processed in at least one of an axial plane, a sagittal plane, and a coronal plane, the analysis device may combine feature maps of all slices.

Furthermore, the analysis device may post-process the feature values by summing the first feature map, the second feature map, and the third feature map or performing other operations. Thereafter, the analysis device may input the post-processed feature map to the pre-separation layer 340 .

It is assumed that the total concatenation layer 340 receives the combined feature map. The total heat transfer layer 340 finally receives the combined feature map and calculates a probability value as to whether or not the subject has bone metastasis.

Learning of the neural network model 300 requires the same type of data as the input data described in FIG. 3 and a label value (whether bone metastases or not) of the corresponding input data. The learning device learns the neural network model 300 through a supervised learning process using the learning data, and repeats the learning process until it shows a certain performance. The learning device refers to a computer device capable of processing data as a device that performs a learning process of a neural network model.

The learning device may perform the same preprocessing process as described in FIG. 2 on the CT image, which is learning data. Learning devices to minimize false positive detections. When training a neural network model, in order to intensively learn data that includes areas that can be mistaken for bone metastasis, such as osteophyte, or to minimize false negatives (undetected), the size is small (about 1,000 pixels or less) or dark ( 300 HU or less) lesions can be intensively studied.

4 is an example of an analysis device 400 that detects bone metastasis using a CT image. The analysis device 400 corresponds to the above-described analysis devices (130 and 140 in FIG. 1). The analysis device 400 may be physically implemented in various forms. For example, the analysis device 400 may have a form of a computer device such as a PC, a network server, and a chipset dedicated to data processing.

The analysis device 400 may include a storage device 410, a memory 420, an arithmetic device 430, an interface device 440, a communication device 450, and an output device 460.

The storage device 410 may store a CT image of the subject.

The storage device 410 may store the aforementioned neural network model.

The storage device 410 may store a program for pre-processing the CT image.

The storage device 410 may store a program for extracting images of different viewpoints from the input CT image and inputting the extracted images of different viewpoints to the neural network model to detect bone metastasis. Images of different viewpoints may include an axial plane image, a coronal plane image, and a sagittal plane image.

The memory 420 may store data and information generated in the course of the analysis device 400 detecting whether or not the subject has bone metastases.

The interface device 440 is a device that receives certain commands and data from the outside. The interface device 440 may receive a CT image of the subject from a physically connected input device or an external storage device. The interface device 440 may transmit the analysis result to an external object.

The communication device 450 refers to a component that receives and transmits certain information through a wired or wireless network. The communication device 450 may receive a CT image of the subject from an external object. Alternatively, the communication device 450 may transmit the analysis result to an external object such as a user terminal.

Since the interface device 440 and the communication device 450 are configured to send and receive certain data from a user or other physical object, they can also be collectively referred to as input/output devices. The interface device 440 and the communication device 450 may also be referred to as input devices if the function of receiving the subject's CT image is limited.

The output device 460 is a device that outputs certain information. The output device 460 may output interfaces and analysis results required for data processing.

The arithmetic device 430 may detect whether or not the subject has bone metastasis using the model or program code stored in the storage device 410 .

The arithmetic device 430 may extract 2D images of different directions from an input CT image of the subject. The computing device 430 may extract axial plane data, coronal plane data, and sagittal plane data from the CT image.

The arithmetic unit 430 may perform preprocessing such as noise removal on CT images or extracted images of a specific direction.

The computing device 430 may input 2D images (eg, axial plane data, coronal plane data, and sagittal plane data) in different directions to the neural network model described in FIG. 3 .

The calculation device 430 may determine whether or not the subject has bone metastasis based on a probability value output by the neural network model. For example, the calculator 430 may determine that bone metastasis occurs when the probability value is greater than or equal to a threshold value. Alternatively, the arithmetic device 430 may determine that the probability value is normal between 0 and 1 if it is close to 0, and if it is close to 1, it is bone metastasis.

The arithmetic device 430 may be a device such as a processor, an AP, or a chip in which a program is embedded that processes data and performs certain arithmetic operations.

In addition, the artificial neural network-based bone metastasis detection method as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided in a temporary or non-transitory computer readable medium.

A non-transitory readable medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, the various applications or programs described above are CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM) Alternatively, it may be stored and provided in a non-transitory readable medium such as EEPROM (Electrically EPROM) or flash memory.

Temporary readable media include static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), and enhanced SDRAM (Enhanced SDRAM). SDRAM, ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM) and Direct Rambus RAM (DRRAM).

This embodiment and the drawings accompanying this specification clearly represent only a part of the technical idea included in the foregoing technology, and those skilled in the art can easily understand it within the scope of the technical idea included in the specification and drawings of the above technology. It will be obvious that all variations and specific examples that can be inferred are included in the scope of the above-described technology.

Claims (10)

receiving a CT image of the subject by an analysis device;
extracting, by the analysis device, a plurality of 2D images of different views from the CT image;
inputting, by the analysis device, the plurality of 2D images to a previously learned neural network model; and
Bone metastasis detection method using a CT image comprising the step of determining, by the analysis device, whether or not the subject has bone metastasis based on a probability value output from the neural network model. According to claim 1,
The plurality of 2D images include an axial plane image, a coronal plane image, and a sagittal plane image. Bone metastasis detection method using CT images. According to claim 1,
The neural network model is
a plurality of convolution layers receiving a plurality of slices for each of the different viewpoints and outputting feature values; and
A method for detecting bone metastasis using a CT image comprising a pre-connected layer receiving a value obtained by combining feature values output from each of the plurality of convolution layers and outputting the probability value. According to claim 1,
The neural network model is
a plurality of attention layers receiving a plurality of slices for each of the different views and generating an attention map for each slice;
a plurality of convolution layers receiving a plurality of slices for each of the different viewpoints and outputting feature values by applying an attention map to a corresponding slice; and
A method for detecting bone metastasis using a CT image comprising a pre-connected layer receiving a value obtained by combining feature values output from each of the plurality of convolution layers and outputting the probability value. According to claim 1,
The neural network model is
a first attention layer generating a first attention map by receiving axial plane data from among the 2D images;
a first convolution layer that receives the axial plane data from among the 2D images and outputs a feature value, and outputs a first feature value by applying the first attention map;
a second attention layer generating a second attention map by receiving coronal plane data from among the 2D images;
a second convolution layer that receives the coronal plane data from among the 2D images and outputs feature values, and outputs second feature values by applying the second attention map;
a third attention layer generating a third attention map by receiving sagittal data from among the 2D images;
a third convolution layer that receives the sagittal plane data from among the 2D images and outputs a feature value, and outputs a third feature value by applying the third attention map; and
A bone metastasis detection method using a CT image comprising a full connection layer receiving a value obtained by combining the first feature value, the second feature value, and the third feature value and outputting the probability value. an input device that receives a CT image of the subject;
a storage device for storing a neural network model that receives a plurality of 2D images of different views from among CT images and calculates a probability value of bone metastasis; and
An arithmetic device for extracting a plurality of 2D images of different viewpoints from the input CT image, inputting the extracted 2D images to the neural network model, and determining whether the subject has bone metastasis based on a probability value output from the neural network model An analysis device that receives CT images and detects bone metastases. According to claim 6,
The plurality of 2D images is an analysis device for detecting bone metastasis by receiving CT images including axial, coronal, and sagittal images. According to claim 6,
The neural network model is
a plurality of convolution layers receiving a plurality of slices for each of the different viewpoints and outputting feature values; and
An analysis device for detecting bone metastasis by receiving a CT image including a pre-connected layer that receives a value obtained by combining feature values output from each of the plurality of convolution layers and outputs the probability value. According to claim 6,
The neural network model is
a plurality of attention layers receiving a plurality of slices for each of the different views and generating an attention map for each slice;
a plurality of convolution layers receiving a plurality of slices for each of the different viewpoints and outputting feature values by applying an attention map to a corresponding slice; and
An analysis device for detecting bone metastasis by receiving a CT image including a pre-connected layer that receives a value obtained by combining feature values output from each of the plurality of convolution layers and outputs the probability value. According to claim 6,
The neural network model is
a first attention layer generating a first attention map by receiving axial plane data from among the 2D images;
a first convolution layer that receives the axial plane data from among the 2D images and outputs a feature value, and outputs a first feature value by applying the first attention map;
a second attention layer generating a second attention map by receiving coronal plane data from among the 2D images;
a second convolution layer that receives the coronal plane data from among the 2D images and outputs feature values, and outputs second feature values by applying the second attention map;
a third attention layer generating a third attention map by receiving sagittal data from among the 2D images;
a third convolution layer which receives the sagittal plane data from among the 2D images and outputs a feature value, and outputs a third feature value by applying the third attention map; and
An analysis device that detects bone metastasis by receiving a CT image including a full connection layer that receives a value obtained by combining the first feature value, the second feature value, and the third feature value and outputs the probability value.

Images