WO2023101203A1

WO2023101203A1 - Method for estimating lesion volume using x-ray image, and analysis device

Info

Publication number: WO2023101203A1
Application number: PCT/KR2022/015555
Authority: WO
Inventors: 정명진; 차윤기; 임채영
Original assignee: 사회복지법인 삼성생명공익재단
Priority date: 2021-11-30
Filing date: 2022-10-14
Publication date: 2023-06-08
Also published as: KR20230080825A

Abstract

A method for estimating lesion volume using an X-ray image comprises the steps in which an analysis device: receives a 2D X-ray image of a patient as input; determines a detection probability for a specific lesion by inputting the X-ray image to a machine learning model; and estimates the 3D volume of the lesion on the basis of the area of the lesion detected from the X-ray image and the detection probability.

Description

Method and analysis device for estimating lesion volume information using X-ray images

A technique described below relates to a technique for calculating information on a 3D volume of a specific lesion using a 2D X-ray image.

An X-ray image is a basic medical image for examination of various diseases. For example, a medical staff may check a patient's pulmonary nodule using a chest X-ray. When a lesion such as a pulmonary nodule is found, the medical staff monitors the size of the lesion through periodic X-ray imaging.

In general, a medical staff checks the size of a crystal by sight or by using a ruler to determine the size of a lesion in an X-ray image. Furthermore, the medical staff may check the area by drawing the outer edge of the crystal using computer aided diagnosis (CAD). However, since a lesion is an object in a 3D space, it is difficult to accurately observe a change in size only with a 2D X-ray image. Furthermore, X-ray images are inferior in sharpness to CT (computer tomography), and thus, there is a limitation in that it is difficult to accurately determine the periphery of the nodule shadow.

The technology described below provides a method for estimating volume information of lesions based on X-ray images based on the research results that the lesion detection probability identified as a learning model in 2D X-ray images and the area of lesions have a constant correlation with the volume of lesions. want to do

The method for estimating lesion volume information using an X-ray image includes the step of receiving a 2D X-ray image of a patient by an analysis device, and determining the detection probability of a specific lesion by the analysis device by inputting the X-ray image to a machine learning model. and estimating, by the analysis device, 3D volume information of the lesion based on the area of the lesion detected in the X-ray image and the detection probability.

An analysis device that estimates lesion volume information using an X-ray image is an input device that receives a patient's 2D X-ray image, and a machine learning model pretrained to classify the possibility of a specific lesion based on the patient's X-ray image. A storage device for storing and inputting an X-ray image of the patient to a machine learning model to determine a detection probability for the specific lesion, based on the area of the lesion detected in the X-ray image and the detection probability, the lesion and an arithmetic unit for estimating 3D volume information of

The technology described below enables an accurate diagnosis of a patient by calculating volume information of a lesion based on a 2D X-ray image. Based on the diagnostic results, medical staff can determine appropriate treatment guidelines for the patient's condition.

1 is an example of a system for estimating lesion volume information using an X-ray image.

Figure 2 is a result showing the correlation between area, volume and detection probability for thoracic pulmonary nodules.

FIG. 3 is a result of fitting (fttting) the area and detection probability to the volume on a CT image for a thoracic pulmonary nodule.

4 is a result showing the correlation between area and volume in the thoracic pulmonary nodules.

5 is another example of a result of fitting an area and a detection probability to a volume on a CT image.

6 is an example of a process of estimating lesion volume information using an X-ray image.

7 is an example of an analysis device for estimating lesion volume information using an X-ray image.

Since the technology to be described below can have various changes and various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, and it should be understood to include all modifications, equivalents, or substitutes included in the spirit and scope of the technology described below.

Terms such as first, second, A, B, etc. may be used to describe various elements, but the elements are not limited by the above terms, and are merely used to distinguish one element from another. used only as For example, without departing from the scope of the technology described below, a first element may be referred to as a second element, and similarly, the second element may be referred to as a first element. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In terms used in this specification, singular expressions should be understood to include plural expressions unless clearly interpreted differently in context, and terms such as “comprising” refer to the described features, numbers, steps, operations, and components. , parts or combinations thereof, but it should be understood that it does not exclude the possibility of the presence or addition of one or more other features or numbers, step-action components, parts or combinations thereof.

Prior to a detailed description of the drawings, it is to be clarified that the classification of components in the present specification is merely a classification for each main function in charge of each component. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function. In addition, each component to be described below may additionally perform some or all of the functions of other components in addition to its main function, and some of the main functions of each component may be performed by other components. Of course, it may be dedicated and performed by .

In addition, in performing a method or method of operation, each process constituting the method may occur in a different order from the specified order unless a specific order is clearly described in context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

A technique to be described below is a technique of estimating information on the size of a lesion of a subject using a medical image. Medical images may be various, such as X-ray images, ultrasound images, CT (Computer Tomography) images, MRI (Magnetic Resonance Imaging) images, and the like. However, the technique described below estimates the size of a lesion using a 2D medical image. Accordingly, the following medical image may be any one of various medical images in a 2D format. For example, it may be an X-ray image or a 2D composite image extracted from a 3D medical image. However, for convenience of explanation, description will be made focusing on X-ray images.

X-ray images may be images of various body parts according to the type of disease. The lesion corresponds to a lesion whose shade increases with disease progression. Hereinafter, for convenience of explanation, the target lesion will be described focusing on the pulmonary nodule. That is, the following X-ray image corresponds to a chest X-ray image.

The technology to be described below uses a machine learning model that receives an X-ray image and outputs a probability of occurrence of a specific lesion for lesion detection probability. A machine learning model can be one of many types, including decision trees, random forests, K-nearest neighbors (KNNs), Naive Bayes, support vector machines (SVMs), and artificial neural networks (ANNs). can

Hereinafter, it will be described that the analysis device estimates volumetric information of a lesion from an X-ray image. The analysis device may be implemented with various devices capable of processing data. For example, the analysis device may be implemented as a PC, a server on a network, a smart device, a chipset in which a dedicated program is embedded, and the like.

The volume information may be a lesion volume in 3D space or information having a certain correlation (proportional relationship) with the lesion volume. That is, the following volume information may be the volume of the lesion itself or may be information (value) capable of quantifying the size of the lesion even if it is not an exact volume.

1 is an example of a system 100 for estimating lesion volume information using an X-ray image. 1 illustrates an example in which the analysis device is a computer terminal 130 and a server 140 .

The X-ray equipment 110 generates a chest X-ray image of a subject. A chest X-ray image of a subject may be stored in an Electronic Medical Record (EMR) 120.

In FIG. 1 , the user A may use the computer terminal 130 to calculate the volume information of the lesion using an X-ray image of the subject's chest. The computer terminal 130 receives an X-ray image of the chest of the subject. The computer terminal 130 may receive a chest X-ray image from the X-ray equipment 110 or the EMR 120 through a wired or wireless network. In some cases, the computer terminal 130 may be a device physically connected to the X-ray equipment 110 .

The computer terminal 130 receives the 2D area of the lesion determined based on the subject's chest X-ray image. Alternatively, the computer terminal 130 may determine the 2D area of the lesion using a chest X-ray image of the subject.

The computer terminal 130 calculates a detection probability of a lesion by inputting the chest X-ray image to a pre-learned machine learning model. The machine learning model is a model that calculates the probability of occurrence of a specific lesion (pulmonary nodule) in an X-ray image. The machine learning model must be trained in a supervised learning method in advance. If a pulmonary nodule is a target lesion, the machine learning model corresponds to a model that calculates a classification result (probability) as to whether or not a pulmonary nodule exists in the X-ray image when an X-ray image is input. The machine learning model can be pre-learned specifically for a specific lesion.

The computer terminal 130 may estimate the volume information of the lesion using the lesion area in the X-ray image and the detection probability described above. User A can check the analysis result.

The server 140 may receive an X-ray image of the subject's chest from the X-ray equipment 110 or the EMR 120 . The server 140 may receive the 2D area of the lesion determined based on the subject's chest X-ray image. In this case, the server 140 may receive information on the area of the lesion from the terminal used by the medical staff. Alternatively, the server 140 may determine the 2D area of the lesion using a chest X-ray image of the subject.

The server 140 calculates a detection probability of a lesion by inputting the chest X-ray image to a pre-learned machine learning model. The machine learning model is a model that calculates the probability of occurrence of a specific lesion (pulmonary nodule) in an X-ray image.

The server 140 may estimate the volume information of the lesion using the lesion area in the X-ray image and the detection probability described above. The server 140 may transmit the analysis result to the terminal of user A. User A can check the analysis result.

The computer terminal 130 and/or the server 140 may store the analysis result in the EMR 120.

The researcher used X-ray images to determine volume information, and used detection probability information calculated by a machine learning model (AI model) in X-ray images. The researcher hypothesized that if the machine learning model has a high probability of detecting a pulmonary nodule in an X-ray image, it is a clear decision, and a clear nodule is a dense or large nodule after all, and verified it.

The researcher collected image information acquired at regular time intervals for patients with pulmonary nodules. The researcher utilized the image information of a total of 315 patients at the affiliated medical institution. In this process, all patient information and image identification information were anonymized to protect personal information. Of these, 72 subjects with changes in the size of pulmonary nodules over a certain period of time and who had both X-ray and CT images were selected and studied. The results derived from the research process and the results of regression analysis are described below.

Figure 2 is a result showing the correlation between area, volume and detection probability for thoracic pulmonary nodules. Figure 2 shows the results of linear regression and non-linear regression (quadratic). Some figures also show Root Mean Square Error (RMSE) values for each regression analysis. The researcher determined the area of the nodule in the X-ray image using a CAD program.

2(A) is a result showing the correlation between the area of the nodule on the X-ray image and the volume on the CT. Referring to FIG. 2(A), it can be seen that the area and the volume are somewhat correlated with a correlation coefficient of 0.58, and the difference between the linear regression and the nonlinear regression is not large.

2(B) is a result showing the correlation between the average detection probability (Prob.Mean) of a nodule in an X-ray image and the volume on a CT image. Here, the detection probability is a probability value that a pre-learned machine learning model receives and outputs an X-ray image. The researcher calculated the average value of the detection probability. That is, the detection probability was calculated several times for the same X-ray image and the average value was used as the detection probability. In FIG. 2(B), the correlation coefficient between the detection probability and the volume on the CT showed a low correlation of 0.22.

2(C) is a result showing the correlation between the area of the nodule and the average detection probability of the nodule in the X-ray image. Referring to FIG. 2(C), the correlation coefficient between the area of the nodule and the average detection probability of the nodule was 0.73, showing a high correlation. In addition, the nodule area and average detection probability of crystals showed high linearity.

3 shows the result of fitting the area and detection probability to the volume on CT for the thoracic pulmonary nodule. 3 shows the result of fitting the nodule area and detection probability to a 2D plane using linear regression with respect to the CT image volume. 3(A), 3(B) and 3(C) show the same result at different angles. The 2D plane equation was set as "14886.637214 + (29.288656) * (area - (57206.109259)) * detection probability = volume". The CT volume is relatively large, and points raised above the plane are observed (circled area in Fig. 2(C)). However, looking at FIG. 3 , it can be seen that the area and detection probability of the nodule are generally located on a plane with respect to the volume of the CT image. Therefore, it can be seen that the “nodule area and detection probability of the X-ray image” has a certain correlation with the volume on the CT image.

4 is a result showing the correlation between the area and the volume of the thoracic pulmonary nodule. 4 corresponds to the result of linear fitting to the CT volume with only the area. FIG. 4 is an example in which a model-based non-linear regression result (order 1.5) is added for the correlation between area and volume for the thoracic pulmonary nodule shown in FIG. 2(A). In model-based nonlinear regression, the model establishes the relationship between volume and area as "CT volume ∝ α * area ^0.5 + β * area + γ * area ^1.5 + δ". Referring to FIG. 4, linear regression and nonlinear regression have similar RMSEs of 9445.95 and 9449.88, respectively, but model-based nonlinear regression has RMSE of 9395.73, which is slightly lower than linear regression or nonlinear regression.

5 is another example of a result of fitting an area and a detection probability to a volume on a CT image. Unlike FIG. 4 , FIG. 4 is a result of 2D fitting the area of the nodule and the probability of detecting the nodule to the volume of the CT image. 5(A) is a result of 2D fitting through linear regression, FIG. 5(B) is a result of 2D fitting through nonlinear regression, and FIG. 5(C) is a result of 2D fitting through model-based nonlinear regression.

As a result of fitting the linear regression in FIG. 4, the RMSE was 9445.95, but looking at FIG. 5(A), the RMSE improved to 8781.24. As a result of fitting the nonlinear regression in FIG. 4, the RMSE was 9449.88, but the result improved to 8351.0 in FIG. 5(B). In addition, as a result of fitting the model-based nonlinear regression in FIG. 4, the RMSE was 9395.73, but in FIG. 5(A), the RMSE was improved to 7975.55.

Referring to FIGS. 4 and 5 , it can be seen that the volume on the CT image is matched when the detection probability is used together with the lesion area, rather than when only the lesion area of the X-ray image is used. That is, information on the crystal volume can be more accurately estimated by using information such as the area of the nodule and the probability of detecting the nodule.

6 is an example of a process 200 for estimating lesion volume information using an X-ray image.

First, the analysis device receives an X-ray image of the patient captured by the X-ray equipment (210). It is assumed that the X-ray image includes a certain lesion area.

The analysis device determines the lesion area based on the input X-ray image (220). The lesion area may be a value calculated by displaying the lesion area by a medical staff using CAD. In this case, the analysis device may input or receive the lesion area calculated through the CAD program.

Furthermore, the analysis device may directly calculate the lesion area using an X-ray image. For example, the analysis device may classify the lesion area using an image processing technique and calculate the area of the corresponding area. Alternatively, the analysis device may input an X-ray image into a previously learned segmentation model to classify the lesion area and calculate the area of the segmented area. The segmentation model may be implemented with U-net, fully convolutional networks (FCN), and the like.

The analysis device classifies the lesion by inputting the patient's X-ray image to the machine learning model (230). 6 illustrates an artificial neural network such as CNN as a machine learning model. An artificial neural network model trained in advance to classify a specific lesion in an X-ray image calculates a probability value (detection probability) that a lesion exists in the corresponding image when an X-ray image is input.

The analysis device may determine volume information of the lesion using the lesion area and detection probability (240). For example, the analysis device may calculate the volume information by multiplying the detection probability by the lesion area. Alternatively, the analysis device may calculate the volume information using a constant function having the lesion area and the detection probability as variables.

7 is an example of an analysis device 300 for estimating lesion volume information using an X-ray image. The analysis device 300 corresponds to the above-described analysis devices (130 and 140 in FIG. 1). The analysis device 300 may be physically implemented in various forms. For example, the analysis device 300 may have a form of a computer device such as a PC, a network server, and a chipset dedicated to data processing.

The analysis device 300 may include a storage device 310, a memory 320, an arithmetic device 330, an interface device 340, a communication device 350, and an output device 360.

The storage device 310 may store an X-ray image of the patient. As described above, an analysis target is a 2D medical image of a patient. Accordingly, the analysis target may be an image of another type other than an X-ray image.

The storage device 310 may store a machine learning model trained for lesion detection.

The storage device 310 may store a program for constantly pre-processing the X-ray image.

The storage device 310 may store a CAD program for calculating the size of a lesion in an X-ray image and a segmentation model for classifying a lesion area in an X-ray image.

The storage device 310 may store lesion volume information as an analysis result.

The memory 320 includes data and information generated during the preprocessing of the X-ray image by the analysis device 300, the process of determining the area of the lesion, the process of determining the detection probability of the lesion, and the process of calculating the volume information of the lesion. can be saved.

The interface device 340 is a device that receives certain commands and data from the outside. The interface device 340 may receive an X-ray image of the patient from a physically connected input device or external storage device. The interface device 340 may receive an input of the lesion area of the X-ray image from an external device.

The interface device 340 may transmit the analysis result to an external object.

The communication device 350 refers to a component that receives and transmits certain information through a wired or wireless network. The communication device 350 may receive an X-ray image of the patient from an external object. The communication device 350 may receive the lesion area of the X-ray image from an external object. Alternatively, the communication device 350 may transmit the analysis result to an external object such as a user terminal.

Since the interface device 340 and the communication device 350 are configured to send and receive certain data from a user or other physical object, they can also be collectively referred to as input/output devices. If the function of receiving an X-ray image is limited, the interface device 340 and the communication device 350 may be referred to as input devices.

The output device 360 is a device that outputs certain information. The output device 360 may output interfaces and analysis results necessary for data processing.

The arithmetic device 330 may receive an X-ray image and estimate lesion volume information using a machine learning model or program stored in the storage device 310 .

The arithmetic device 330 may pre-process the received X-ray image at a constant level. For example, the arithmetic device 330 may perform tasks such as noise removal and brightness control of an X-ray image.

The arithmetic device 330 may calculate the lesion area from the X-ray image using a CAD program. During this process, the interface device 340 may receive a command to select a lesion area from the user.

The arithmetic device 330 may classify the lesion area by inputting the X-ray image to the segmentation model. The arithmetic device 330 may calculate the area of the divided lesion area.

The arithmetic device 330 may calculate a lesion detection probability by inputting the X-ray image to a pre-learned machine learning model.

The arithmetic device 330 may estimate the volume information using the lesion area and detection probability. For example, the calculator 330 may calculate the volume information by multiplying the lesion area and the detection probability. Alternatively, the calculation device 330 may calculate the volume information through a mathematical calculation process having the lesion area and detection probability as variables.

The arithmetic device 330 may be a device such as a processor, an AP, or a chip in which a program is embedded that processes data and performs certain arithmetic operations.

Also, the method for estimating lesion volume information as described above may be implemented as a program (or application) including an executable algorithm that may be executed on a computer. The program may be stored and provided in a temporary or non-transitory computer readable medium.

A non-transitory readable medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, the various applications or programs described above are CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM) Alternatively, it may be stored and provided in a non-transitory readable medium such as EEPROM (Electrically EPROM) or flash memory.

Temporary readable media include static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), and enhanced SDRAM (Enhanced SDRAM). SDRAM, ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM) and Direct Rambus RAM (DRRAM).

This embodiment and the drawings accompanying this specification clearly represent only a part of the technical idea included in the foregoing technology, and those skilled in the art can easily understand it within the scope of the technical idea included in the specification and drawings of the above technology. It will be obvious that all variations and specific examples that can be inferred are included in the scope of the above-described technology.

Claims

receiving a 2D X-ray image of the patient by an analysis device;

determining, by the analysis device, a detection probability for a specific lesion by inputting the X-ray image to a machine learning model; and

Estimating, by the analysis device, 3D volume information of the lesion based on the area and the detection probability of the lesion detected in the X-ray image

A method for estimating lesion volume information using X-ray images.
According to claim 1,

The area of the lesion is

It is determined using a computer aided diagnosis (CAD) program for the X-ray image,

A method for estimating lesion volume information using an X-ray image determined by using a segmentation model that receives the X-ray image and divides the lesion area.
According to claim 1,

The analysis device estimates the volume information based on a result of multiplying the detection probability by the area of the lesion.
According to claim 1,

The method of estimating lesion volume information using an X-ray image, wherein the machine learning model is a model learned in advance to receive an X-ray image of a patient and classify the possibility of occurrence of the specific lesion.
an input device for receiving a 2D X-ray image of the patient;

a storage device for storing a pre-learned machine learning model to classify the possibility of occurrence of a specific lesion based on an X-ray image of the subject; and

Inputting the patient's X-ray image into a machine learning model to determine the detection probability for the specific lesion, and estimating 3D volume information of the lesion based on the area and detection probability of the lesion detected in the X-ray image including an arithmetic device that

An analysis device that estimates lesion volume information using X-ray images.
According to claim 5,

wherein the input device estimates lesion volume information using an X-ray image further inputting an area of the specific lesion determined based on the patient's X-ray image.
According to claim 5,

The storage device receives a CAD (computer aided diagnosis) program or an X-ray image and further stores a segmentation model for dividing the lesion area,

wherein the calculator analyzes the X-ray image of the patient using the CAD program or the segmentation model to determine the area of the lesion, and estimates lesion volume information using the X-ray image.
According to claim 5,

wherein the calculator estimates lesion volume information using an X-ray image for estimating the volume based on a result of multiplying the detection probability by the area of the lesion.