KR102227605B1 - Apparatus and method for learning peripheral artery diagnosis image, diagnosis apparatus and method for peripheral lesion artery having significant diffrences using the learning model costructed by the same - Google Patents

Abstract

According to the present invention, provided is a device for diagnosing peripheral artery disease based on artificial intelligence. An artificial intelligence-based peripheral arterial stenosis diagnosis device includes: a data input unit for inputting an image of a peripheral artery based on an angiographic image of a diagnostic area including the peripheral artery; and an interpolated image learning model which includes a learning-based sub-peripheral artery image based on a change in motion of the peripheral artery image acquired based on a predetermined time unit; a peripheral artery image interpolator for generating the sub-peripheral artery image corresponding to the peripheral artery image by using the interpolated image learning model; and a peripheral arterial image analyzer which analyzes significant peripheral arterial disease by checking information provided by the data input unit and the peripheral arterial image interpolator.

Description

Peripheral artery diagnostic image learning device and method, and a significant peripheral artery disease diagnosis device and method using the learning model built through the learning device and method TECHNICAL FIELD LESION ARTERY HAVING SIGNIFICANT DIFFRENCES USING THE LEARNING MODEL COSTRUCTED BY THE SAME}

The present disclosure relates to a deep learning model learning technology, and more specifically, a method and apparatus for learning a peripheral artery diagnostic image based on a medical image, and a learning model built based on a medical image. A method and apparatus for diagnosing peripheral arterial disease are described.

Deep learning is to learn a very large amount of data, and when new data is input, the highest probability is selected based on the learning result. Such deep learning can operate adaptively according to an image, and since it automatically finds a characteristic factor in the process of learning a model based on data, there are increasing attempts to utilize this in the field of artificial intelligence in recent years.

Angiography is used as a method of analyzing peripheral arterial disease, but there is a problem in that the peripheral arterial disease in an actual patient cannot be accurately diagnosed due to the fundamental limitations of angiography when the lesion is evaluated only by angiography. In particular, high-speed x-ray angiography is required in order to accurately analyze peripheral arterial disease, but there is a problem in that it is difficult to apply it due to a radiation exposure problem that occurs during imaging of x-ray angiography.

An object of the present disclosure is to provide a method and apparatus for constructing a peripheral arterial diagnosis image using a learning model, and determining peripheral arterial disease based on this.

Another technical problem of the present disclosure is to provide a method and apparatus for constructing a learning model capable of constructing a highly reliable peripheral artery diagnostic image.

Another technical problem of the present disclosure is to provide a method and apparatus for more accurately measuring the rate of filling or washing of a contrast agent in a diagnostic image of a peripheral artery.

Another technical problem of the present disclosure is to provide a method and apparatus for confirming a state of reperfusion after a procedure based on a rate at which a contrast agent is filled or washed out in a peripheral artery diagnostic image.

Another technical problem of the present disclosure is to provide a method and apparatus for verifying prognosis predictive power based on a rate at which a contrast agent is filled or washed out in a peripheral artery diagnostic image.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present disclosure belongs from the following description. I will be able to.

According to an aspect of the present disclosure, an apparatus for diagnosing peripheral arterial stenosis based on artificial intelligence may be provided. The apparatus includes a data input unit for inputting a peripheral artery image based on an angiographic image of a diagnostic area including the peripheral artery, and a learning-based learning based on a motion change of the peripheral artery image acquired based on a predetermined time unit. A peripheral artery image interpolation unit having an interpolation image learning model constituting a sub peripheral artery image, and generating a sub peripheral artery image corresponding to the peripheral artery image using the interpolation image learning model, the data input unit and the peripheral artery It may include a peripheral artery image analysis unit for analyzing significant peripheral artery disease by checking information provided from the image interpolation unit.

According to another aspect of the present disclosure, a method of diagnosing peripheral arterial disease based on artificial intelligence may be provided. The method includes a process of learning an interpolated image learning model constituting a learning-based sub-peripheral artery image based on a change in motion of a peripheral artery image acquired based on a predetermined time unit, and the interpolation image learning model using the interpolated image learning model. A process of generating a learning-based sub-peripheral artery image corresponding to a peripheral artery image, and a process of analyzing a significant peripheral artery disease by checking the peripheral artery image and the learning-based sub-peripheral artery image.

Features briefly summarized above with respect to the present disclosure are only exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, a method and apparatus for constructing a peripheral arterial diagnosis image using a learning model and diagnosing peripheral arterial disease based on this may be provided.

In addition, according to the present disclosure, a method and an apparatus for accurately determining peripheral arterial disease using a peripheral arterial diagnosis image configured based on a learning model may be provided.

In addition, according to the present disclosure, a method and apparatus for constructing a learning model capable of constructing a highly reliable diagnostic image of a peripheral artery may be provided.

In addition, according to the present disclosure, a method and apparatus for more accurately measuring the rate at which a contrast agent is filled or washed out in a peripheral artery diagnostic image may be provided.

In addition, according to the present disclosure, a method and apparatus for checking a state of reperfusion after a procedure may be provided based on a rate at which a contrast agent is filled or washed out in a peripheral artery diagnostic image.

In addition, according to the present disclosure, a method and apparatus for verifying prognosis predictive power may be provided based on a rate at which a contrast agent is filled or washed out in a peripheral artery diagnostic image.

The effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art from the following description. will be.

1 is a block diagram showing the configuration of a peripheral artery diagnostic image learning apparatus according to an embodiment of the present disclosure.
2A is a diagram illustrating a learning data set used in a peripheral artery diagnostic image learning apparatus according to an embodiment of the present disclosure.
2B is a diagram illustrating a relationship between a peripheral artery image and a sub-peripheral artery image used in a peripheral artery diagnostic image learning apparatus according to an embodiment of the present disclosure.
3 is a diagram illustrating a structure of an interpolation image learning model provided in the peripheral artery diagnostic image learning apparatus according to an embodiment of the present disclosure.
4 is a block diagram illustrating a configuration of an apparatus for diagnosing peripheral arterial disease according to an embodiment of the present disclosure.
5 is a flowchart illustrating a procedure of a method for learning a diagnostic image of a peripheral artery according to an exemplary embodiment of the present disclosure.
6 is a flowchart illustrating a procedure of a method for diagnosing peripheral arterial disease according to an embodiment of the present disclosure.
7 is a block diagram illustrating an apparatus and method for learning peripheral artery diagnostic images and a computing system that executes the apparatus and method for diagnosing peripheral artery diseases according to an exemplary embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the embodiments. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

In describing an embodiment of the present disclosure, when it is determined that a detailed description of a known configuration or function may obscure the subject matter of the present disclosure, a detailed description thereof will be omitted. In addition, parts not related to the description of the present disclosure in the drawings are omitted, and similar reference numerals are attached to similar parts.

In the present disclosure, when a component is said to be "connected", "coupled" or "connected" with another component, it is not only a direct connection relationship, but also an indirect connection relationship in which another component exists in the middle. It may also include. In addition, when a certain component "includes" or "have" another component, it means that other components may be further included rather than excluding other components unless otherwise stated. .

In the present disclosure, terms such as first and second are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise noted. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment is referred to as a first component in another embodiment. It can also be called.

In the present disclosure, components that are distinguished from each other are intended to clearly describe each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated into one hardware or software unit, or one component may be distributed to form a plurality of hardware or software units. Therefore, even if not stated otherwise, such integrated or distributed embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, an embodiment consisting of a subset of components described in an embodiment is also included in the scope of the present disclosure. In addition, embodiments including other elements in addition to the elements described in the various embodiments are included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a peripheral artery diagnostic image learning apparatus according to an embodiment of the present disclosure.

Referring to FIG. 1, the peripheral artery diagnostic image learning apparatus 100 targets a region (eg, arms, legs, etc.) in which a peripheral arterial exists (hereinafter referred to as a'peripheral vascular region (PAR)'). It may be configured to receive an angiographic image captured by the user and learn an image of a peripheral artery region (hereinafter, referred to as a “peripheral artery image”). Specifically, the peripheral artery diagnostic image learning apparatus 100 may include a peripheral artery image learning unit 110 and an interpolation image learning unit 130.

In an embodiment of the present disclosure, the PAR angiography image is an image obtained by angiography of PAR, and may be a 2D X-ray angiography image. And, the peripheral dong?? The image may be an image in which a region in which a peripheral artery is located is segmented from an angiographic image of a PRA.

The peripheral artery image learning unit 110 is a component that performs learning on a peripheral artery detection learning model (hereinafter referred to as'PA detection learning model') 111, and inputs an angiographic image of PAR. Learning of the PA detection learning model 111 that outputs the peripheral artery image corresponding to may be performed. To this end, the peripheral artery image learning unit 110 may include a PA detection learning control unit 113 that controls learning of a PA detection learning model.

The peripheral artery diagnostic image learning apparatus 100 may prepare a learning data set 200 (refer to FIG. 2A) required for learning by the peripheral artery image learning unit 110 and the interpolation image learning unit 130. The training data set 200 includes PAR angiographic images 201, 202, and 203, and peripheral artery images 211 extracted and segmented from the PAR angiographic images 201, 202, and 203. , 212, 213).

Additionally, PAR angiographic images (201, 202, 203) may be 2D X-ray-based images.Since X-ray-based images are images acquired using radiation, There is a limit to increasing the number of frames. Therefore, there is a limitation in diagnosing peripheral arterial disease using PAR angiographic images (201, 202, 203), and reliability problems in diagnosing diseases based on PAR angiographic images (201, 202, 203) May be raised. The angiographic images 201, 202, and 203 of the PAR may be images configured based on a commonly used number of frames per second, and the interpolation image learning unit 130 includes a peripheral arterial image 211 of a commonly used number of frames per second. , 212, 213), we train the interpolated image learning model to interpolate a relatively larger number of frames. To this end, the training data set 200 is a sub-peripheral artery image 221 acquired at a relatively larger frame rate than the peripheral arterial images 211, 212, 213 based on the angiographic images 201, 202, and 203 of the PAR. , 222, 223). For example, the peripheral artery images 211, 212, 213 based on the angiographic images 201, 202, 203 may be images configured based on 7.5 fps, 10 fps, 15 fps, etc., and the sub peripheral artery images 221, 222, 223) is 30fps, 60fps. It is configured based on 120 fps or the like, and may be an image that exists between the peripheral arterial images 211, 212, and 213 in time (see FIG. 2B).

The PA detection learning control unit 113 inputs PAR angiographic images 201, 202, and 203 into the PA detection learning model 111, and the peripheral artery image ( 211, 212, 213) can be controlled to be provided (see Fig. 3A). Accordingly, the PA detection learning model 111 may be configured as a CNN-based learning model, and output peripheral artery images 211, 212, 213 corresponding to the angiographic images 201, 202, and 203 of the PAR. Can be learned.

In addition, the PA detection learning control unit 113 performs learning of the PA detection learning model 111, and then inputs the angiography images 201, 202, and 203 of the PAR to the PA detection learning model 111, and responds thereto. It is possible to check the output value, and by comparing the determined output value with the values of the peripheral artery images 211, 212, 213 of the training data set 200, it is possible to evaluate the performance of the PA detection learning model 111. In addition, the PA detection learning control unit 113 may tune a variable included in the PA detection learning model 111 to control the performance of the PA detection learning model 111.

Further, the peripheral artery image learning unit 110 may further include a preprocessor 112 that performs preprocessing of the angiographic images 201, 202, and 203 of the PAR input to the PA detection learning model 111. The pre-processing unit 112 includes a BVE processing unit 112a that processes Blood Vessel Enhancement (BVE) for angiography images 201, 202, and 203 of the PAR, and the angiography image subjected to blood vessel enhancement. It may include a binarization processing unit 112b for configuring a binarized image by binarizing it based on a threshold value. Further, the binarization processing unit 112b may actively control a threshold value, which is a criterion for binarization, in consideration of the state and environment of the angiographic image subjected to the blood vessel enhancement process. Specifically, the binarization processing unit 112b may construct a binarized image by binarizing the contrast image processed by the BVE processing unit 112a based on a threshold value. The threshold value, which is the binarization criterion, may be determined by combining the statistical aspects of the angiographic images 201, 202, and 203 of the PAR extracted from the preprocessor 112 with a statistical aspect and a filter. For example, the binarization processing unit 112b may set a threshold value based on a statistical value of a pixel value for an initial image (e.g., an image photographed before contrast medium) of an angiography image 201, 202, 203 of the PAR. have. In addition, the binarization processing unit 112b resets the threshold value by applying a filter such as a Kalman filter that is strong in removing noise, a histogram equalization filter, and the like, and actively sets the threshold value for each image (or frame). Can be set. The binarization processing unit 112b may set a threshold value by reflecting the pixel values of the angiographic images 201, 202, and 203 of the PAR, and may flexibly adjust the binarization with the set threshold. As a result, the binarization processing unit 112b can actively control binarization even when an angiography image 201, 202, 203 of PAR in an environment having different brightness or density is input.

Further, the preprocessor 112 includes a noise removal unit 112c that removes noise from the angiography images 201, 202, and 203 of the PAR, and a data normalization unit that normalizes the angiography image of the PAR from which the noise has been removed ( 112d) may be further included. It is preferable that the noise removal unit 112c and the data normalization unit 112d are provided at the front end of the BVE processing unit 112a.

In addition, the peripheral artery images 211, 212, 213 detected through the PA detection learning model 111 may be separated by recognizing the same blood vessel as different entities, or to a size that is relatively too small to diagnose peripheral artery stenosis. I can. In consideration of this, the peripheral artery image learning unit 110 may further include a post-processing unit 114 for post-processing the peripheral artery images 211, 212, 213 detected through the PA detection learning model 111.

When the peripheral arterial image detected through the PA detection learning model 111 is the same blood vessel object, the post-processing unit 114 may be configured to connect the corresponding image. In addition, when the peripheral artery image detected through the PA detection learning model 111 is configured to have a size relatively smaller than a predetermined size, the post-processing unit 114 may delete the peripheral artery image. Specifically, the post-processing unit 114 removes noise from the peripheral artery image detected by the PA detection learning model 111, maps the same blood vessels detected to be cut off, and detects a peripheral artery image with a size that is relatively smaller than a predetermined size. Delete processing in can be performed. The post-processing unit 114 may remove noise by adjusting a value of artificial intelligence data detected from the PA detection learning model 111. That is, the post-processing unit 114 may remove noise by applying a predetermined filter to image data configured through the PA detection learning model 111. In this case, the filter used for noise removal may be one or a combination of two or more selected from among an average filter, a Gaussian filter, and a median filter. Preferably, the post-processing unit 114 may smooth the image between pixels using an average value filter and a Gaussian filter, and then apply an intermediate value filter to the processed result to remove noise.

The post-processing unit 114 may perform mapping of the same blood vessels, deletion of peripheral artery images, and the like using a morphology operation and connection element labeling. In this case, since the image used is a binary image, a binarization process is required. For example, the post-processing unit 114 may first perform a “closed” operation “erosion after expansion”, which is a “closed” operation, in order to map the same blood vessels, and then process connection element labeling and the like. In this way, the post-processing unit 114 may effectively remove the darkly expressed area (ie, the area expressed close to 0) as noise through the “closed” operation. In addition, the post-processing unit 114 may connect a wide range that is not connected by a “closed” operation using connection element labeling. That is, the post-processing unit 114 can separate the region by assigning the same value to the region where the pixel value is not cut to 0, and further, to the same blood vessel but by giving the same value to the region that is expressed to be disconnected to connect the same blood vessel. And so on.

Furthermore, the post-processing unit 114 can perform a “open” operation to minimize noise of an image generated after mapping and delete it from a peripheral artery image, thereby removing noise and unnecessary areas (eg, A protruding area, a blood vessel area having a size relatively smaller than a predetermined size, etc.) can be removed.

Meanwhile, the interpolation image learning unit 130 is a constituent unit that builds the interpolation image learning model 131, and can perform learning of an interpolated image corresponding to the peripheral artery image provided by the peripheral artery image learning unit 110. have. To this end, the interpolated image learning unit 130 may include an interpolated image learning control unit 133 that controls learning of the interpolated image learning model 131.

The interpolation image learning unit 130 may receive peripheral artery images 211, 212, 213 included in the data set 200 from the peripheral artery image learning unit 110, and the peripheral artery images 211, 212, and 213) can be used as an input of the interpolated image learning model 131, and the above-described sub peripheral artery images 221, 222, 223 can be controlled to be used as a target variable of the interpolated image learning model 131 (FIG. 3B. Reference).

In particular, the interpolation image learning model 131 may be configured as a CNN-based learning model, and detects the motion field using a neural network that detects the motion field of the peripheral artery images 211, 212, 213, and detects the motion field. It may be learned to configure the sub peripheral artery images 221, 222, and 223 based on the field.

The interpolated image learning control unit 133 performs learning of the interpolated image learning model 131, and then inputs the peripheral artery images 211, 212, 213 to the interpolated image learning model 131, and checks an output value corresponding thereto. In addition, the determined output value may be compared with values of the sub peripheral artery images 221, 222, and 223 of the training data set 200 to evaluate the performance of the interpolated image learning model 131. In addition, the interpolated image learning control unit 133 controls the retraining of the interpolated image learning model 131 or tunes a variable included in the interpolated image learning model 131 to control the performance of the interpolated image learning model 131. I can.

In addition, the interpolation image learning unit 130 detects an angiography image (or a peripheral artery image) and a motion field included in the interpolation image, and detects a blood flow amount and a blood flow velocity based on the detected motion field ( 135). The interpolated image learning control unit 133 may control the operation of the blood flow information analysis unit 135, and learn the interpolated image learning model 131 based on the amount of blood flow and the blood flow rate output from the blood flow information analysis unit 135. Or, the performance of the interpolated image learning model 131 may be controlled.

In addition, the interpolation image learning unit 130 is an arithmetic interpolation processing unit 137 that configures an arithmetic-based interpolation image in consideration of the relationship and motion change between the angiographic images (or peripheral arterial images) acquired based on a predetermined time unit. ) May be further included. The arithmetic interpolation processing unit 135 may construct an arithmetic-based interpolation image and provide the (arithmetic-based interpolation image) to the blood flow information analysis unit 135. Based on this, the blood flow information analysis unit 135 may output a blood flow amount and a blood flow velocity calculated based on the arithmetic-based interpolation image. Based on the foregoing, the interpolation image learning control unit 133 calculates the amount of blood flow, the blood flow velocity, etc. calculated based on the arithmetic-based interpolation image, and the sub peripheral artery image 221 used as the objective variable of the interpolation image learning model 131. The performance of the arithmetic interpolation processing unit 135 may be evaluated by comparing 222 and 223, a blood flow amount, a blood flow rate, and the like. Further, the interpolated image learning control unit 133 tunes the variables included in the interpolated image learning model 131 based on the performance of the interpolated image learning model 131 and the arithmetic interpolation processing unit 137 to tune the interpolated image learning model ( 131) performance can be controlled. For example, when the performance of the interpolated image learning model 131 is relatively lower than that of the arithmetic interpolation processing unit 137, the interpolated image learning control unit 133 controls the retraining of the interpolated image learning model 131 or interpolated. The performance of the interpolated image learning model 131 may be controlled by tuning variables included in the image learning model 131. At this time, the performance of the interpolation image learning model 131 and the performance of the arithmetic interpolation processing unit 137 are determined by analyzing the image characteristics of the sub-peripheral artery image and the arithmetic-based interpolation image, or the sub-peripheral artery image and the arithmetic-based interpolated image. It can be determined by analyzing the calculated blood flow rate and blood flow rate.

4 is a block diagram illustrating a configuration of an apparatus for diagnosing peripheral arterial disease according to an embodiment of the present disclosure.

Referring to FIG. 4, the peripheral arterial disease diagnosis apparatus 400 may be configured to receive an angiographic image of a PAR and detect a state of a peripheral arterial disease corresponding thereto. An image interpolation unit 430 and a peripheral artery image analysis unit 450 may be included.

The peripheral artery image detection unit 410 may include a PA detection learning model 411, which may be a learning model constructed by the peripheral artery diagnostic image learning apparatus 100 described above. The PA detection learning model 411 may be configured as a CNN-based learning model, and may be a model trained to output a peripheral artery image corresponding to an angiographic image of the PAR.

The peripheral artery image detection unit 410 may include a preprocessor 412 that performs preprocessing of an input angiographic image. The pre-processing unit 412 may include a BVE processing unit 412a that processes blood vessel enhancement (BVE) for an angiographic image.

Furthermore, the preprocessor 412 may further include a noise removing unit 412b for removing noise from an angiographic image, and a data normalizing unit 412c for normalizing the angiographic image from which noise has been removed. It is preferable that the noise removal unit 412b and the data normalization unit 412c are provided in front of the BVE processing unit 412a.

In addition, the peripheral artery image detection unit 410 may further include a post-processing unit 414. The post-processing unit 414 may detect a center line of a blood vessel region from the peripheral artery image output from the PA detection learning model 411 and output an image obtained by applying the detected center line to the peripheral artery image. In this case, the post-processing unit 414 may perform center line detection based on a minimal path approach.

It is illustrated that the post-processing unit 414 provides an image in which the center line is applied to the peripheral artery image, but the present disclosure is not limited thereto, and may be variously changed by a person having ordinary knowledge in the technical field of the present disclosure. . For example, the post-processing unit 414 may output an image obtained by applying the center line of the blood vessel region to the binarized image by dividing the blood vessel region and the background region.

Additionally, the peripheral arterial image detected through the PA detection learning model 411 may be separated by recognizing the same blood vessel as different entities, or to a size that is relatively too small to diagnose peripheral artery disease. In consideration of this, the post-processing unit 414 may be configured to connect the image when the peripheral artery image detected through the PA detection learning model 411 is the same blood vessel entity. In addition, when the peripheral artery image detected through the PA detection learning model 411 is configured to have a size relatively smaller than a predetermined size, the post-processing unit 414 may delete the peripheral artery image.

The process of merging or deleting the peripheral artery image may be performed on the peripheral artery image output from the PA detection learning model 411, and the center line of the blood vessel region is detected for the image on which the merge or deletion process of the peripheral artery image has been completed. , The detected central line can be applied to the peripheral artery image.

Additionally, the peripheral artery image detection unit 410 may further include a 3D image reconstruction unit 417 that combines at least one peripheral artery image output through the post-processing unit 414 to form a 3D peripheral artery image. have. In this case, the 3D image reconstruction unit 417 may configure a 3D image by combining at least one peripheral artery image based on a center line of the peripheral artery image. In addition, it is possible to configure and output a 3D interpolated image based on a 2D interpolated image.

Meanwhile, the peripheral artery image interpolation unit 430 may include an interpolation image learning model 431, which may be a learning model constructed by the aforementioned peripheral artery diagnostic image learning apparatus 100. . The interpolated image learning model 431 may be configured as a CNN-based learning model, and may generate a sub-peripheral artery image corresponding to the peripheral artery image. As described above, the peripheral artery image input to the interpolation image learning model 431 may be a frame image generated based on the first frame rate, and the sub peripheral artery image is an image generated based on the second frame rate. The second frame rate may be set to a relatively larger value than the first frame rate. For example, if the peripheral artery image is a frame image generated based on 7.5fps, 10fps, 15fps, etc., the sub peripheral artery image is 30fps, 60fps. It is generated based on 120 fps or the like, and may be a frame image that exists between peripheral artery images in time. Although, in the embodiment of the present disclosure, the frame configuration of the peripheral artery image and the sub-peripheral artery image is illustrated, the present disclosure is not limited thereto, and the frame acquisition speed of the peripheral artery image and the sub-peripheral artery image may be variously changed. I can.

In particular, the interpolation image learning model 431 may be configured as a CNN-based learning model, detects a motion field of a peripheral artery image, and constructs a sub-peripheral artery image based on the detected motion field.

In this case, the motion tracking unit 435 may track the start point and the end point of the blood vessel region (or a pixel corresponding thereto) based on a motion tracking technique for tracking the motion of an object included in the image.

Additionally, the peripheral artery image interpolation unit 430 may further include an interpolation control unit 437 that analyzes at least one sub-peripheral artery image and controls interpolation of the sub-peripheral artery image based on whether or not the contrast medium is filled. . For example, the interpolation control unit 437 checks whether the peripheral artery area or the sub-peripheral artery area is sufficiently filled with a contrast agent, and when the contrast agent is not sufficiently filled in the pixels of the corresponding area, the interpolation image learning model 431 The primaryly outputted sub-peripheral artery image may be controlled to be provided as an input of the interpolated image learning model 431 again.

Accordingly, the interpolation image learning model 431 is a secondary sub peripheral artery constructed by re-interpolating the primaryly output sub peripheral artery image (e.g., a primary sub peripheral artery image (e.g., an image interpolated based on 30 fps)). An image (eg, an image interpolated at 60 fps mood) may be configured and output, and post-processing and motion tracking may be performed on the secondary sub-peripheral artery image.

Furthermore, the interpolation control unit 437 includes an interpolation image learning model 431, and a motion tracking unit 435 so that the above-described re-interpolation, post-processing, and motion tracking are performed until the pixels of the target area are sufficiently filled with the contrast agent. Can control the operation of

Meanwhile, the peripheral artery image analysis unit 450 may check information provided from the peripheral artery image interpolation unit 430 and analyze peripheral arterial disease based on the information.

For example, the peripheral artery image analysis unit 450 may include a PFC analysis unit 451 and a PCFC analysis unit 452.

The PFC analyzer 451 may check a blood flow rate or a blood flow rate using the peripheral artery image and the sub peripheral artery image, and calculate the flow rate at a desired point using a peripheral artery frame count (PFC) analysis. In addition, the PFC analysis unit 451 may analyze the peripheral artery image and the sub-peripheral artery image to analyze a time required to reach a specific boundary region of the distal portion of the same peripheral artery from the time when the contrast agent is introduced into the peripheral artery. For example, when a contrast agent is injected into a blood vessel, a change in brightness within the blood vessel may occur in an angiographic image, and the PFC analysis unit 451 may check the filling time of the contrast agent in consideration of this. That is, the PFC analyzer 451 may check the change in brightness of the reference points based on the reference points on the center line of the blood vessel, and check the time when the contrast medium is filled between the reference points based on this (change in brightness of the reference points). In this case, the PFC analyzer 451 may calculate a time for filling the contrast medium by counting frames in which the brightness change of the reference points occurs, that is, a frame of a peripheral artery image and a frame of a sub-peripheral artery image.

Furthermore, in the case of diabetic patients dialysis with blood, there is a difference in the resistance of microvessels, thus exhibiting a PFC characteristic different from that of the general patient group. In consideration of this, it is necessary to divide peripheral arterial disease irrespective of the general patient group and the patient group having a specific disease. The PCFC analysis unit 452 may perform a peripheral artery clearance frame count (PCFC) to analyze a time when the contrast medium is washed out from the peripheral artery region.

Further, the PFC analysis unit 451 or the PCFC analysis unit 452 may analyze peripheral arterial disease in consideration of an index for confirming a blood vessel state such as an Ankle-Brachial Index (ABI) or Toe Brachial Index (TBI). . For example, the PFC analysis unit 451 or the PCFC analysis unit 452 checks PFC, PCFC, ABI, TBI, etc. before and after treatment for peripheral artery disease, and based on this, the reperfusion state or prognostic power after the procedure. can confirm.

5 is a flowchart illustrating a procedure of a method for learning a diagnostic image of a peripheral artery according to an exemplary embodiment of the present disclosure.

The peripheral artery diagnostic image learning method according to an embodiment of the present disclosure may be performed by the peripheral artery diagnostic image learning apparatus described above.

Referring to FIG. 5, first, in step S500, the peripheral arterial diagnostic image learning apparatus may prepare a training data set 200 (refer to FIG. 2A) required for learning the interpolated image learning model 131. The peripheral arterial images 201, 202, and 203 may be 2D X-ray based images.Since X-ray based images are images acquired using radiation, it is difficult to increase the number of frames per second due to the radiation exposure problem of the patient. There is a limit. Therefore, there is a limitation in diagnosing peripheral arterial disease using peripheral arterial images (201, 202, 203), and a reliability problem for diagnosing disease based on vascular peripheral angiography images (201, 202, 203) is raised. It is also possible. The peripheral arterial images 201, 202, and 203 may be images configured based on a commonly used number of frames per second, and the peripheral arterial diagnostic image learning apparatus includes a commonly used peripheral arterial image 201, 202, and the number of frames per second. 203), the interpolation image learning model is trained to interpolate a relatively larger number of frames. To this end, the training data set 200 may include sub-peripheral artery images 221, 222, and 223 acquired at a relatively larger frame rate than the peripheral artery images 201, 202, and 203. For example, the peripheral artery images 201, 202, and 203 may be images configured based on 7.5 fps, 10 fps, 15 fps, etc., and the sub peripheral artery images 221, 222, 223 are 30 fps, 60 fps. It is configured based on 120 fps or the like, and may be an image that exists between the peripheral arterial images 201, 202, and 203 in time.

In step S510, the peripheral artery diagnostic image learning apparatus may perform pre-processing of the peripheral artery images 201, 202, and 203 (refer to FIG. 2). Specifically, the peripheral artery diagnostic image learning apparatus may process blood vessel enhancement (BVE) for the peripheral artery images 201, 202, and 203 (S511c). In addition, the peripheral artery diagnostic image learning apparatus may construct a binarized image by binarizing the peripheral artery image processed for strengthening blood vessels based on a predetermined threshold (S511d). In this case, the peripheral artery diagnostic image learning apparatus may actively control a threshold value, which is a criterion for binarization, in consideration of a state and environment of an angiographic image subjected to blood vessel enhancement. Specifically, the peripheral artery diagnostic image learning apparatus may construct a binarized image by binarizing the contrast image processed for enhancement of blood vessels based on a threshold value. The threshold value serving as the binarization criterion may be determined by combining a statistical aspect of the peripheral artery images 201, 202, and 203 with a filter. For example, the peripheral arterial diagnostic image learning apparatus may set a threshold value based on a statistical value of a pixel value for an initial image (eg, an image taken before contrast medium is not injected) of the peripheral artery images 201, 202, and 203. . In addition, the peripheral artery diagnostic image learning device resets the threshold by applying a filter such as a Kalman filter that is strong in removing noise and a histogram equalization filter, and actively thresholds each image (or frame). You can set the value. The peripheral artery diagnostic image learning apparatus may set a threshold value by reflecting the pixel values of the peripheral artery images 201, 202, and 203, and may flexibly adjust the binarization by using the set threshold value as a mood. As a result, the peripheral artery diagnostic image learning apparatus can actively control the binarization even if peripheral artery images 201, 202, and 203 in an environment having different brightness or concentration are input.

Further, the peripheral artery diagnostic image learning apparatus is a step of removing noise from the angiographic images 201, 202, and 203, prior to performing blood vessel strengthening (BVE) (S510c) and binarization (S510d) when performing preprocessing. In step S510a, normalizing the angiographic image from which noise has been removed (S510b) may be further processed.

Meanwhile, the peripheral artery diagnostic image learning apparatus may learn an interpolated image corresponding to the peripheral artery image (S520). To this end, the peripheral artery diagnostic image learning apparatus may receive peripheral artery images 201, 202, and 203 included in the data set 200, and use the peripheral artery images 201, 202, and 203 as an interpolation image learning model ( 131), and the above-described sub-peripheral artery images 221, 222, 223 may be controlled to be used as a target variable of the interpolated image learning model 131 (see FIG. 3B).

In particular, the interpolation image learning model 131 may be composed of a CNN-based learning model, and detects a motion field using a neural network that detects the motion field of the peripheral artery images 201, 202, and 203, and detects the motion field. It may be learned to configure the sub peripheral artery images 221, 222, and 223 based on the field.

The peripheral artery diagnostic image learning apparatus performs learning of the interpolated image learning model 131, and then inputs the peripheral artery image into the interpolated image learning model 131, and can check the output value corresponding thereto, and learn the confirmed output value. By comparing the values of the sub-peripheral artery images 221, 222, and 223 of the data set 200, the performance of the interpolated image learning model 131 may be evaluated (S530). In addition, the peripheral artery diagnostic image learning apparatus may control the retraining of the interpolated image learning model 131 or tune the variables included in the interpolated image learning model 131 to control the performance of the interpolated image learning model 131. Yes (S540).

6 is a flowchart illustrating a procedure of a method for diagnosing peripheral arterial disease according to an embodiment of the present disclosure.

A method for diagnosing peripheral arterial disease according to an exemplary embodiment of the present disclosure may be performed by the aforementioned apparatus for diagnosing peripheral arterial disease.

First, the peripheral arterial disease diagnosis apparatus may be prepared by constructing an interpolated image learning model (S600).

Next, the peripheral arterial disease diagnosis apparatus may perform pre-processing of the angiographic image of the input PAR (S610). For example, the apparatus for diagnosing peripheral arterial disease may include processing a blood vessel enhancement (BVE) for an angiographic image of a PAR (S613).

Furthermore, the apparatus for diagnosing peripheral arterial disease may further include removing noise from an angiographic image of the PAR (S611) and normalizing the angiographic image from which the noise has been removed (612).

Meanwhile, the peripheral artery disease diagnosis apparatus may include an interpolation image learning model, which may be a learning model constructed by the aforementioned peripheral artery diagnosis image learning apparatus 100. The interpolated image learning model may be composed of a CNN-based learning model, and a sub-peripheral artery image corresponding to the peripheral artery image may be generated. As described above, the peripheral artery image input to the interpolation image learning model may be a frame image generated based on the first frame rate, and the sub-peripheral artery image is an image generated based on the second frame rate. The frame rate may be set to a relatively larger value than the first frame rate. For example, if the peripheral artery image is a frame image generated based on 7.5fps, 10fps, 15fps, etc., the sub peripheral artery image is 30fps, 60fps. It is generated based on 120 fps or the like, and may be a frame image that exists between peripheral artery images in time. Although, in the embodiment of the present disclosure, the frame configuration of the peripheral artery image and the sub-peripheral artery image is illustrated, the present disclosure is not limited thereto, and the frame acquisition speed of the peripheral artery image and the sub-peripheral artery image may be variously changed. I can.

In step S620, the apparatus for diagnosing peripheral arterial disease may input the peripheral artery image to the above-described interpolation image learning model, and configure a sub-peripheral artery image corresponding to this (peripheral artery image).

In this case, the apparatus for diagnosing peripheral arterial disease may track a start point and an end point of a blood vessel region (or a pixel corresponding thereto) based on a motion tracking technique for tracking the movement of an object included in the image (S630).

Additionally, the peripheral arterial disease diagnosis apparatus may configure a 3D peripheral artery image by combining at least one post-processed peripheral artery image (S640). In this case, the apparatus for diagnosing peripheral arterial disease may construct a 3D image by combining at least one peripheral artery image based on a center line of the peripheral artery image. In addition, it is possible to reconstruct a 3D interpolated image based on a 2D interpolated image.

Meanwhile, in step S650, the apparatus for diagnosing peripheral arterial disease may generate a sub-peripheral artery image corresponding to the peripheral arterial image using an interpolated image learning model. The peripheral artery image input to the interpolation image learning model may be a frame image generated based on the first frame rate, and the sub peripheral artery image is an image generated based on the second frame rate, and the second frame rate is the first. It may be set to a relatively larger value than the frame rate. For example, if the peripheral artery image is a frame image generated based on 7.5fps, 10fps, 15fps, etc., the sub peripheral artery image is 30fps, 60fps. It is generated based on 120 fps or the like, and may be a frame image that exists between the peripheral artery images 211, 212, and 213 in time.

Additionally, the apparatus for diagnosing peripheral arterial disease may analyze at least one sub-peripheral artery image and control interpolation of the sub-peripheral artery image based on whether the contrast medium is filled. For example, the peripheral arterial disease diagnosis apparatus checks whether the peripheral artery area or the sub-peripheral artery area is sufficiently filled with a contrast agent (S670), and when the pixel of the corresponding area is not sufficiently filled with the contrast agent (S670-No), The sub-peripheral artery image primarily output from the interpolated image learning model may be controlled to be provided again as an input of the interpolated image learning model (S680).

Accordingly, the interpolation image learning model is a secondary sub-peripheral artery image (e.g., a secondary sub-peripheral artery image (ex. , An image interpolated at 60 fps mood) may be configured and output, and post-processing and motion tracking may be performed on the secondary sub-peripheral artery image.

Further, the peripheral arterial disease diagnosis apparatus may control an operation such that the above-described re-interpolation, post-processing, and motion tracking are performed until the pixels of the target region are sufficiently filled with a contrast agent.

Meanwhile, the apparatus for diagnosing peripheral arterial disease may analyze the peripheral arterial disease based on the peripheral arterial image and the sub-peripheral artery image (S690).

For example, the peripheral artery disease diagnosis apparatus can check the blood flow volume or blood flow rate using peripheral artery images and sub-peripheral artery images, and calculate the flow rate at a desired point using PFC (peripheral artery frame count) analysis. have. In addition, the peripheral arterial disease diagnosis apparatus may analyze a peripheral artery image and a sub-peripheral artery image to analyze a time required to reach a specific boundary region of a distal portion of the same peripheral artery from a time point when a contrast agent is introduced into the peripheral artery. For example, when a contrast agent is injected into a blood vessel, a change in brightness in the blood vessel may occur in an angiographic image, and the peripheral arterial disease diagnosis apparatus may check the time to fill the contrast agent in consideration of this. That is, the apparatus for diagnosing peripheral arterial disease may check the change in brightness of the reference points based on the reference points on the center line of the blood vessel, and check the time when the contrast medium is filled between the reference points based on this (change in brightness of the reference points). In this case, the apparatus for diagnosing peripheral arterial disease may calculate a time for filling the contrast medium by counting frames in which brightness changes of reference points occur, that is, a frame of a peripheral artery image and a frame of a sub peripheral artery image.

Furthermore, in the case of diabetic patients dialysis with blood, there is a difference in the resistance of microvessels, thus exhibiting a PFC characteristic different from that of the general patient group. In consideration of this, it is necessary to divide peripheral arterial disease irrespective of the general patient group and the patient group having a specific disease. The apparatus for diagnosing peripheral arterial disease may analyze the time that the contrast agent is washed out from the peripheral artery area by processing PCFC (peripheral artery clearance frame count).

Furthermore, the apparatus for diagnosing peripheral arterial disease may analyze peripheral arterial disease in consideration of an index for confirming a blood vessel state such as an Ankle-Brachial Index (ABI) or Toe Brachial Index (TBI). For example, the apparatus for diagnosing peripheral arterial disease may check PFC, PCFC, ABI, TBI, etc. before and after treatment for peripheral arterial disease, and based on this, it is possible to check the reperfusion state or prognosis after the procedure.

7 is a block diagram illustrating an apparatus and method for learning peripheral artery diagnostic images and a computing system that executes the apparatus and method for diagnosing peripheral artery stenosis according to an embodiment of the present disclosure.

Referring to FIG. 7, the computing system 1000 includes at least one processor 1100 connected through a bus 1200, a memory 1300, a user interface input device 1400, a user interface output device 1500, and a storage device. (1600), and a network interface (1700).

The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and/or the storage 1600. The memory 1300 and the storage 1600 may include various types of volatile or nonvolatile storage media. For example, the memory 1300 may include read only memory (ROM) and random access memory (RAM).

Accordingly, the steps of the method or algorithm described in connection with the embodiments disclosed herein may be directly implemented in hardware executed by the processor 1100, a software module, or a combination of the two. The software module resides in a storage medium (i.e., memory 1300 and/or storage 1600) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM. You may. An exemplary storage medium is coupled to the processor 1100, which is capable of reading information from and writing information to the storage medium. Alternatively, the storage medium may be integral with the processor 1100. The processor and storage media may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and storage medium may reside as separate components within the user terminal.

Although the exemplary methods of the present disclosure are expressed as a series of operations for clarity of description, this is not intended to limit the order in which steps are performed, and each step may be performed simultaneously or in a different order if necessary. In order to implement the method according to the present disclosure, the exemplary steps may include additional steps, other steps may be included excluding some steps, or may include additional other steps excluding some steps.

Various embodiments of the present disclosure are not intended to list all possible combinations, but to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or may be applied in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, or the like.

The scope of the present disclosure is software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or computer, and such software It includes a non-transitory computer-readable medium (non-transitory computer-readable medium) which stores instructions and the like and is executable on a device or a computer.

Claims (12)

In the device for diagnosing peripheral arterial disease,
A data input unit for inputting a peripheral artery image based on an angiographic image of a diagnosis area including the peripheral artery;
It has an interpolation image learning model for constructing a learning-based sub-peripheral artery image based on the motion change of the peripheral artery image acquired based on a predetermined time unit, and corresponds to the peripheral artery image using the interpolated image learning model. A peripheral artery image interpolation unit that generates a sub-peripheral artery image
A peripheral arterial disease diagnosis apparatus based on artificial intelligence, comprising: a peripheral artery image analysis unit for analyzing significant peripheral artery disease by checking information provided from the data input unit and the peripheral artery image interpolation unit. The method of claim 1,
The data input unit,
A BVE processing unit that processes blood vessel enhancement (BVE) for the angiographic image,
A binarization processing unit configured to construct a binarized image by binarizing the angiographic image subjected to a blood vessel enhancement process based on a predetermined threshold value,
A peripheral artery disease diagnosis apparatus based on artificial intelligence, comprising a preprocessing unit that provides the binarized image to a learning model for detecting a Peripheral Artery (PA). The method of claim 2,
The pretreatment unit,
A noise removing unit that removes noise from the angiographic image,
A device for diagnosing peripheral arterial disease based on artificial intelligence, comprising: a data normalization unit for normalizing the angiographic image from which noise has been removed. The method of claim 1,
The data input unit,
The apparatus for diagnosing peripheral arterial diseases based on artificial intelligence, comprising a motion tracking unit configured to track a movement of a peripheral artery in the different peripheral artery images based on at least one reference point included in the different peripheral artery images. The method of claim 1
The peripheral artery image interpolation unit,
The apparatus for diagnosing peripheral arterial disease based on artificial intelligence, comprising estimating a blood flow rate and a blood flow rate based on a change in motion of the sub peripheral artery image corresponding to the peripheral artery image. The method of claim 1,
The interpolated image learning model,
The apparatus for diagnosing peripheral arterial diseases based on artificial intelligence, comprising detecting a motion field of the peripheral artery image and configuring a sub peripheral artery image based on the detected motion field. The method of claim 1,
The peripheral artery image interpolation unit,
In the sub peripheral artery image, it is checked whether or not a contrast agent is filled in the peripheral artery region,
And an interpolation control unit for requesting generation of the sub-peripheral artery image using the interpolated image learning model in consideration of whether the contrast medium is filled. The method of claim 1,
The peripheral artery image analysis unit,
Comprising a PFC (Peripheral Artery Frame Count) analyzer for analyzing the peripheral artery image and the sub-peripheral artery image to analyze the time taken for the contrast agent to reach a specific boundary area of the distal peripheral artery Artificial intelligence-based peripheral artery disease diagnosis apparatus, characterized in that. The method of claim 8,
The peripheral artery image analysis unit,
A peripheral artery disease diagnosis apparatus based on artificial intelligence, further comprising a PCFC (Peripheral Artery Clearance Frame Count) analysis unit that analyzes a time when the contrast agent is washed out from the peripheral artery region. The method of claim 9,
The peripheral artery image analysis unit,
An apparatus for diagnosing peripheral arterial disease based on artificial intelligence, comprising analyzing a state of peripheral arterial disease based on a state in which the contrast agent is washed out from the peripheral artery region and a state in which the contrast agent is filled in the peripheral artery. In a method for analyzing peripheral arterial disease information by an electronic device including at least one processor and a storage medium,
A process in which the electronic device learns an interpolation image learning model constituting a learning-based sub-peripheral artery image based on a motion change of a peripheral artery image acquired based on a predetermined time unit;
The electronic device generates a learning-based sub-peripheral artery image corresponding to the peripheral artery image by using the interpolated image learning model, and
And analyzing, by the electronic device, a significant peripheral artery disease by checking the peripheral artery image and the learning-based sub-peripheral artery image. The method of claim 11,
The process of analyzing the peripheral arterial disease,
Peripheral artery disease information analysis, characterized in that it comprises a process of processing a PCFC (Peripheral Artery Clearance Frame Count) on the peripheral artery image and the sub-peripheral artery image, and analyzing the time at which the contrast agent is washed out from the peripheral artery area. Way.

Images