KR102565381B1 - Real-time Micro Blood Flow Analysis Method based on Artificial Intelligence Technology - Google Patents

Abstract

The present invention relates to a real-time micro blood flow analysis method based on artificial intelligence technology, and more specifically, an ICG image obtained by taking a tissue injected with indocyanin green (ICG) from a patient by a near-infrared camera. An ICG image acquisition step obtained in real time, an ICG image preprocessing step in which the ICG image is preprocessed by a processor, and an ICG pattern displaying a change in ICG fluorescence brightness over time from the preprocessed ICG image is recognized by the processor ICG pattern recognition step, an analysis model generation step in which an analysis model is generated by using a data set including ICG learning images based on artificial intelligence technology by a processor, and a risk level from the ICG pattern using the analysis model by a processor It relates to an artificial intelligence technology-based real-time micro blood flow analysis method comprising an analysis result output step in which an analysis result including is output.

Description

Real-time Micro Blood Flow Analysis Method based on Artificial Intelligence Technology}

The present invention relates to a real-time micro blood flow analysis method based on artificial intelligence technology, and more specifically, angiography image using indocyanine green (ICG) is analyzed based on artificial intelligence technology, and then the state of real-time micro blood flow is confirmed. It relates to a real-time micro blood flow analysis method based on artificial intelligence technology.

Anastomosis complications may occur due to poor perfusion of the anastomotic area during colorectal cancer surgery. Among anastomosis complications, acute hypoperfusion can lead to anastomotic leakage or necrosis, and in severe cases, death can result from sepsis or multiple organ failure.

Anastomosis complications were predicted by analyzing micro blood flow using angiography using Indocyanine Green (ICG). However, until now, it has been evaluated subjectively through the naked eye of a surgeon with sufficient experience or analyzed using some quantitative parameters.

Related Document 1 relates to an apparatus and method for analyzing cerebral blood flow using ICG fluorescence dynamic analysis, which receives an electrical signal from an image sensor, analyzes the dynamics of a fluorescence signal observed on the brain surface, and measures cerebral blood flow to generate a blood flow map. can Related Document 2 relates to a tissue perfusion analysis device using indocyanin green blood concentration dynamics and a tissue perfusion analysis method using the same. Depending on the intensity of ICG fluorescence, it is possible to analyze whether the tissue is normal or ischemic,

However, related documents 1 and 2 have limitations in not being able to discriminate in detail various near-infrared fluorescence patterns of indocyanine green (ICG) generated by the patient's complex vascular structure and environmental factors. In addition, the analysis method using quantitative parameters takes a considerable amount of time, and real-time micro blood flow analysis during surgery is difficult, so there are limitations in actual clinical application.

Therefore, there is a need for a method that can check the state of micro blood flow in the anastomosis area in real time by analyzing angiography images using indocyanine green (ICG) during surgery.

KR 10-1166165 KR 10-0867977

The present invention is to solve the above problems, to improve the accuracy and consistency of the analysis result of analyzing the state of micro blood flow, and to assist in the judgment of medical staff by grasping the state of micro blood flow in the tissue to be anastomosis during surgery in real time. After an analysis model is created by using a data set containing ICG learning images of patients diagnosed with anastomotic complications based on artificial intelligence technology, AI technology-based real-time fine An object of the present invention is to obtain a blood flow analysis method.

In order to achieve the above object, the artificial intelligence technology-based real-time micro blood flow analysis method of the present invention uses a near-infrared camera to capture an ICG image of a tissue injected with indocyanin green (ICG) for a patient. an ICG image acquisition step obtained in real time; an ICG image pre-processing step in which the ICG image is pre-processed by a processor; an ICG pattern recognition step of recognizing, by a processor, an ICG pattern displaying a change in ICG fluorescence brightness over time from the preprocessed ICG image; An analysis model generation step in which an analysis model is generated by using a data set including an ICG learning image based on artificial intelligence technology by a processor; and an analysis result output step of outputting, by a processor, an analysis result including a degree of risk from the ICG pattern using the analysis model.

As described above, according to the present invention, an analysis model is generated by using a data set including ICG learning images of patients diagnosed with anastomosis complications based on artificial intelligence technology, and then the analysis results including the risk are output from the ICG pattern By having it, there is an effect of improving the accuracy and consistency of the analysis result of analyzing the state of micro blood flow.

In addition, the present invention has an effect of lowering the incidence of anastomosis complications because it is possible to grasp the state of micro blood flow in a tissue to be anastomosis in real time during surgery and assist in the judgment of medical staff.

1 relates to the real-time micro blood flow analysis method based on artificial intelligence technology of the present invention.
2 is a detailed flowchart of the ICG image pre-processing step of the present invention.
3 is a detailed flow chart of the analysis model generation step of the present invention.
4 is a diagram showing ICG local patterns for each region of interest (ROI) according to an embodiment of the present invention.
5 is a diagram showing analysis results (a) according to the present invention and analysis results (b, c, d) according to conventional parameters according to an embodiment of the present invention.
6 is a diagram showing clustering of a plurality of ICG learning patterns according to an embodiment of the present invention.

The terms used in this specification have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art, precedent, or the emergence of new technologies. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this application, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. 1 relates to the real-time micro blood flow analysis method based on artificial intelligence technology of the present invention. 2 is a detailed flowchart of the ICG image pre-processing step (S200) of the present invention. 3 is a detailed flowchart of the analysis model generation step (S400) of the present invention.

Meanwhile, the artificial intelligence technology-based real-time micro blood flow analysis method of the present invention may be implemented by the near-infrared camera 100 and the processor 200. Since the near-infrared camera 100 can capture light with a longer wavelength than visible light, which cannot be captured by the human eye or a general camera, dark green pigment can be captured when ICG fluorescence is expressed. The near-infrared camera 100 may be embodied as an endoscope camera, and may be positioned 5 cm to 15 cm from the large intestine to capture an area into which indocyanine green (ICG) is injected for a predetermined time. The processor 200 may most preferably be a simulator, and may be connected to the near-infrared camera 100 by wire to acquire the ICG image in real time. And the algorithm of the present invention can be implemented and stored in a program.

Referring to FIG. 1, the artificial intelligence technology-based real-time micro blood flow analysis method of the present invention includes an ICG image acquisition step (S100), an ICG image preprocessing step (S200), an ICG pattern recognition step (S300), an analysis model generation step (S400), and and an analysis result output step (S500).

More specifically, in the ICG image acquisition step (S100), the near-infrared camera 100 obtains an ICG image of a tissue injected with indocyanin green (ICG) from a random patient in real time. do.

In general, in order to perform ICG fluorography, indocyanine green (ICG), a contrast agent, is diluted in distilled water and a minimum dose of several mg can be slowly injected into the peripheral vein of a vein. In addition, intravenous indocyanine green (ICG) binds to globulin or albumin in blood vessels and remains in blood vessels, and can serve as a medium that allows blood flow to be visually confirmed. Here, indocyanine green (ICG), a contrast agent, is a dark green pigment and can be easily used in surgical operations to confirm fluorescence expression with little side effects.

That is, in the ICG image acquisition step (S100), the ICG image in which fluorescence expressed by indocyanine green (ICG) is photographed may be obtained through the near-infrared camera 100.

Next, in the ICG image pre-processing step (S200), the ICG image is pre-processed by the processor 200. This is because the near-infrared camera 100 captures not only the tissues around the anastomosis area to be identified, but also background noise and reflection of light sources such as fluorescent lights due to the influence of the surrounding environment. The preprocessing step (S200) is an essential process.

First, in the ICG image preprocessing step (S200), ICG fluorescence extraction is performed through color space conversion based on color (H), saturation (S), and brightness (V) to remove elements such as background and light source reflection. can Most preferably, the ICG image pre-processing step (S200) can be expressed as the following [Equation 1], different threshold values can be applied for each color channel, and only ICG fluorescence can be finally extracted.

Here, I(n,m,k) is the preprocessed ICG image, n and m are resolutions of the ICG image within the range of 0<n<N, 0<m<M according to the photographing resolution N×M, k is the frame rate of the ICG image within the range of 0<k<R _f according to the capturing frame rate (R _f ), and i _r , i _g , and i _b are the red, green, and blue channels of the ICG image, respectively.

More specifically, the ICG image pre-processing step (S200) is based on a Matlab program, and a Gaussian blur operation is applied to the ICG image in the following [Equation 2] to extract only the ICG component It can be.

Here, I is the ICG image obtained from the ICG image acquisition step (S100), is a Gaussian mask with a standard deviation is the ICG image to which Gaussian blur is applied. The symbol '*' is a convolution operator, which is a method of operation.

In general, Gaussian blur is a type of image filter that uses a Gaussian mask. It maintains the low-frequency part of the image and disappears all the high-frequency part, such as the boundary between objects in the image. Make the high frequency part blur. In addition, the Gaussian blur can adjust the degree of blur by adjusting the standard deviation.

Therefore, in the ICG image pre-processing step (S200), only the ICG components can be extracted from the ICG image by blurring high-frequency portions of the near-infrared image using Gaussian blur.

2, the ICG image preprocessing step (S200) includes an HSV extraction step (S210) of extracting color (H), saturation (S), and brightness (V) from the ICG image, and color (H of the ICG image). ), a comparison step (S220) in which saturation (S) is compared with each reference value, the absolute value of the difference between the color (H) of the ICG image and the reference value and the absolute value of the difference between the chroma (S) of the near-infrared image and the reference value are A brightness calculation step (S230) in which a new brightness (V') is calculated if it is equal to or close to 1 within the error range, and the new brightness (V') and the color (H) and chroma (S) of the ICG image are used. It may include a noise removal step (S240) in which noise of the ICG image is removed by doing so.

Accordingly, in the ICG image preprocessing step (S200), noise can be additionally removed so that the extracted ICG components can be more clearly identified from the ICG image.

More specifically, in the HSV extraction step (S210), color (H), saturation (S), and brightness (V) may be extracted from the ICG image by the processor 200. Since the ICG image is RGB-based, it is converted to HSV-based and color (H), saturation (S), and brightness (V) can be extracted. The extracted color (H) may be expressed as I _H , the extracted chroma (S) as I _S , and the extracted lightness (V) as I _V .

Next, in the comparison step (S220), by the processor 200, the color (I _H ) and chroma ( _IS ) extracted from the HSV extraction step (S210) may be compared with respective reference values. The color reference value may be denoted as h _ref , and the chroma reference value may be denoted as s _ref , which may be values preset in the processor 200 . In the comparison step (S220), the absolute value of the difference between the color (I _H ) extracted from the ICG image and the color reference value (h _ref ), the chroma ( _IS ) extracted from the ICG image, and the chroma reference value (s _ref ) The absolute values of the differences in can be compared, respectively.

Here, in the comparison step (S220), if the absolute value of the difference between the color (I _H ) extracted from the ICG image and the color reference value (h _ref ) is equal to or close to 0, the color (I _{H )} extracted from the ICG image ) may be determined to be similar to the color reference value (h _ref ), and if it is equal to or close to 1, the color (I _H ) extracted from the ICG image may be determined to be different from the color reference value (h _ref ).

In this way, in the comparison step (S220), if the absolute value of the difference between the chroma (I _S ) extracted from the ICG image and the chroma reference value (s _ref ) is equal to or close to 0, the chroma (I S ) extracted from the ICG image _S ) is similar to the chroma reference value (s _ref ), and if it is equal to or close to 1, the chroma (I _S ) extracted from the ICG image is different from the chroma reference value (s _ref ).

On the other hand, in the comparison step (S220), the absolute value of the difference between the color (H) of the ICG image and the color reference value and the absolute value of the difference between the chroma (S) of the ICG image and the chroma reference value are both equal to 0 or 0. If it approaches, the ICG image may be output as it is from the ICG image acquisition step (S100). That is, in the comparison step (S220), if each absolute value is equal to or close to 0, it is determined that the absolute value is similar to the reference value, so there is no need to additionally remove noise from the ICG image from which the ICG component is extracted, and accordingly, the brightness The calculation step (S230) and the noise removal step (S240) may not be performed.

Next, in the brightness calculation step (S230), if at least one of the absolute value of the difference between the color of the ICG image and the color reference value and the absolute value of the difference between the chroma of the ICG image and the chroma reference value is equal to or close to 1. A new lightness (V') can be calculated. The new lightness (V') It is represented by , and can be calculated by the following [Equation 3].

here, is the absolute value of the difference between the color (I _H ) extracted from the ICG image and the color reference value (h _ref ), is the absolute value of the difference between the chroma (I _S ) extracted from the ICG image and the chroma reference value (s _ref ).

Next, in the noise removal step (S240), noise of the ICG image may be removed using the new brightness (V'), color (H), and chroma (S) of the ICG image. That is, in the noise removal step (S240), the new brightness calculated from the brightness calculation step (S230) is reversed from the HSV-based image converted to HSV-based in the HSV extraction step (S210). It can be converted back to an RGB-based image using H and the color (I _H ) and saturation (I _S ) of the ICG image. Accordingly, an ICG image in which the noise is removed from the noise removal step (S240) and the extracted ICG components can be more clearly identified can be finally derived.

Next, in the ICG pattern recognition step (S300), the processor 200 recognizes an ICG pattern displaying a change in ICG fluorescence intensity (I) over time (T) from the preprocessed ICG image.

In general, ICG fluorescence brightness (I) is dark when blood circulation is not smooth, and ICG fluorescence brightness (I) is bright when blood circulation is smooth. However, since ICG fluorescence intensity (I) can be affected by many factors, such as tissue thickness, the imaging distance between the near-infrared camera 100 and tissue, and the type of near-infrared light, only ICG fluorescence intensity (I) is used for quantitative evaluation of blood circulation. is difficult In order to solve this problem, the present invention can recognize an ICG pattern displaying a change in ICG fluorescence intensity (I) according to time (T) from the ICG pattern recognition step (S300).

Here, if the ICG image has a certain length, such as 2 to 3 minutes, the delay time (Latency) from when indocyanine green (ICG) is injected to reaching the tissue to be confirmed and expressing the ICG component or ICG fluorescence is can exist That is, the ICG pattern recognition step (S300) is an ICG fluorescence expression time point estimating step (S310) in which the ICG fluorescence expression time point, which is the time point at which ICG fluorescence is expressed after the delay time (Latency), is estimated by the processor 200. can include

In other words, the change in ICG fluorescence intensity (I) obtained during the delay time (Latency) is a meaningless value, and after the delay time (Latency), the ICG component reaches the tissue to be identified and the ICG fluorescence from the time point of the ICG fluorescence expression A change in brightness (I) can be a key factor in analyzing micro blood flow.

More specifically, in the step of estimating the ICG fluorescence expression point (S310), the following [Equation 4] is used so that the background brightness (I _B ) of the ICG image can be estimated, and the ICG fluorescence occurs for several seconds from the start point of the ICG image. An average value for the change in fluorescence brightness (I) can be calculated.

Here, n and m are resolutions of the ICG image within the range of 0<n<N, 0<m<M according to the capturing resolution N×M, and k is 0<k<R according to the capturing frame rate (R _f ) _f is the frame rate of the ICG image within the range, and f _g (k) is an ICG pattern showing a change in ICG fluorescence intensity (I) according to the shooting frame rate (R _f ).

That is, since the change in fluorescence intensity (I) obtained during the delay time (Latency) is a meaningless value, it can be used to estimate the background brightness (I _B ) of the ICG image, and accordingly, the ICG fluorescence intensity (I) A section for obtaining an average value for change may be included in the Latency.

Clinically, the latency may be within a range of 35 seconds to 45 seconds from the start of the ICG pattern, and most preferably may be 40 seconds. Therefore, most preferably, in the step of estimating the ICG fluorescence expression point (S310), the ICG fluorescence brightness (I) for 5 seconds to 15 seconds from the start point of the ICG image can be estimated so that the background brightness (I _B ) of the ICG image can be estimated. An average value for the change in can be calculated.

In addition, in the ICG fluorescence expression point estimation step (S310), a first threshold value may be calculated by inputting the average value calculated from [Equation 4] to the following [Equation 5], and the first threshold value is a reference value. As a result, the average value may be binarized.

In addition, in the step of estimating the time point of ICG fluorescence expression (S310), the ICG fluorescence is expressed through cross-correlation between the edge detection mask (M _e ) and binary f _b (k) to enable noise removal and clear section setting. A time point k _l may be calculated. This is as shown in [Equation 6] below.

Here, it may be the total length l of the edge detection mask M _e , and −1 may be composed of 1/2 pieces and 1 may be composed of 1/2 pieces.

In the step of estimating the ICG fluorescence expression point (S310), it is possible to find the position of a section that changes from a continuous 0 to a continuous 1 value in f _b (k) through [Equation 6], and the ICG fluorescence expression point can be finally estimated.

In addition, the ICG pattern recognition step (S300) is a region of interest setting step (S320) in which a part of the tissue to be identified around the anastomosis is set as one or more regions of interest (ROI) by the processor 200. ) may be further included.

This, as mentioned above, the ICG image is taken together with the tissue other than the tissue to be confirmed around the anastomosis, such as the background, which increases the amount of meaningless calculation and slows down the analysis speed, so it may be difficult to realize real-time analysis. Therefore, it is very important that a part of the tissue to be identified around the anastomosis is set as one or more regions of interest (ROI).

In this case, the size of the region of interest (ROI) may be determined according to the quality of the ICG image. If the ICG image is shaken and darkened by the near-infrared camera 100, the size of an arbitrary region of interest (ROI) may increase on a pixel basis. On the other hand, when brightly photographed by the near-infrared camera 100, the size of an arbitrary region of interest (ROI) may be reduced on a pixel basis.

More specifically, in the step of setting the region of interest (S320), the location captured in the ICG image for a predetermined time from the ICG fluorescence expression time point estimated in the ICG fluorescence expression time point estimation step (S310) is set. An ICG local pattern, which is the amount of change in brightness (I _diff (n, m)) according to time (T) for each star, can be calculated. This can be calculated by the following [Equation 7].

Here, n and m are resolutions of the ICG image within the range of 0<n<N, 0<m<M according to the capturing resolution N×M, and k is 0<k<R according to the capturing frame rate (R _f ) _f is the frame rate of the ICG image within the range, and I _s is the fluorescence intensity at an arbitrary position.

In the step of setting the region of interest (S320), 10% of the maximum value of the brightness change amount (I _diff (n, m)) according to time T may be set as the second threshold value, and the time for each location (T ) may be finally set as the region of _interest ROI. The ICG local pattern may be obtained from the set region of interest (ROI).

4 is a diagram showing ICG local patterns for each region of interest (ROI) according to an embodiment of the present invention. Referring to FIG. 4, most preferably, in the ICG pattern recognition step (S300), ICG local patterns for a plurality of regions of interest (ROIs) extracted from the region of interest setting step (S320) can be recognized respectively.

In addition, in the ICG pattern recognition step (S300), a quantitative parameter may be extracted from the ICG local pattern, which is then calculated by calculating the ICG learning pattern pre-learned in the analysis model and the ICG local pattern from the analysis result output step (S500). The above quantitative parameters may be used to determine similarity.

The quantitative parameters may include T _max , T _1/2max , F _max , time ratio (TR), and rising slope (RS). T _max is an interval in which the maximum variation (F _max ) in the ICG local pattern recognized from an arbitrary region of interest (ROI) is expressed, and T _1/2max is 1/2 of the interval in which the maximum variation (F _max ) is expressed. interval, the maximum change amount (F _max ) is the value obtained by subtracting the minimum fluorescence brightness (I _min ) from the maximum fluorescence brightness (I _max ), and the time ratio (TR) is a value displaying the ratio of T _1/2max to T _max , the rising slope (RS) is a value expressing the ratio of F _max to T _max .

Here, if the state of micro blood flow is analyzed only with the quantitative parameters as in the prior art, there is a disadvantage of not being able to cope with the error of the ICG local pattern, which may be caused by the complex action of shaking and shaking of the near-infrared camera 100 and noise. Therefore, the error rate of the analysis result may be high.

5 is a diagram showing analysis results (a) according to the present invention and analysis results (b, c, d) according to conventional parameters according to an embodiment of the present invention. Referring to FIG. 5, the analysis result (a) generated based on the artificial intelligence technology of the present invention is analyzed in more detail the part analyzed in a steady state in the analysis result (b, c, d) according to the conventional parameter to determine the conventional parameter. In the following analysis results (b, c, d), it can be confirmed that the part analyzed only in the normal state can be distinguished as a normal state and a good state, and the part analyzed as a normal state or good state is analyzed as a part in a dangerous state. . Therefore, the present invention has a remarkable effect of providing more accurate analysis results.

Next, in the analysis model generation step (S400), an analysis model is generated by the processor 200 by using a data set including an ICG learning image based on artificial intelligence technology.

In order to generate the analysis model with high accuracy, the analysis model creation step (S400) includes ICG learning image preprocessing step (S410), ICG learning pattern recognition step (S420), ICG learning pattern set generation step (S430), and labeling step. (S440), an ICG test image analysis step (S450), and an analysis model determination step (S450).

More specifically, referring to FIG. 3, in the ICG training image pre-processing step (S410), the ICG training image may be pre-processed. The ICG learning image may be an image previously acquired from a patient diagnosed with an anastomotic complication or an image previously acquired from a patient not diagnosed with an anastomotic complication. That is, the ICG image of the random patient acquired in the ICG image acquisition step (S100) is data without result values, and the ICG learning image is data including result values. The ICG learning image pre-processing step (S410) is the same as the ICG image pre-processing step (S200), except that the object to be pre-processed is different.

Next, in the ICG learning pattern recognition step (S420), the ICG learning pattern can be recognized from the preprocessed ICG learning image. The ICG learning pattern recognition step (S420) is the same as the ICG pattern recognition step (S300) except for a different object.

Next, in the step of generating the ICG learning pattern set (S430), the ICG learning pattern set may be generated after the ICG learning patterns are clustered based on similarity.

6 is a diagram showing clustering of a plurality of ICG learning patterns according to an embodiment of the present invention. Referring to FIG. 6, N P ICG training images can be obtained from N _P patients diagnosed with anastomosis complications, and _{N c ICG} learning patterns for N _c regions of interest (ROIs) from one ICG training image. this can be created. Here, the ICG learning pattern is an ICG pattern for a region of interest (ROI) in the ICG training image, not an ICG pattern for the entire ICG training image, and may most preferably be an ICG local pattern recognized from the ICG training image.

In addition, in the ICG learning pattern recognition step (S420), N _P × N _c ICG learning patterns can be generated. Since the number of ICG learning pattern sets is relatively small compared to the number of ICG learning patterns, the accuracy of learning when learned is high. If the number of neural network layers is increased to solve this problem, the amount of computation exponentially increases, which may be unsuitable for real-time analysis.

In order to solve this problem, in the present invention, a plurality of ICG learning patterns may be clustered based on similarity from the ICG learning pattern set generating step (S430). For example, 10000 ICG learning patterns may be clustered into 25 ICG learning pattern sets. Most preferably, clustering can be performed based on a Self-Orgnizing Map (SOM) structure, and similarity between the ICG learning patterns is determined using each of the quantitative parameters for the ICG learning patterns. It can be.

At this time, in the ICG learning pattern set generating step (S430), a variance value for an ICG learning pattern included in any ICG learning pattern set is calculated, and if the variance value is within a preset variance value range, the ICG learning pattern It can be determined that clustering accuracy is high. On the other hand, if the variance value exceeds the preset variance value range, it is determined that the clustering accuracy of the ICG learning pattern is low, and the ICG learning pattern set can be regenerated.

For example, among 25 ICG learning pattern sets, variance values for a plurality of ICG learning patterns included in an arbitrary ICG learning pattern set may be calculated. That is, 25 variance values can be calculated from each ICG learning pattern set. In addition, if the variance value is within the preset variance value range, it can be determined that similar ICG learning patterns are properly clustered, and if at least one of the 25 variance values exceeds the preset variance value, the ICG learning pattern set can be regenerated.

In regenerating the ICG learning pattern set, in the generating of the ICG learning pattern set (S430), if the variance value of the ICG learning pattern included in any ICG learning pattern set exceeds a preset variance value, the learning pattern set is generated. Clustering may be repeated by increasing the number. If the number of learning pattern sets is small, the neural network in the preliminary prediction model is underfitting, making it difficult to properly classify patterns. There is a problem that the classification performance is significantly lowered when it is. Therefore, in the ICG learning pattern set generation step (S430), clustering is first performed with a small number of learning pattern sets, and then, when the preset variance value is exceeded, clustering is performed with a preset larger number of learning pattern sets. Thus, the appropriateness of the number of learning pattern sets can be repeatedly determined.

Next, in the labeling step (S440), the ICG learning pattern set may be labeled as one of a normal state, a good state, and a dangerous state. That is, each of the 25 ICG learning pattern sets may be labeled as one of three states.

Describing the criterion for labeling the ICG learning pattern set as one of a normal state, a good state, and a dangerous state, the labeling step (S440) is about the number of ICG learning patterns included in an arbitrary ICG learning pattern set. According to the ratio of the number of ICG learning patterns similar to the ICG learning patterns included in the corresponding ICG learning pattern set among the ICG learning patterns of patients diagnosed with anastomosis complications, a random ICG learning pattern set is selected from among normal, good, and dangerous states. can be classified as one.

For example, a data set including ICG learning images may include ICG learning images for both a patient diagnosed with an anastomotic complication and a patient not diagnosed with an anastomotic complication. That is, 10000 ICG learning patterns can be recognized from a data set including ICG learning images, and among them, 1400 ICG learning patterns of patients diagnosed with anastomosis complications can be recognized. Also, the number of ICG learning pattern sets may be 25. In addition, the number of ICG learning patterns included in a random ICG learning pattern set among 25 may be 384 out of 10000, and the number of ICG learning patterns of patients diagnosed with anastomotic complications may be 10 out of 1400. Accordingly, the ratio may be calculated as a value of about 2% as 10/384.

That is, in the labeling step (S440), if the ratio is 50% or more for any ICG learning pattern set, it may be labeled as a dangerous state, if it is from 10% to less than 50%, it may be labeled as a good state, and if it is less than 10% In this case, it can be labeled as normal.

Next, in the ICG test image analysis step (S450), a preliminary analysis model is generated after learning the labeled ICG learning pattern set, and the ICG test image can be analyzed using the preliminary analysis model. The ICG test image may also be data having a result value in the same way as the learning image.

That is, the ICG test image may be input to the preliminary analysis model after undergoing preprocessing and pattern recognition processes in the same manner as the ICG training image, and result values may be derived through the preliminary analysis model.

Next, in the analysis model determining step (S460), if the accuracy ( _DS ) of the preliminary analysis model is greater than or equal to the preset accuracy (D _PRE ) based on the analysis result of the ICG test image, the preliminary analysis model determines the analysis can be determined by the model. On the other hand, the analysis model determining step (S460) may repeatedly return to the ICG test image analysis step (S450) when the accuracy ( _DS ) of the preliminary analysis model is less than the preset accuracy (D _PRE ). That is, the step of determining the analysis model (S460) is to prove whether or not a meaningful result value can be calculated from the preliminary analysis model.

Next, in the analysis result output step (S500), the processor 200 outputs an analysis result including a risk level from the ICG pattern using the analysis model. Here, the risk level may include normal state, good state, and dangerous state according to the classification of a random ICG learning pattern set from the labeling step (S440) into one of normal state, good state, and dangerous state.

In addition, in the outputting of the analysis result (S500), the risk level according to time for each location of the tissue photographed on the ICG image is displayed as a color map, and the color map may be combined with the ICG image.

Referring to FIG. 5(a), which is the analysis result output from the present invention, the analysis result output step (S500) is set from the region of interest setting step (S320) so that the medical staff can intuitively check the state of micro blood flow during surgery. The risk according to time for each region of interest (ROI) may be displayed as a color map. The color map is most preferably synthesized into a white light image of the ICG image. Further, in the color map, a normal state, a good state, and a dangerous state may be displayed in red, green, and blue for each region of interest (ROI).

Conventionally, if the ICG fluorescence is bright in the ICG image, it can be assumed that the state of micro blood flow is good, and if not, it can be estimated that the state of micro blood flow is poor. However, there is a problem in that it is difficult to distinguish the state of micro blood flow by visually observing subtle changes in ICG fluorescence due to unclear visualization of ICG fluorescence in the ICG image only with the clinical experience of medical staff. That is, the present invention has a remarkable effect of easily distinguishing microscopic blood flow conditions that are difficult to visually observe in real time.

In addition, in the case of conventional methods of analyzing perfusion based on various parameters for quantitative evaluation, analysis results according to quantitative parameters due to differences in the anatomical structure of blood vessels or noise in the ICG image captured by the NIR camera 100 While may be different every time, the present invention can further improve accuracy and consistency by analyzing ICG images in real time based on artificial intelligence technology, and has a remarkable effect of assisting medical staff in diagnosing the anastomosis during surgery.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

100.. NIR camera
200.. processor

Claims (8)

an ICG image acquisition step in which an ICG image of a tissue injected with indocyanin green (ICG) of a patient is acquired in real time by a near-infrared camera;
an ICG image pre-processing step in which the ICG image is pre-processed by a processor;
an ICG pattern recognition step of recognizing, by the processor, an ICG pattern representing a change in ICG fluorescence brightness over time from the preprocessed ICG image;
an analysis model generation step in which an analysis model is generated by using a data set including an ICG learning image based on artificial intelligence technology by the processor; and
An analysis result output step of outputting, by the processor, an analysis result including a degree of risk from the ICG pattern using the analysis model; artificial intelligence technology-based real-time micro blood flow analysis method comprising: According to claim 1,
The ICG image pre-processing step,
an HSV extraction step of extracting color (H), saturation (S), and brightness (V) from the ICG image;
a comparison step of comparing the color (H) and saturation (S) of the ICG image with respective reference values;
If at least one of the absolute value of the difference between the color (H) of the ICG image and the color reference value and the absolute value of the difference between the saturation (S) of the near-infrared image and the attitude reference value is equal to 1 or close to 1 within the error range, a new brightness value is obtained. A lightness calculation step in which (V') is calculated; and
and removing noise from the ICG image by using the new brightness (V'), color (H), and chroma (S) of the ICG image. According to claim 1,
The ICG pattern recognition step,
An artificial intelligence technology-based real-time micro blood flow analysis method comprising the step of estimating the ICG fluorescence expression time point after the delay time (Latency) of the ICG image. According to claim 3,
The ICG pattern recognition step,
The method further includes a region of interest setting step in which a part of the tissue to be identified around the anastomosis in the ICG image is set as one or more regions of interest (ROIs) for a predetermined time from the time point of the ICG fluorescence expression. Artificial intelligence technology-based real-time micro blood flow analysis method. According to claim 4,
In the step of outputting the analysis result,
Artificial intelligence technology-based real-time micro blood flow analysis method, characterized in that the risk according to time for each region of interest (ROI) set from the region of interest setting step is displayed as a color map. According to claim 1,
The analysis model generation step,
an ICG learning image pre-processing step in which the ICG learning image is pre-processed;
an ICG learning pattern recognition step of recognizing an ICG learning pattern from the preprocessed ICG learning image;
an ICG learning pattern set generating step in which an ICG learning pattern set is generated after the ICG learning patterns are clustered based on similarity;
a labeling step of labeling the ICG learning pattern set as one of a normal state, a good state, and a dangerous state;
an ICG test image analysis step of learning the labeled ICG learning pattern set, generating a preliminary analysis model, and analyzing the ICG test image using the preliminary analysis model; and
An analysis model determination step in which the preliminary analysis model is determined as the analysis model when the accuracy of the preliminary analysis model is greater than or equal to a preset accuracy based on the analysis result of the ICG test image;
Wherein the analysis model determination step returns to the ICG test image analysis step when the preliminary analysis model is less than a preset accuracy. According to claim 6,
In the step of generating the ICG learning pattern set,
A variance value for an ICG learning pattern included in any of the ICG learning pattern sets is calculated;
If the variance value is within a preset variance value range, it is determined that the clustering accuracy of the ICG learning pattern is high. According to claim 6,
The labeling step is
According to the ratio of the number of ICG learning patterns that are similar to the ICG learning patterns included in the corresponding ICG learning pattern set among the ICG learning patterns of patients diagnosed with anastomotic complications to the number of ICG learning patterns included in any ICG learning pattern set An artificial intelligence technology-based real-time micro blood flow analysis method, characterized in that a set of random ICG learning patterns is classified into one of normal, good, and dangerous conditions.