KR102566890B1 - Method for surgical site monitoring and device using the same - Google Patents

Abstract

The present invention is a surgical site monitoring method implemented by a processor, comprising the steps of receiving a medical image including a surgical site of an object, using a prediction model configured to predict abnormal blood flow using the medical image as an input, Provided is a surgical site monitoring method and a device for monitoring a surgical site using the same, including determining whether or not there is an abnormality and providing an abnormality.

Description

Surgical site monitoring method and device using the same {METHOD FOR SURGICAL SITE MONITORING AND DEVICE USING THE SAME}

The present invention relates to a surgical site monitoring method and a device using the same.

A living tissue transplant is a biological tissue that is chemically treated to remove immune properties or impart a specific function to a biological tissue of a human or animal, and may refer to a biological tissue that can be transplanted into a human tissue or organ damaged by a disease or accident.

Such living tissue transplants may include heart valves, pericardium, blood vessels, ligaments, meningeal membrane substitutes, wound dressings for burn treatment, and the like.

Although transplantation surgery or bonding surgery of such biological tissue and autologous tissue is frequently performed, it may be difficult to accurately recognize and continuously monitor the degree of recovery of the surgical site, so that accurate treatment according to the progress may be impossible.

For example, in the case of a patient who has undergone free flap surgery, continuous monitoring of the surgical site may be required. However, surgical site monitoring requires excessive time and effort from medical personnel, and it may be practically impossible to reside next to a patient for 24 hours.

On the other hand, failure of monitoring the surgical site may cause deterioration of the patient's course and increase in medical costs, and thus, development of a new surgical site monitoring system is continuously required.

The background description of the invention has been prepared to facilitate understanding of the present invention. It should not be construed as an admission that matters described in the background art of the invention exist as prior art.

Korean Patent Laid-open Publication No. 10-2014-0134566 (November 24, 2014) A medical device capable of combining radiation therapy and thermal therapy and a method for treating an affected area using the same

As a way to overcome the above-mentioned limitations, a surgical site monitoring system has been proposed that photographs the surgical site using a hyperspectral camera and visualizes hemoglobin and oxygen in the tissue.

However, the proposed surgical site monitoring system may have limitations in that it is difficult to apply to actual clinical practice due to high cost burden due to the use of a hyperspectral camera and low sensitivity and specificity.

On the other hand, the inventors of the present invention, as a way to solve the above-mentioned problems, by tracking the surgical site of the patient, it is possible to continuously monitor the surgical site even in the patient's movement, and apply a deep learning-based prediction model to detect abnormal blood flow We wanted to develop a new surgical site monitoring system with high sensitivity and specificity.

More specifically, the inventors of the present invention could design a surgical site monitoring system to obtain an image of the surgical site by arranging an identification factor in the area around the surgical site and recognizing the identification factor by a camera.

Accordingly, the inventors of the present invention recognized that even if the object moves, the surgical site can be tracked through an identification factor and an image of the surgical site can be provided, enabling continuous image acquisition.

In addition, the inventors of the present invention could further apply a prediction model configured to predict an abnormal time point by detecting a color change of a surgical site based on a deep learning algorithm.

At this time, the inventors of the present invention apply a preprocessing algorithm that keeps conditions such as saturation or brightness of the medical image of the surgical site the same to the surgical site monitoring system, thereby detecting changes in blood flow with high accuracy. was able to perceive

That is, the inventors of the present invention provide such a deep learning-based surgical site monitoring system, so that the surgical site can be monitored in real time without a medical staff residing, and blood flow instability can be predicted with high accuracy regardless of skill level could be expected

Furthermore, the inventors of the present invention could expect that when abnormal blood flow is detected with respect to the surgical site, the time point of medical treatment can be advanced by providing a real-time alarm to the medical device.

Accordingly, an object to be solved by the present invention is a surgical site monitoring method configured to detect abnormal blood flow in real time with respect to a medical image of a surgical site using a deep learning epoch prediction model and provide abnormal blood flow, and a surgical site monitoring method using the same to provide the device.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a surgical site monitoring method according to an embodiment of the present invention is provided. A surgical site monitoring method according to an embodiment of the present invention is implemented by a processor, and includes the steps of receiving a medical image including a surgical site of an object, predicting abnormal blood flow using the medical image as an input. Using the constructed predictive model, determining whether or not there is an abnormality in the surgical site, and providing whether or not there is an abnormality.

According to a feature of the present invention, the method further comprises, prior to the receiving step, determining a position of the surgical site from the object, and the receiving of the medical image comprises receiving the medical image of the determined surgical site position. steps may be included.

According to another feature of the present invention, the method, before the step of determining the position of the surgical site, may further include the step of arranging an identification factor in a region spaced apart from the surgical site. In this case, determining the position of the surgical site may include recognizing the identification factor through a camera, and determining the position of the surgical site based on the identification factor.

According to another feature of the present invention, the identifying factor may include clinical information about the individual.

According to another feature of the present invention, the identification factor may be at least one of a QR code, a barcode, an NFC tag, an RFID tag, and an OCR code.

According to another feature of the present invention, the method may further include, after receiving the medical image, adjusting saturation and brightness of the medical image to a predetermined level.

According to another feature of the present invention, the method further comprises, prior to the step of adjusting to a predetermined level, disposing a reference color mark in an area spaced apart from the surgical site, wherein the adjusting to the predetermined level comprises: , adjusting the saturation and brightness of the surgical site region based on the reference color mark.

According to another feature of the present invention, the providing may include providing a notification to a medical device when abnormal blood flow to the surgical site of the subject is determined.

According to another feature of the present invention, the surgical site may be a surgical site according to skin transplantation.

According to another feature of the present invention, the prediction model may be a model learned to predict abnormal blood flow by detecting a color change of the surgical site.

In order to solve the above problems, a device according to another embodiment of the present invention is provided. The device includes a receiver configured to receive a medical image including a surgical site of an object, and a processor. In this case, the processor may be configured to determine whether or not there is an abnormality in the surgical site using an abnormality prediction model configured to predict abnormal blood flow based on the medical image, and provide the abnormality.

According to a feature of the present invention, the processor may be further configured to determine a surgical site location from an object, and the receiving unit may be further configured to receive a medical image of the determined surgical site location.

According to another feature of the present invention, the processor may be further configured to, when an identification factor is disposed in an area spaced apart from the surgical site, recognize the identification factor through a camera and determine a position of the surgical site based on the identification factor. .

According to another feature of the present invention, a device for monitoring a surgical site, wherein the identification factor includes clinical information about the subject.

According to another feature of the present invention, the identification factor is at least one of a QR code, a barcode, an NFC tag, an RFID tag, and an OCR code, a device for monitoring a surgical site.

According to another feature of the present invention, the processor may be further configured to adjust saturation and brightness of the medical image to a predetermined level.

According to another feature of the present invention, the processor may be further configured to adjust saturation and brightness of the surgical site region based on the reference color mark when the reference color mark is disposed in an area spaced apart from the surgical site.

According to another feature of the present invention, the processor may be further configured to provide a notification to a medical practitioner device when abnormal blood flow to a surgical site of a subject is determined.

According to another feature of the present invention, the surgical site may be a surgical site according to skin transplantation.

According to another feature of the present invention, the predictive model may be a model learned to predict abnormal blood flow by detecting a color change of a surgical site.

Other embodiment specifics are included in the detailed description and drawings.

The present invention may provide a surgical site monitoring system configured to predict abnormal blood flow in a surgical site medical image based on a deep learning algorithm and monitor in real time whether or not there is an abnormality in the surgical site.

More specifically, the present invention provides a deep learning-based surgical site monitoring system, so that real-time monitoring of the surgical site is possible even without medical personnel residing there, and there is an effect of predicting blood flow instability with high accuracy.

Furthermore, according to the present invention, when an abnormal blood flow is detected in a surgical site, a medical treatment point can be advanced by providing a real-time alarm to a medical device.

For example, the present invention, in reconstructive surgery such as free flap surgery, can reduce the time for medical staff to stay for a patient who has undergone skin transplantation, which is highly difficult and requires 24-hour monitoring as it is directly related to the patient's life. can In addition, the present invention can provide an alarm when abnormal blood flow is detected, so that medical staff can quickly treat patients, especially patients who have undergone free flap surgery requiring rapid revascularization when abnormal blood flow is detected in the surgical site. Accordingly, a positive prognosis of the patient can be provided.

That is, the present invention provides a surgical site monitoring system based on a deep learning algorithm, thereby preventing medical staff from misinterpreting and improving the medical staff's workflow in actual clinical practice.

Effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

Figure 1a shows a surgical site monitoring system based on a device for monitoring the surgical site according to an embodiment of the present invention by way of example.
Figure 1b illustrates the configuration of a device for monitoring a surgical site according to an embodiment of the present invention by way of example.
FIG. 1C illustrates the configuration of a medical staff device configured to receive abnormal blood flow in a surgical site from a device for monitoring a surgical site according to an embodiment of the present invention.
Figures 2a and 2b illustratively illustrates the procedure of the surgical site monitoring method according to an embodiment of the present invention.
3A illustrates a pre-processing procedure of a medical image in a surgical site monitoring method according to an embodiment of the present invention.
3B illustrates changes in a medical image according to pre-processing of the medical image in the method for monitoring a surgical site according to an embodiment of the present invention.
4 exemplarily illustrates the structure of a predictive model used in various embodiments of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various forms different from each other, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like elements.

In this document, expressions such as "has," "may have," "includes," or "may include" indicate the existence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document may modify various elements, regardless of order and/or importance, and refer to one element as It is used only to distinguish it from other components and does not limit the corresponding components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, without departing from the scope of rights described in this document, a first element may be named a second element, and similarly, the second element may also be renamed to the first element.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that the certain component may be directly connected to the other component or connected through another component (eg, a third component). On the other hand, when an element (eg, a first element) is referred to as being “directly connected” or “directly connected” to another element (eg, a second element), the element and the above It may be understood that other components (eg, third components) do not exist between the other components.

As used in this document, the expression "configured to" means "suitable for," "having the capacity to," depending on the circumstances. ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware. Instead, in some contexts, the expression "a device configured to" may mean that the device "is capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (eg, an embedded processor) to perform those operations, or by executing one or more software programs stored on a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe a specific embodiment, and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted as having the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, an ideal or excessively formal meaning. not be interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude the embodiments of this document.

Each feature of the various embodiments of the present invention can be partially or entirely combined or combined with each other, and as those skilled in the art can fully understand, various interlocking and driving operations are possible, and each embodiment can be implemented independently of each other. It may be possible to implement together in an association relationship.

For clarity of interpretation of this specification, terms used in this specification will be defined below.

As used herein, the term "surgical site" is a site where a surgical operation has been performed, and may refer to a site where a diseased area is excised or a treatment is performed. For example, the surgical site may be a site where skin grafting such as free flap surgery or local flap surgery has been performed, but is not limited thereto.

As used herein, the term "subject" may refer to any subject whose surgical site is to be monitored. For example, the object may be an object that has undergone a surgical operation. In this case, the subject disclosed in this specification may be any mammal except for humans, but is not limited thereto.

As used herein, the term "medical image" refers to a digital image of a surgical site. Preferably, the medical image may be an RGB color image, but is not limited thereto.

According to the features of the present invention, an identification factor such as a QR code, barcode, NFC tag, RFID tag, or OCR code may exist around the surgical site, and the camera recognizes this and tracks the surgical site to obtain a medical image of the surgical site area. can provide. In particular, even if the surgical site moves out of the angle of the camera according to the movement of the object, it is possible to continuously track the surgical site due to recognition of the identification factor, and thus image acquisition can be facilitated. In this case, the identification factor may include various clinical information such as age, gender, underlying disease, hospitalization time, operation time, and type of medical treatment performed.

Meanwhile, the medical image may be a video consisting of a plurality of frames. For example, in a medical image composed of a plurality of frames, abnormal blood flow may be predicted for each frame of the video according to the surgical site monitoring method according to an embodiment of the present invention. That is, abnormal blood flow can be predicted at the same time as medical images are received from the camera, so real-time monitoring of the surgical site can be possible.

According to a feature of the present invention, the medical image may be a 2D image or a 3D image.

As used herein, the term “prediction model” may be a model configured to output whether there is abnormal blood flow by taking a medical image of a diseased part as an input.

More specifically, the predictive model may be configured to detect a color change of the affected area by using a medical image as an input, and probabilistically predict normal or abnormal (abnormal blood flow) of the image.

For example, when a training medical image of a normal or abnormal surgical site image is input to the predictive model, features of the medical image are extracted by a convolution layer and a fully connected layer of the predictive model. , Normal or abnormal blood flow can be probabilistically predicted by a fully convolutional network (FNC) layer.

Optionally, the predictive model may be further configured to predict the occurrence of abnormal blood flow based on the color change in the medical image.

Meanwhile, prediction models according to various embodiments of the present invention are Resnet50, Resnet-v2, Resnet101, Inception-v3, or VGG net, R, DenseNet, and FCN, SegNet, DeconvNet, and DeepLAB V3+ having an encoder-decoder structure. , a deep neural network (DNN) such as U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, and GoogLeNet. Furthermore, the prediction models may be ensemble models based on at least two algorithm models among the aforementioned algorithms.

Hereinafter, a surgical site monitoring system based on a device for monitoring a surgical site according to an embodiment of the present invention will be described with reference to FIGS. 1A to 1C .

1A illustrates a surgical site monitoring system based on a device for monitoring a surgical site according to an embodiment of the present invention. Figure 1b illustrates the configuration of a device for monitoring a surgical site according to an embodiment of the present invention by way of example. FIG. 1C illustrates a configuration of a medical staff device configured to receive abnormal blood flow in a surgical site from a device for monitoring a surgical site according to an embodiment of the present invention.

First, referring to FIG. 1A , a surgical site monitoring system 1000 may be a system configured to provide information related to a surgical site based on a medical image of an object. At this time, the surgical site monitoring system 1000 includes a surgical site monitoring device 100 configured to predict abnormal blood flow and determine whether or not there is an abnormality in the surgical site based on the medical image, and receive information on surgical site monitoring. It may be composed of a medical staff device 200 and a device 300 for providing medical images that provide images of a surgical site.

First, the surgical site monitoring device 100 includes a general-purpose computer, a laptop, and/or a general-purpose computer that performs various calculations to diagnose whether or not the surgical site is abnormal based on the medical image of the object provided from the device 300 for providing medical images. It may include a data server and the like. In this case, the medical staff device 200 may be a device for accessing a web server providing a web page or a mobile web server providing a mobile web site, but is not limited thereto.

More specifically, the surgical site monitoring device 100 may receive a medical image from the medical image providing device 300 and provide information related to whether or not the surgical site is abnormal from the received medical image. In this case, the device 300 for providing medical images may detect abnormalities after detecting changes in blood flow for the medical images using a predictive model.

The surgical site monitoring device 100 may provide data related to an abnormality of the surgical site of the subject to the medical device 200 .

Data provided from the surgical site monitoring device 100 may be provided in the form of a web page, application, or program through a web browser installed in the medical device 200 . In various embodiments, this data may be provided in a form incorporated into the platform in a client-server environment.

Next, the medical staff device 200 is an electronic device that provides a user interface for requesting provision of information on an abnormality of a surgical site of an object and displaying monitoring result data, such as a smartphone, a tablet PC (Personal Computer), or a laptop computer. and/or a PC, and the like.

The medical staff device 200 may receive a monitoring result of a surgical site of an object from the surgical site monitoring device 100 and display the received result through a display unit. At this time, when an abnormality is detected in the surgical site of the object by the surgical site monitoring device 100, the display unit of the medical staff device 200 may further output a notification. Accordingly, the medical staff can more quickly perform appropriate medical treatment for the object.

On the other hand, the monitoring result may include not only abnormal blood flow in the surgical site predicted by the predictive models, but also a treatment prognosis, an appropriate treatment method for the surgical wound, and the like.

Next, with reference to FIG. 1B , components of the surgical site monitoring device 100 of the present invention will be described in detail.

Referring to FIG. 1B , the surgical site monitoring device 100 includes a storage unit 110 , a communication unit 120 and a processor 130 .

First, the storage unit 110 may store various data generated while diagnosing whether or not an object has an abnormal surgical site. For example, the storage unit 110 may be configured to store medical images received from the device 300 for providing medical images through the communication unit 120 to be described later, and various products in a classification process of a predictive model. . In various embodiments, the storage unit 110 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory , a magnetic disk, and an optical disk may include at least one type of storage medium.

The communication unit 120 connects the surgical site monitoring device 100 to enable communication with an external device. The communication unit 120 can transmit/receive various data by being connected to the medical staff device 200 and furthermore the device 300 for providing medical images using wired/wireless communication. In detail, the communication unit 120 may receive a medical image of a surgical site of an object from the device 300 for providing medical images. For example, the communication unit 120 may receive an RGB image of a surgical site from the device 300 for providing medical images. Furthermore, the communication unit 120 may transmit monitoring results to the medical device 200 .

The processor 130 is operatively connected to the storage unit 110 and the communication unit 120, and can perform various commands for analyzing a medical image of an object.

Specifically, the processor 130 may be configured to detect a change in blood flow in the surgical site based on the medical image received through the communication unit 120 and determine whether there is an abnormality.

In this case, the processor 130 may be based on a predictive model configured to determine whether or not there is an abnormality in the surgical site by detecting a change in blood flow based on the medical image.

On the other hand, the surgical site monitoring device 100 is not limited to being designed in terms of hardware. For example, the processor 130 of the device 100 for monitoring a surgical site may be implemented as software. Accordingly, the monitoring result for the surgical site may be displayed through the display unit of the device 300 for providing medical images to which the software is connected.

Meanwhile, referring to FIG. 1C together, the medical device 200 includes a communication unit 210, a display unit 220, a storage unit 230, and a processor 240.

The communication unit 210 may be configured to allow the medical device 200 to communicate with an external device. The communication unit 210 may be connected to the surgical site monitoring device 100 using wired/wireless communication to transmit various data related to diagnosis of the surgical site. Specifically, the communication unit 210 may receive, from the device for monitoring the surgical site 100, whether there is abnormal blood flow in the surgical site predicted by the prediction models, the timing of abnormal blood flow, a treatment prognosis, and an appropriate treatment method for the surgical wound. can

The display unit 220 may display various interface screens for displaying monitoring results of an object's surgical site. Furthermore, the display unit 220 may be further configured to output a notification when an abnormality is detected in the surgical site.

For example, the display unit 220 may visually output a monitoring result of the surgical site, that is, information on normal or abnormal (abnormal blood flow), or may output an alarm sound when abnormal blood flow is detected. Accordingly, when detecting abnormal blood flow in the surgical site, prompt treatment by medical staff may be possible for an individual who has undergone free flap surgery requiring rapid revascularization.

In various embodiments, the display unit 220 may include a touch screen, and for example, a touch using an electronic pen or a part of the user's body, a gesture, a proximity, a drag, or a swipe A swipe or hovering input may be received.

The storage unit 230 may store various data used to provide a user interface for displaying result data. In various embodiments, the storage unit 230 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (for example, SD or XD memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.

The processor 240 is operably connected to the communication unit 210, the display unit 220, and the storage unit 230, and can perform various commands to provide a user interface for displaying monitoring result data.

Hereinafter, a surgical site monitoring method according to an embodiment of the present invention will be described in detail with reference to FIGS. 2a, 2b, 3a, and 3b.

Figures 2a and 2b illustratively illustrates the procedure of the surgical site monitoring method according to an embodiment of the present invention. 3A illustrates a pre-processing procedure of a medical image in a surgical site monitoring method according to an embodiment of the present invention. 3B illustrates changes in a medical image according to preprocessing of the medical image in the method for monitoring a surgical site according to an embodiment of the present invention.

First, referring to FIG. 2A , a surgical site monitoring procedure according to an embodiment of the present invention is as follows. First, the position of the surgical site is determined for the object (S210), and a medical image of the surgical site is received (S220). Then, using the predictive model, whether or not the surgical site is abnormal is determined (S230), and finally, the monitoring result is provided (S240).

According to a feature of the present invention, before the step of determining the position of the surgical site (S210) is performed, an identification factor may be disposed in an area spaced apart from the surgical site. Accordingly, in step S210 of determining the position of the surgical site, an identification factor is recognized by a medical image providing device such as a camera, and the position of the surgical site is determined by the identification factor. Then, in the step of receiving a medical image of the surgical site (S220), an image of the surgical site whose position is determined is acquired.

For example, referring to FIG. 2B together, in step S210 of determining the position of the surgical site, a medical image providing device (not shown) recognizes a QR code 314 pre-arranged around the surgical site 312. can do. Then, a medical image may be acquired at the position of the recognized surgical site 312 by the medical image providing apparatus. That is, in the step of receiving a medical image of the surgical site (S220), a medical image 320 including the surgical site may be acquired. Meanwhile, the medical image 320 may include various recognizable identification factors such as figures other than the QR code 314 .

According to a feature of the present invention, the medical image 320 may be received in real time in step S220 of receiving the medical image of the surgical site.

According to another feature of the present invention, in the step of receiving the medical image of the surgical site (S220), the medical image is accessed by accessing the assigned web server IP from a micro PC (eg, Raspberry Pi) connected to the medical image providing device. (320) may be received.

According to another feature of the present invention, the surgical site may be a site where skin grafting such as free flap surgery or local flap surgery has been performed, but is not limited thereto.

Next, referring to FIG. 2A again, in the step of determining whether or not the surgical site is abnormal (S230), whether the medical image is abnormal is determined by a predictive model, and the result may be provided (S240).

For example, again referring to FIG. 2B , in step S230 of determining whether or not there is an abnormality in the surgical site, the medical image 320 may be input to the predictive model 330 . More specifically, when a normal or abnormal medical image 320 is input to the prediction model 330 composed of a plurality of layers, features of the medical image 320 are extracted, and normal or abnormal blood flow is monitored based on the features. 340 may be stochastically output. Optionally, in the step (S230) of determining whether or not there is an abnormality in the surgical site, the occurrence time point of abnormal blood flow can be predicted. Then, in step S240 of providing the result, the monitoring result 340 may be transmitted to the medical device 200 . At this time, the monitoring result 340 may be visually provided on the display unit 220 of the medical device 200 . Furthermore, when abnormal blood flow to the surgical site is detected as a result of determining whether or not there is an abnormality in the surgical site (S230), the display unit 220 may further output a notification sound in the step of providing the result (S240). there is. In this case, a notification sound may be output at the time of detecting an abnormality. Accordingly, the medical staff can more easily recognize the danger to the object and proceed with appropriate medical treatment more quickly.

According to a feature of the present invention, in the step of providing a result (S240), when an abnormality is determined for a medical image, a medical staff device through an unrestricted communication method such as a transmission control protocol/internet protocol (TCP/IP) network or Bluetooth. A notification message may be transmitted to 200 .

According to the features of the present invention, the position of the surgical site to be described above is determined (S210), a medical image of the surgical site is received (S220), and whether or not there is an abnormality in the surgical site is determined (S230), and the step of providing the result (S240) can be performed simultaneously. That is, abnormal blood flow may be predicted simultaneously with the reception of an image from the medical image providing apparatus, and the predicted information may be transmitted to a medical device, so that real-time monitoring of a surgical site may be possible.

A real-time surgical site monitoring system based on the surgical site monitoring method according to various embodiments above can overcome the limitations of conventional surgical site monitoring systems. In particular, the system can overcome the limitations of the conventional surgical site monitoring system in that it can monitor the surgical site in real time even without a resident medical staff and can predict blood flow instability with high accuracy.

Moreover, the system can advance the time of medical treatment by providing a real-time alarm to a medical device when abnormal blood flow is detected in the surgical site, and can contribute to early treatment and good treatment prognosis.

Meanwhile, the surgical site monitoring procedure is not limited to the above features.

For example, according to another embodiment of the present invention, a step of adjusting saturation and brightness of the medical image to a predetermined level may be further performed after the step of receiving the medical image of the surgical site.

Referring to FIG. 3A , a reference color mark is pre-arranged in an area separated from the surgical site (S310), and saturation and brightness of the surgical site area are adjusted based on the reference color mark (S320).

More specifically, in the step of pre-arranging the reference color mark (S310), a reference color mark may be arranged to maintain the same conditions of the received medical image except for a change in blood flow in the surgical site. there is.

Then, in the step of adjusting the saturation and brightness of the surgical site region (S320), the saturation and brightness of the new medical image may be adjusted based on the saturation or brightness of the reference color mark disposed around the surgical site.

For example, referring to FIG. 3B together, in step S320 of adjusting the saturation and brightness of the surgical site area, the received medical image (FIG. 3B (a) and In (c)), the color and brightness values may be adjusted so that the brightness and saturation are constant. As a result, adjusted medical images ((b) and (d) of FIG. 3B) can be obtained.

That is, conditions such as brightness and saturation are equally adjusted for a plurality of medical images according to the progress of monitoring for the same object, and as the adjusted medical images are input to the predictive model, changes in blood flow can be determined with high sensitivity and accuracy. can

According to a feature of the present invention, the step of adjusting the saturation and brightness of the surgical site region based on the reference color mark ( S320 ) may be implemented as an automatic image processing algorithm for a received image.

Hereinafter, structural characteristics of a predictive model used in various embodiments of the present invention will be described with reference to FIG. 4 .

Referring to FIG. 4 , a prediction model 330 applied to various embodiments of the present invention may include an input layer, 50 hidden layers, and an output layer.

More specifically, when a medical image is input through an input layer, features of the medical image may be extracted by 50 convolution layers and a fully connected layer. In this case, the feature extraction value of the image may vary according to a color change of the surgical site, such as abnormal blood flow. Next, normal or abnormal blood flow may be probabilistically output based on the extracted features by the last fully convolutional network (FNC) layer.

That is, the surgical site monitoring device based on such a predictive model can perform an objective analysis of the surgical site, so that the accuracy of abnormal classification can be increased. In addition, the surgical site monitoring device can reduce unnecessary time for medical personnel to reside. Furthermore, the device for monitoring the surgical site can contribute to a positive prognosis of the subject by providing an alarm at the time of occurrence of abnormal blood flow so that medical staff can quickly treat the condition.

Meanwhile, prediction models according to various embodiments of the present invention are Resnet50, Resnet-v2, Resnet101, Inception-v3, or VGG net, R, DenseNet, and FCN, SegNet, DeconvNet, and DeepLAB V3+ having an encoder-decoder structure. , a deep neural network (DNN) such as U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, and GoogLeNet. Furthermore, the prediction models may be ensemble models based on at least two algorithm models among the aforementioned algorithms. Preferably, the prediction model may be a model based on Resnet50, but is not limited thereto.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and may be variously modified and implemented without departing from the technical spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: device for monitoring the surgical site
110, 230: storage unit
120, 210: communication department
130, 240: processor
200: medical staff device
220: display unit
300: device for providing medical images
312 surgical site
314: QR code
320: medical imaging
330: prediction model
340: monitoring result

Claims (20)

As a surgical site monitoring method implemented by a processor,
Recognizing an identification factor disposed in an area separated from a surgical site of an object photographed by the camera;
Obtaining clinical information including at least one of age, gender, underlying disease, hospitalization time, surgery time, and type of medical treatment of the individual included in the identification factor, and determining the surgical site based on the identification factor tracking, determining the location of the surgical site of the subject;
receiving, in real time, a medical image corresponding to the position of the surgical site by transmitting photographing data for photographing the position of the surgical site determined through a communication unit operatively connected to the processor to the camera;
Determining whether there is an abnormality in the surgical site using a predictive model configured to predict abnormal blood flow using the medical image as an input; and
When abnormal blood flow to the surgical site of the subject is determined, providing a monitoring result including a treatment prognosis or treatment for the surgical site along with notification of the clinical information and abnormality to a medical device through the communication unit; Including, surgical site monitoring method. delete delete delete According to claim 1,
Wherein the identification factor is at least one of a QR code, a barcode, an NFC tag, an RFID tag, and an OCR code. According to claim 1,
After receiving the medical image,
The surgical site monitoring method further comprising adjusting saturation and brightness of the medical image to a predetermined level. According to claim 6,
Prior to the step of adjusting to the predetermined level,
Further comprising arranging a reference color mark in an area spaced apart from the surgical site,
The step of adjusting to the predetermined level,
and adjusting saturation and brightness of the surgical site region based on the reference color mark. delete According to claim 1,
The surgical site is
Surgical site monitoring method, which is a surgical site according to skin grafting. According to claim 1,
The predictive model,
A method for monitoring a surgical site, which is a model learned to predict abnormal blood flow by detecting a color change of the surgical site. A communication unit configured to receive a medical image including a surgical site of an object, and
contains a processor;
the processor,
recognizing an identification factor disposed in an area spaced apart from the surgical site of the object photographed by the camera through the communication unit;
Obtaining clinical information including at least one of age, gender, underlying disease, hospitalization time, surgery time, and type of medical treatment of the individual included in the identification factor, and determining the surgical site based on the identification factor Tracking to determine the location of the surgical site of the subject;
Transmits photographing data for photographing the position of the surgical site determined through the communication unit to the camera, and receives a medical image corresponding to the position of the surgical site in real time;
Using an abnormality prediction model configured to predict abnormal blood flow based on the medical image, determining whether or not there is an abnormality in the surgical site;
When abnormal blood flow to the surgical site of the subject is determined, a monitoring result including a treatment prognosis or treatment for the surgical site along with notification of the clinical information and the abnormality is provided to the medical device through the communication unit. Configured to provide, Device for surgical site monitoring. delete delete delete According to claim 11,
The identification factor is at least one of a QR code, a barcode, an NFC tag, an RFID tag, and an OCR code, a device for monitoring a surgical site. According to claim 11,
the processor,
A device for monitoring a surgical site, further configured to adjust saturation and brightness of the medical image to a predetermined level. According to claim 16,
the processor,
When a reference color mark is disposed in an area spaced apart from the surgical site, the surgical site monitoring device further configured to adjust saturation and brightness of the surgical site area based on the reference color mark. delete According to claim 11,
The surgical site is a surgical site according to skin transplantation, a device for monitoring a surgical site. According to claim 11,
The prediction model is a model learned to detect a color change of the surgical site and predict abnormal blood flow, a surgical site monitoring device.

Images