WO2024111846A1

WO2024111846A1 - Method and device for detecting intraoperative bleeding through spatiotemporal feature fusion model

Info

Publication number: WO2024111846A1
Application number: PCT/KR2023/014458
Authority: WO
Inventors: 홍슬기; 홍승범; 김근영; 장준영; 최민국
Original assignee: (주)휴톰
Priority date: 2022-11-22
Filing date: 2023-09-22
Publication date: 2024-05-30
Also published as: KR20240077534A

Abstract

The present disclosure relates to a method and a device for detecting intraoperative bleeding through a spatiotemporal feature fusion model. The method may comprise the steps of: acquiring first temporal feature data through a surgical video set to a first frame rate, and acquiring second temporal feature data through a surgical video set to a second frame rate; acquiring spatial feature data through a representative frame from among a plurality of frames constituting the surgical video; acquiring first intermediate data on the basis of the first temporal feature data and the spatial feature data, and acquiring second intermediate data on the basis of the second temporal feature data and the spatial feature data; concatenating the first intermediate data and the second intermediate data so as to generate training data; and training, on the basis of the training data, a first artificial intelligence model to output surgical index data enabling bleeding frequency and bleeding sections in the surgical video to be identified.

Description

Method and device for detecting intraoperative bleeding through spatiotemporal feature fusion model

The present disclosure relates to methods and devices for detecting bleeding during surgery. More specifically, the present disclosure relates to a method and device for detecting intraoperative bleeding through a spatiotemporal feature fusion model.

Intraoperative active bleeding (iAB) is a representative adverse event associated with surgery. If iAB occurs, it can delay surgery time and damage organs, adversely affecting the patient's surgical results.

Therefore, when the occurrence of iAB is detected, it is essential to accurately identify the point requiring hemostasis in real time and quickly stop the bleeding. However, there is a problem that considerable cost and technology are required to detect the occurrence of iAB.

To solve this problem, methods for automatically detecting the occurrence of iAB have been continuously researched and developed. However, since the color and texture of iAB are similar to those of organs and inactive bleeding, a method for efficiently detecting the occurrence of iAB has not been developed.

The purpose of the embodiments disclosed in the present disclosure is to provide a method and device for detecting bleeding during surgery through a spatiotemporal feature fusion model.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

A method of detecting intraoperative bleeding through a surgical image performed by a device according to an embodiment of the present disclosure to solve the above-described technical problem includes using the surgical image set at a first frame rate. Obtaining first temporal feature data through and acquiring second temporal feature data through the surgical image set to a second frame rate; Obtaining spatial feature data through a representative frame among a plurality of frames constituting the surgical image; Obtaining first intermediate data based on the first temporal feature data and the spatial feature data, and acquiring second intermediate data based on the second temporal feature data and the spatial feature data; generating learning data by concatenating the first intermediate data and the second intermediate data; And a first artificial intelligence (AI) model to output surgical index data that can identify the bleeding count and bleeding duration in the surgical image based on the learning data. It may include a learning step.

In addition, the device according to the present disclosure for solving the above-described technical problem includes: a memory storing at least one process for detecting bleeding during surgery through a surgical image; And a processor that performs an operation of detecting bleeding during the surgery as the process is executed, wherein the processor acquires first temporal feature data through the surgical image set at a first frame rate, and Obtain second temporal feature data through the surgical image set at a 2 frame rate, obtain spatial feature data through a representative frame among a plurality of frames constituting the surgical image, and obtain the first temporal feature data and Obtain first intermediate data based on the spatial feature data, obtain second intermediate data based on the second temporal feature data and the spatial feature data, and connect the first intermediate data and the second intermediate data. (concatenation) to generate learning data, and output surgical index data that can identify the bleeding count and bleeding duration in the surgical image based on the learning data. Artificial intelligence (AI) models can be trained.

In addition to this, a computer program stored in a computer-readable recording medium for implementing the present disclosure may be further provided.

In addition, a computer-readable recording medium recording a computer program for implementing the present disclosure may be further provided.

According to the means for solving the above-described problem of the present disclosure, a method and device for detecting bleeding during surgery can be provided through a spatiotemporal feature fusion model.

According to the means for solving the above-described problem of the present disclosure, by more efficiently recognizing bleeding during surgery, it is possible to evaluate the surgery and accurately predict/analyze the patient's prognosis.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned may be clearly understood by those skilled in the art from the description below.

1 is a schematic diagram of a system for implementing a method for detecting intraoperative bleeding through surgical images, according to an embodiment of the present disclosure.

Figure 2 is a block diagram for explaining the configuration of an apparatus for detecting bleeding during surgery through surgical images, according to an embodiment of the present disclosure.

Figure 3 is a flowchart for explaining a method of detecting bleeding during surgery through a surgical image, according to an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating an architecture for performing a method of detecting intraoperative bleeding through surgical images, according to an embodiment of the present disclosure.

Figure 5 is a diagram for explaining a method of detecting bleeding during surgery through a surgical image using an AI model, according to an embodiment of the present disclosure.

Figure 6 shows examples of temporal feature data and spatial feature data to which the present disclosure can be applied.

Like reference numerals refer to like elements throughout this disclosure. This disclosure does not describe all elements of the embodiments, and general content or overlapping content between embodiments in the technical field to which this disclosure pertains is omitted. The term 'part, module, member, block' used in the specification may be implemented as software or hardware, and depending on the embodiment, a plurality of 'part, module, member, block' may be implemented as a single component, or It is also possible for one 'part, module, member, or block' to include multiple components.

Throughout the specification, when a part is said to be “connected” to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection through a wireless communication network. do.

Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

Throughout the specification, when a member is said to be located “on” another member, this includes not only cases where a member is in contact with another member, but also cases where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the present disclosure will be described with reference to the attached drawings.

In this specification, 'device according to the present disclosure' includes all various devices that can perform computational processing and provide results to the user. For example, the device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may take the form of any one.

Here, the computer may include, for example, a laptop, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

The server device is a server that processes information by communicating with external devices and may include an application server, computing server, database server, file server, game server, mail server, proxy server, and web server.

The portable terminal is, for example, a wireless communication device that guarantees portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), and PDA. (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone ), all types of handheld wireless communication devices, and wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-device (HMD). may include.

In explaining the present disclosure, a “user” is a medical professional and may be a doctor, nurse, clinical pathologist, medical imaging expert, etc., and may be a technician who repairs/controls a medical device, but is not limited thereto.

In explaining the present disclosure, “surgery” refers to a surgical treatment performed by incising the skin or mucous membrane for disease or trauma.

1 is a schematic diagram of a system 1000 for implementing a method for detecting intraoperative bleeding through surgical images, according to an embodiment of the present disclosure.

As shown in Figure 1, the system 1000 for implementing a method of detecting bleeding during surgery through surgical images includes a device 100, a hospital server 200, a database 300, and an AI model 400. ) may include.

Here, in FIG. 1, the device 100 is shown to be implemented in the form of a single desktop, but it is not limited thereto. As described above, device 100 may refer to various types of devices or a group of devices in which one or more types of devices are connected.

The device 100, hospital server 200, database 300, and artificial intelligence (AI) model 400 included in the system 1000 can communicate through the network (W). . Here, the network W may include a wired network and a wireless network. For example, the network may include various networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN).

Additionally, the network W may include the known World Wide Web (WWW). However, the network (W) according to an embodiment of the present disclosure is not limited to the networks listed above, and may include at least some of a known wireless data network, a known telephone network, and a known wired and wireless television network.

The device 100 may acquire a surgical image consisting of a plurality of frames corresponding to a plurality of surgical steps through the hospital server 200 and/or the database 300. However, this is only an example, and the device 100 can acquire surgical images captured through a camera connected wirelessly/wired to the device 100.

The device 100 may acquire temporal feature data based on surgical images with different frame rates set, and may acquire spatial feature data through a representative frame among a plurality of frames.

The device 100 performs operations on temporal feature data and spatial feature data to obtain spatiotemporal fusion feature data, and calculates the bleeding count and bleeding section in the surgical image based on the spatiotemporal fusion feature data. The AI model 400 can be trained to output surgical index data that can identify duration.

Operations related to this will be described in detail with reference to the drawings described later.

The hospital server 200 (eg, cloud server, etc.) may capture and store a patient's surgical video. The hospital server 200 may transmit the stored surgical image to the device 100, the database 300, or the AI model 400.

The hospital server 200 can protect the personal information of the person in the surgery video by pseudonymizing or anonymizing the person in the surgery video. Additionally, the hospital server may encrypt and store information related to the age/gender/height/weight/parity of the patient who is involved in the surgery image input by the user.

The database 300 may store various feature data generated by the device 100 and one or more parameters/instructions for utilizing the AI model 400. Although FIG. 1 illustrates the case where the database 300 is implemented outside the device 100, the database 300 may also be implemented as a component of the device 100.

The AI model 400 is an artificial intelligence model trained to output surgical index data that can identify the number of bleedings and bleeding sections within the surgical image. The AI model 400 can be trained to output surgery index data through a data set built with feature data related to actual surgery images. Learning methods may include, but are not limited to, supervised training/unsupervised training. Detection data output through the AI model 400 may be stored in the database 300 or/and the memory of the device 100.

1 illustrates a case where the AI model 400 is implemented outside of the device 100 (e.g., implemented as cloud-based), but is not limited thereto and is a component of the device 100. It can be implemented as:

FIG. 2 is a block diagram illustrating the configuration of an apparatus 100 for detecting intraoperative bleeding through surgical images, according to an embodiment of the present disclosure.

As shown in FIG. 2 , device 100 may include memory 110, communication module 120, display 130, input module 140, and processor 150. However, it is not limited to this, and the software and hardware configuration of the device 100 may be modified/added/omitted depending on the required operation within the range obvious to those skilled in the art.

The memory 110 can store data supporting various functions of the device 100 and at least one process and program for the operation of the processor 150, and can prevent bleeding during surgery through surgical images according to the present disclosure. At least one process for detection can be stored, input/output data (e.g., entire surgical image consisting of multiple frames, surgical index data, etc.) can be stored, and a number of applications run on the device. A program (application program or application), data for operation of the device 100, and commands can be stored. At least some of these applications may be downloaded from an external server via wireless communication.

The memory 110 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), or a multimedia card micro type. micro type), card-type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), EEPROM (electrically erasable) It may include at least one type of storage medium among programmable read-only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk.

Additionally, the memory 110 is separate from the device, but may include a database connected by wire or wirelessly. That is, the database shown in FIG. 1 may be implemented as a component of the memory 110.

The communication module 120 may include one or more components that enable communication with an external device, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module. may include.

Wired communication modules include various wired communication modules such as Local Area Network (LAN) modules, Wide Area Network (WAN) modules, or Value Added Network (VAN) modules, as well as USB (Universal Serial Bus) modules. ), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).

In addition to Wi-Fi modules and WiBro (Wireless broadband) modules, wireless communication modules include GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), and UMTS (universal mobile telecommunications system). ), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, 6G, etc. may include a wireless communication module that supports various wireless communication methods.

The display 130 displays (outputs) information processed by the device 100 (for example, a patient's surgical image, feature data output through specific frames constituting the surgical image, surgical index data, etc.). For example, the display may display execution screen information of an application (for example, an application) running on the device 100, or UI (User Interface) and GUI (Graphic User Interface) information according to such execution screen information. You can.

The input module 140 is for receiving information from the user. When information is input through the user input unit, the processor 150 can control the operation of the device 100 to correspond to the input information.

The input module 140 includes hardware-type physical keys (e.g., buttons, dome switches, jog wheels, jog switches, etc. located on at least one of the front, back, and sides of the device) and software-type keys. May include touch keys. As an example, the touch key consists of a virtual key, soft key, or visual key displayed on the touch screen type display 130 through software processing, or the above It may consist of a touch key placed in a part other than the touch screen. Meanwhile, the virtual key or visual key can be displayed on the touch screen in various forms, for example, graphics, text, icons, videos, or these. It can be made up of a combination of .

The processor 150 may control the overall operation and functions of the device 100. Specifically, the processor 150 has a memory that stores data for an algorithm for controlling the operation of components within the device 100 or a program that reproduces the algorithm, and performs the above-described operations using the data stored in the memory. It may be implemented with at least one processor (not shown). At this time, the memory and processor may each be implemented as separate chips. Alternatively, the memory and processor may be implemented as a single chip.

In addition, the processor 150 can control any one or a combination of the above-described components in order to implement various embodiments according to the present disclosure described in FIGS. 3 to 6 below on the device 100. You can.

Figure 3 is a flowchart for explaining a method of detecting bleeding during surgery through a surgical image performed by a device, according to an embodiment of the present disclosure.

Referring to FIG. 3, the processor 150 of the device 100 acquires first temporal feature data through a surgical image set at a first frame rate and second temporal feature data through a surgical image set at a second frame rate. can be obtained (S310).

Here, the first temporal feature data and the second temporal feature data may mean temporal context information at different tempos.

The processor 150 may input each of the surgical image set to the first frame rate and the surgical image set to the second frame rate into the second AI model to obtain first temporal feature data and second temporal feature data.

The second AI model may be a convolutional neural network (CNN) model consisting of one or more convolutional layers that perform convolutional operations. The second AI model can be trained in advance to recognize objects included in the surgery image.

And, the first frame rate may be set to a value smaller than the second frame rate. The first temporal feature data acquired through a surgical image set to a first frame rate may be feature data based on a slow pathway. The second temporal feature data acquired through the surgical image set to the second frame rate may be feature data based on a fast pathway (slow pathway).

Since feature data based on the fast pathway has a lower channel capacity, the amount of computation related to the feature data based on the fast pathway may be small. Therefore, feature data based on fast path may have high temporal modeling ability.

In contrast, feature data based on the slow pathway has high channel acceptance and may have a large ability to express spatial semantic characteristics.

Referring to FIG. 4, the processor 150 generates first temporal feature data (or feature data based on slow pathway) 410-1 based on a surgical image composed of a plurality of frames (or clips) 400. And second temporal feature data (or feature data based on fast pathway) 420-1 can be obtained.

The processor 150 may acquire spatial feature data through a representative frame among the plurality of frames constituting the surgical image (S320). Here, the representative frame may mean any frame among a plurality of frames or a center frame located in the middle in time.

Referring to FIG. 4, the processor 150 may obtain spatial feature data by inputting the intermediate frame 420 into the AI model 430 learned to perform a semantic segmentation algorithm. Spatial feature data may include data indicating whether active bleeding exists in an area on the middle frame 430.

The processor 150 may apply a sigmoid function after adjusting the size of the spatial feature data. Additionally, the processor 150 may classify the spatial feature data into first class spatial feature data 440-1 and second class spatial feature data 440-2.

Here, the first class of spatial feature data includes feature data for the background area (or non-bleeding area) in the surgical image, and the second class of spatial feature data includes feature data for the active bleeding area in the surgical image. It may include feature data for

The processor 150 may acquire first intermediate data based on the first temporal feature data and spatial feature data, and acquire second intermediate data based on the second temporal feature data and spatial feature data (S330).

Here, intermediate data refers to data that is a fusion of temporal feature data (e.g., temporal annotation data indicating the time when bleeding occurred, etc.) and spatial feature data (e.g., spatial annotation data indicating the location of bleeding, etc.). can do.

The processor 150 may utilize spatiotemporal feature data as learning data to train an AI model to detect intraoperative bleeding through surgical images. To generate spatiotemporal feature data, the device may generate intermediate data based on temporal feature data and spatial feature data.

Referring to FIG. 4, the processor 150 performs a Hadamard product between the spatial feature data of the first class and the first temporal feature data and the result of performing the Hadamard product between the spatial feature data of the second class and the first temporal feature data. First temporal-space specific data 450-1 can be obtained by adding up the results of the Mar multiplication.

In addition, the processor 150 performs a Hadamard product between first class spatial feature data and second temporal feature data and a Hadamard product performance result between second class spatial feature data and second temporal feature data. By summing, second temporal-spatial feature data 450-2 can be obtained.

Additionally, the processor 150 may obtain first intermediate data 460-1 by adding the first temporal-spatial feature data 450-1 and the first temporal feature data 410-1. And, the device may obtain second intermediate data 460-2 by adding the second temporal-spatial feature data 450-2 and the second temporal feature data 410-2.

The processor 150 may generate learning data by concatenating the first intermediate data and the second intermediate data (S340). Additionally, the processor 150 may train the first AI model to output surgical index data that allows identification of the number of bleedings and bleeding sections within the surgical image based on the learning data (S350).

Referring to FIG. 4, the processor 150 generates training data by concatenating the first intermediate data 450-1 and the second intermediate data 450-2, and based on the generated training data, the processor 150 generates training data. 1 AI model can be trained.

FIG. 5 is a diagram illustrating a method of performing learning and inference steps related to a first AI model, according to an embodiment of the present disclosure.

In the training step, the processor 150 uses a first AI model (or a fusion model or AMAGI (Image segmentation-guided active bleeding detection model) (520) can be trained.

For example, the learning data includes data related to the time when bleeding occurred (e.g., data shown in (a) of Figure 6) and data related to the spatial area where bleeding occurred (e.g., data shown in (b) of Figure 6). It can be configured based on

In the inference (or test) step, the processor 150 acquires a plurality of frames 540-1, 540-2, ... 540-N from a specific surgery image (e.g., Gastrectomy) 530. You can. At this time, the number of multiple frames may vary depending on the frame rate.

The processor 150 inputs a plurality of frames 540-1, 540-2, ... 540-N into the learned first AI model 520, thereby determining the number of bleeding and bleeding duration within the surgical image. Surgery index data 550 that allows identification can be obtained. That is, the surgical index data may be configured to indicate a time region in which bleeding is detected for the entire surgery in a specific area of the surgical image.

The processor 150 may perform post-processing on the surgical index data. For example, the device may perform noise filtering on surgical index data through a filter capable of filtering out noise. The processor 150 may identify the number of bleeding and the bleeding section within the surgical image based on post-processed information (i.e., measure surgical index data).

That is, the processor 150 may identify/detect the number of bleedings and bleeding sections in one or more specific areas (i.e., areas where bleeding occurs) within the surgical image through the first AI model 520.

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. Instructions may be stored in the form of program code, and when executed by a processor, may create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media storing instructions that can be decoded by a computer. For example, there may be Read Only Memory (ROM), Random Access Memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, etc.

As described above, the disclosed embodiments have been described with reference to the attached drawings. A person skilled in the art to which this disclosure pertains will understand that the present disclosure may be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

a memory storing at least one process for detecting intraoperative bleeding through surgical images; and

And a processor that performs an operation of detecting bleeding during the surgery as the process is executed,

The processor,

Obtaining first temporal feature data through the surgical image set to a first frame rate, and acquiring second temporal feature data through the surgical image set to a second frame rate,

Obtaining spatial feature data through a representative frame among the plurality of frames constituting the surgical image,

Obtaining first intermediate data based on the first temporal feature data and the spatial feature data, and acquiring second intermediate data based on the second temporal feature data and the spatial feature data,

Generate learning data by concatenating the first intermediate data and the second intermediate data,

Based on the learning data, a first artificial intelligence (AI) model is used to output surgical index data that can identify the bleeding count and bleeding duration in the surgical image. Learning device.
According to paragraph 1,

The processor,

When acquiring the first temporal feature data and the second temporal feature data, each of the surgical image set to the first frame rate and the surgical image set to the second frame rate is input to a second AI model to obtain the first Obtaining 1 temporal feature data and the second temporal feature data,

The second AI model includes a convolutional neural network (CNN) model.
According to paragraph 1,

The spatial feature data is classified into first class spatial feature data and second class spatial feature data,

The spatial feature data of the first class includes feature data for a background area in the surgical image,

The second class of spatial feature data includes feature data for an active bleeding area in the surgical image.
According to paragraph 3,

The processor,

When acquiring the first intermediate data, a Hadamard product performance result between the spatial feature data of the first class and the first temporal feature data and the spatial feature data of the second class and the first temporal feature Obtaining first temporal-space specific data by adding up the results of performing the Hadamard product between the data,

When acquiring the second intermediate data, a Hadamard product performance result between the spatial feature data of the first class and the second temporal feature data and the spatial feature data of the second class and the second temporal feature An apparatus for obtaining the second temporal-spatial feature data by summing the results of performing a Hadamard product between data.
According to clause 4,

The processor,

When acquiring the first intermediate data, obtain the first intermediate data by adding the first temporal-spatial feature data and the first temporal feature data,

When acquiring the second intermediate data, the device acquires the second intermediate data by adding the second temporal-spatial feature data and the second temporal feature data.
According to paragraph 1,

The processor,

Perform post-processing on the surgical index data,

A device that identifies the number of bleeding and the bleeding section in the surgical image based on the information on which the post-processing was performed.
According to paragraph 1,

The first frame rate is set to a value smaller than the second frame rate.
In clause 7,

The representative frame is a center frame among the plurality of frames.
In a method of detecting bleeding during surgery through surgical images, performed by a device,

Obtaining first temporal feature data through the surgical image set to a first frame rate and acquiring second temporal feature data through the surgical image set to a second frame rate;

Obtaining spatial feature data through a representative frame among a plurality of frames constituting the surgical image;

Obtaining first intermediate data based on the first temporal feature data and the spatial feature data, and acquiring second intermediate data based on the second temporal feature data and the spatial feature data;

generating learning data by concatenating the first intermediate data and the second intermediate data; and

Based on the learning data, a first artificial intelligence (AI) model is used to output surgical index data that can identify the bleeding count and bleeding duration in the surgical image. A method comprising the step of learning.
According to clause 9,

Obtaining the first temporal feature data and the second temporal feature data includes:

Inputting each of the surgical image set to the first frame rate and the surgical image set to the second frame rate into a second AI model to obtain the first temporal feature data and the second temporal feature data; ,

The method wherein the second AI model includes a convolutional neural network (CNN) model.
According to clause 9,

The spatial feature data is classified into first class spatial feature data and second class spatial feature data,

The spatial feature data of the first class includes feature data for a background area in the surgical image,

The method wherein the second class of spatial feature data includes feature data for an active bleeding area in the surgical image.
According to clause 9,

The step of acquiring the first intermediate data is,

By adding up the Hadamard product performance results between the spatial feature data of the first class and the first temporal feature data and the Hadamard product performance results between the spatial feature data of the second class and the first temporal feature data, comprising acquiring first temporal-space specific data,

The step of acquiring the second intermediate data is,

By adding up the Hadamard product performance results between the spatial feature data of the first class and the second temporal feature data and the Hadamard product performance results between the spatial feature data of the second class and the second temporal feature data, Obtaining the second temporal-spatial feature data.
According to clause 12,

The step of acquiring the first intermediate data is,

Obtaining the first intermediate data by adding the first temporal-spatial feature data and the first temporal feature data,

The step of acquiring the second intermediate data is,

Adding the second temporal-spatial feature data and the second temporal feature data to obtain the second intermediate data.
According to clause 9,

Performing post-processing on the surgical index data; and

The method further comprising identifying the number of bleeding and the bleeding section in the surgical image based on the information on which the post-processing was performed.
According to clause 14,

The first frame rate is set to a value smaller than the second frame rate,

The representative frame is a center frame among the plurality of frames.