WO2020159276A1

WO2020159276A1 - Surgical analysis apparatus, and system, method, and program for analyzing and recognizing surgical image

Info

Publication number: WO2020159276A1
Application number: PCT/KR2020/001475
Authority: WO
Inventors: 이연주
Original assignee: 주식회사 아이버티
Priority date: 2019-02-01
Filing date: 2020-01-31
Publication date: 2020-08-06

Abstract

The present invention relates to a system, method, and program for analyzing and recognizing a surgical image. A method for analyzing surgery by using a computer according to an embodiment of the present invention comprises: a step for acquiring a surgical image by a computer; a step for inputting, by the computer, the surgical image to one or more surgical element recognition models; a surgery recognition result combination acquisition step for acquiring, by the computer, a combination of surgery recognition results calculated by each surgical element recognition model, wherein the one or more surgical element recognition models are included in a surgical element recognition layer, which is the lowest level in a surgical analysis system; and a surgical analysis result acquisition step for acquiring, by the computer, one or more analysis results by inputting the combination of surgery recognition results to one or more surgical analysis models, wherein the one or more surgical analysis models are included in a surgical analysis layer on the surgical element recognition layer and selected according to a request of a user.

Description

Surgical analysis device, surgical image analysis and recognition system, method and program

The present invention relates to a surgical image analysis and recognition method.

Medical or medical procedures (i.e., medical treatment acts) can be classified into open surgery, laparoscopic surgery and robotic surgery, including minimally invasive surgery (MIS), radiation therapy, and endoscopic surgery. . Laparoscopic surgery refers to surgery performed by a medical staff who directly sees and touches the area to be treated, and minimally invasive surgery is also called keyhole surgery. Laparoscopic surgery and robotic surgery are typical. Laparoscopic surgery is performed through a video monitor by inserting a surgical instrument into the body and a laparoscopic instrument with a special camera by making a small hole in the required part without opening, and performing microsurgery using a laser or special instrument. In addition, robot surgery is to perform minimally invasive surgery using a surgical robot. Furthermore, radiation therapy refers to the surgical treatment with radiation or laser light from outside the body. In addition, the endoscopic procedure refers to a procedure performed by inserting an endoscope into a digestive system or the like and inserting a tool into a passage provided in the endoscope.

In the case of such medical treatment, in many cases, an image is acquired based on actual practice. That is, laparotomy is performed by a medical staff while directly viewing the patient's organs with the naked eye, and laparoscopic or endoscopic procedures are performed while viewing images obtained through laparoscopic or endoscopic procedures. Therefore, it is important to provide a variety of information to medical staff through images obtained when performing treatment.

In addition, there is a need to develop technologies capable of providing information to assist doctors in medical procedures or medical procedures. In order to provide information for assisting a surgery or procedure, it is important to recognize an operation or a variety of surgery information performed in the surgery process or procedure and grasp the meaning of the recognized information. Accordingly, there is a need to develop a technology that allows a computer to recognize motion or various surgical information from an image.

In addition, deep learning has been widely used in the analysis of surgical images in recent years. Deep learning is defined as a set of machine-learning algorithms that attempt high-level abstractions (abstraction of key content or functions in large amounts of data or complex data) through a combination of several nonlinear transformation methods. Deep learning can be seen as a field of machine learning that teaches a person's mindset to computers in a large framework.

In order to analyze or recognize a surgical image, various surgical elements included in the surgical image must be recognized. However, each surgical element included in the surgical image is difficult to acquire with only one learning model.

Therefore, the present invention is a surgical analysis device, a surgical image analysis and recognition system, and a method capable of quickly obtaining accurate recognition results by recognizing each of a plurality of surgical elements in a surgical image through a surgical element recognition model formed in parallel. And programs.

In addition, the present invention can obtain all or selectively various analysis results required for surgical analysis through a plurality of surgical analysis models capable of calculating each result using multiple surgical element recognition results calculated from the surgical element recognition model. The present invention is to provide a surgical analysis device, a surgical image analysis and recognition system, a method, and a program.

In addition, the present invention includes a plurality of surgical solution model that easily calculates the service result required for a medical staff based on a number of surgical analysis results through a plurality of surgical analysis models, surgical analysis device, surgical image analysis and recognition system, method And programs.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A computer-aided surgical analysis method according to an embodiment of the present invention includes: a computer acquiring a surgical image; A computer inputting the surgical image into one or more surgical element recognition models; A computer acquiring a combination of surgical recognition results calculated from each surgical element recognition model, wherein one or more surgical element recognition models are included in a surgical element recognition layer at a lowest level in the surgical analysis system; And a computer inputs the combination of the surgical recognition results into one or more surgical analysis models to obtain one or more analysis results, wherein the one or more surgical analysis models are included in the surgical analysis layer above the surgical element recognition layer. It is selected according to, surgical analysis result acquisition step; includes.

In addition, in another embodiment, each surgical analysis model in the surgical analysis layer is characterized in that a connection relationship is established with one or more surgical element recognition models in the surgical element recognition layer based on data required for analysis.

In addition, in another embodiment, the surgical element recognition model includes: an organ recognition model that recognizes an organ in the surgical image; A surgical tool recognition model for recognizing the movement of the surgical tool and the surgical tool in the surgical image; And an event recognition model that recognizes an event occurring within the surgical image, and the event is a non-ideal situation during surgery including bleeding.

In addition, in another embodiment, the surgical analysis layer, blood that calculates the degree of blood loss based on a combination of surgical recognition results including types of surgical organs recognized by the surgical element recognition layer, surgical operations, and events occurring during surgery Loss recognition model; And a long-term damage detection model for calculating the degree of long-term damage based on a combination of surgical recognition results including the operation step and the operation time recognized by the surgical element recognition layer, and includes the blood loss recognition model and the long-term damage detection model Is used to calculate the analysis results for each surgical procedure during or after surgery.

In addition, in another embodiment, when analyzing the surgical results after the surgery is completed, the surgical analysis layer is a surgical tool recognized by the surgical element recognition layer, an organ performing an operation with the surgical tool, and the surgical tool It further includes; a surgical operation misuse detection model, which detects the use of the wrong surgical instrument, based on the detailed surgical operation during the entire operation performed, and an event generated in the detailed surgical operation.

In addition, in another embodiment, the surgical analysis layer further includes an optimal surgical plan calculation model for calculating an optimal surgical plan based on a combination of surgical recognition results obtained from the surgical element recognition layer, but the optimal surgical plan The calculation model is characterized in that, by analyzing the surgical image data acquired in real time in real time, an optimal surgical plan to be performed later is calculated and provided.

In addition, in another embodiment, the surgical analysis layer further includes an optimal surgical plan calculation model for calculating an optimal surgical plan based on a combination of surgical recognition results obtained from the surgical element recognition layer, but the optimal surgical plan The calculation model is characterized by calculating and providing a surgical plan for a real patient based on a combination of surgical recognition results for a virtual operation performed on a patient's virtual body model before surgery.

In addition, in another embodiment, the computer inputs one or more analysis results into a specific surgical solution model, thereby providing a summary of the surgical output based on the analysis results.

In addition, in another embodiment, the surgical solution model includes: a surgical evaluation model for calculating an evaluation result for surgery based on one or more analysis results obtained from the surgical analysis layer; A chart generation model that generates a chart for surgery based on one or more analysis results obtained from the surgical analysis layer; And a surgical complexity calculation model for calculating the complexity or difficulty of surgery based on one or more analysis results obtained from the surgical analysis layer.

In addition, in another embodiment, as learning one or more analysis results obtained from the surgical analysis layer, the surgical Q&A model that calculates an answer to a question about the surgery; includes, and the surgical output providing step, the medical staff The answer is calculated and provided by entering a question about a specific surgery.

A computer-aided surgical analysis program according to another embodiment of the present invention may be stored in a medium in combination with a computer that is hardware, and to execute the computer-aided surgical analysis method.

The surgical image analysis apparatus according to another embodiment of the present invention includes one or more surgical element recognition models for calculating a surgical recognition result as a surgical image is input, wherein one or more surgical element recognition models are the lowest in the surgical analysis system. A surgical element recognition layer included in the level of the surgical element recognition layer; And one or more surgical analysis models for obtaining an analysis result based on a combination of surgical recognition results that are a combination of results provided by the one or more surgical element recognition models, wherein the surgical analysis layer is formed as an upper layer of the surgical element recognition layer. ;.

In addition, in another embodiment, a surgical solution providing layer including one or more surgical solution models that provide a surgical output arranged based on one or more analysis results obtained from the surgical analysis layer; further includes a.

According to the present invention as described above, each surgical element can be accurately recognized by an individual surgical element recognition model that recognizes individual surgical elements (surgical instruments, bleeding, camera, etc.) in the surgical image. That is, it is possible to provide a recognition result with high accuracy compared to a method of recognizing various surgical elements included in a surgical image through one recognition model.

In addition, according to the present invention, since surgical analysis is performed by combining the surgical element recognition results calculated by the surgical element recognition model, it is possible to provide accurate surgical analysis results to the medical staff.

In addition, according to the present invention, as each surgical solution model automatically calculates a result required for a medical staff by inputting one or more surgical analysis results, the post-operative work of the medical staff can be simplified.

In addition, according to the present invention, when a new surgical element recognition is required, when a new surgical analysis is required or when a new service type solution is required, a new surgical element recognition model, a surgical analysis model, or a surgical solution model in the surgical analysis system To add a new function, you only need to add and form a connection relationship with one or more models included in the upper layer or lower layer.

1 is a layer configuration diagram of a surgical analysis device according to an embodiment of the present invention.

Figure 2 is a flow chart of a computer-aided surgical analysis method according to an embodiment of the present invention.

3 is a flow chart of a computer-aided surgical analysis method further comprising a procedure for providing a surgical output according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the embodiments allow the publication of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform the person having the scope of the invention, and the present invention is only defined by the scope of the claims. The same reference numerals refer to the same components throughout the specification.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless specifically defined.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components other than the components mentioned.

In this specification, the'surgery image' is an image of the surgical procedure. For example, the surgical image includes an image obtained by an endoscope inserted into the body during laparoscopic surgery including robotic surgery. Further, the surgical image may include a surgical image performed through an endoscope inserted through the oral cavity or anus.

In this specification, "virtual body model" refers to a model generated in conformity with an actual patient's body based on medical image data. The "virtual body model" may be generated by modeling medical image data in three dimensions as it is, or may be corrected as in actual surgery after modeling. The virtual body model may be used for guiding or navigation during surgery, post-surgery analysis, and the like. In addition, the virtual body model may be implemented in the same manner as the actual patient body by reflecting the color, texture, elasticity, etc. of the actual patient body.

As used herein, "virtual surgery data" means data including rehearsal or simulation actions performed on a virtual body model. The "virtual surgery data" may be image data in which rehearsal or simulation is performed on a virtual body model in a virtual space, or may be data recorded on a surgical operation performed on a virtual body model.

In this specification, "actual surgical data" refers to data obtained as actual medical personnel perform surgery. The "actual surgical data" may be image data obtained by photographing a surgical site in an actual surgical procedure, or may be data recorded about a surgical operation performed in an actual surgical procedure.

In this specification, "computer" includes all of various devices capable of performing arithmetic processing and providing a result to a user. For example, the computer is not only a desktop PC and a notebook, but also a smart phone, a tablet PC, a cellular phone, and a personal communication service phone (PCS phone), synchronous/asynchronous. Mobile terminals of the International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), and a Personal Digital Assistant (PDA) may also be applicable. In addition, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. Also, the computer may be a server that receives a request from a client and performs information processing.

Hereinafter, with reference to the drawings, a detailed description of a computer-aided surgical analysis method, a program, and a surgical analysis apparatus according to embodiments of the present invention will be described.

1 is a block diagram of a surgical image analysis apparatus according to an embodiment of the present invention.

Referring to Figure 1, the surgical image analysis apparatus according to an embodiment of the present invention, the surgical element recognition layer 10; And a surgical analysis layer 20.

The surgical element recognition layer is a layer for recognizing elements involved in surgery and elements generated by surgery. The elements involved in the surgery include a camera (eg, an endoscope used in laparoscopic surgery), a surgical tool, an organ, a blood vessel, and the like, for photographing a surgical image. In addition, the elements generated by the surgery include the operation of the surgical tool, an event such as bleeding, and the like. The surgical element recognition layer forms the lowest level in the surgical analysis system.

The surgical element recognition layer includes one or more surgical element recognition models for calculating a surgical recognition result as a surgical image is input. Hereinafter, a detailed description of the surgical element recognition model in each surgical element recognition layer will be described.

A plurality of surgical element recognition models may have a connection relationship with each other. That is, the recognition result of the B surgical element recognition model may be additionally used in order for the A surgical element recognition model to calculate the recognition result.

In one embodiment, an organ recognition model for recognizing an organ in the surgical image is included. That is, the organ recognition model serves to recognize one or more organs existing in each image frame in the surgical image. In addition, the organ recognition model may include a blood vessel model composed of arteries and veins.

The organ recognition model can recognize organs in the surgical image in various ways. For example, the organ recognition model may calculate an organ or blood vessel in which a surgical operation is performed based on a surgical step in a camera position obtained in a camera position recognition model described later or in a surgical step recognition model described later.

In another embodiment, a surgical tool recognition model for recognizing a surgical tool (Instrument) in the surgical image is included. The surgical tool recognition model may recognize the type of surgical tool appearing in the image. For example, the surgical tool recognition model may recognize images of surgical instruments appearing in each frame according to the placement by learning the images of each surgical tool. Further, for example, the surgical tool recognition model may recognize a surgical tool performing a surgical operation inside the body as each surgical tool learns a plurality of surgical image data performing an operation on an organ.

In another embodiment, the surgical tool motion (Instrument Action) recognition model; further includes. That is, the surgical tool motion recognition model plays a role of recognizing the meaning of the motion of the surgical tool performed in a specific surgical step (that is, an operation for obtaining a result).

Specifically, the surgical tool motion recognition model recognizes a basic surgical motion by acquiring a surgical image (or a specific section in the surgical image) that requires recognition of motion semantics and learning a plurality of image frames included in the surgical image. The basic surgical operation refers to a basic operation such as cutting and grasping using a specific surgical tool. Thereafter, the surgical tool motion recognition model extracts a set of consecutive video frames from the plurality of video frames based on the recognized surgical motion, and derives the meaning of the unit surgical motion through learning. The unit surgical operation refers to a surgical operation having a meaning for producing a specific result as an action for a specific organ in the surgical process.

To this end, the surgical tool motion recognition model includes: a basic surgical motion recognition module that recognizes a basic surgical motion that is a minimum unit of a surgical motion that does not reflect meaning during surgery; And a surgical meaning recognition module that recognizes a unit surgical operation based on the continuous basic surgical operation.

In one embodiment, as the basic surgical motion recognition module learns an image of a surgical tool, it recognizes the surgical tool in a continuous frame and calculates a state change of the surgical tool to recognize the basic surgical action of the surgical tool.

In addition, in one embodiment, as the surgical meaning recognition module trains learning data that matches the continuous basic surgical operation and the unitary surgical operation, the continuous basic in the new surgical image recognized by the basic surgical operation recognition module When surgical operation data is input, a unit surgical operation is calculated. In addition, at this time, the learning data may further include organ information on which a continuous basic surgical operation is performed, and the organ information may be used as a result calculated by the organ recognition model in the surgical element recognition layer.

In addition, in another embodiment, a camera position recognition model for recognizing the position of the camera during minimally invasive surgery is further included. Specifically, when the camera is an endoscopic used in laparoscopic surgery, the camera may be a forward/reverse, up/down/left/right movement within the abdominal cavity after the endoscope is inserted into the abdominal cavity. It serves to calculate the position of the camera in real time within the abdominal cavity.

In one embodiment, the camera position recognition model sets a reference position of a camera based on one or more reference points of a reference object in an image photographed by the camera, and calculates a relative position change based on a change in an object in the image Calculate camera position in real time.

Specifically, in the method of calculating the camera position using the actual surgical image, the camera position recognition model obtains a reference object from the actual surgical image photographed by the camera entering the body, sets a reference position for the camera, and the camera As it moves, the position change amount of the camera is calculated. For example, the camera position recognition model is based on an object change in a real-time image acquired by the camera (for example, a change in size of an object, a change in the position of an object, a change in an object appearing in the image, etc.). Calculate. Then, the camera position recognition model calculates the current position of the camera based on the amount of change of the position of the camera with respect to the reference position. At this time, the actual surgical image may be a stereoscopic 3D image, and accordingly, the actual surgical image may be a three-dimensional stereoscopic image, that is, an image having a depth sense. Therefore, it is possible to accurately grasp the position in the three-dimensional space of the surgical tool through the depth map of the actual surgical image.

The reference object is easy to detect a feature from an image, exists in a fixed position inside the body, has little or no movement during surgery, does not cause shape deformation, is not affected by surgical tools, and medical image data It is also possible to use an organ or internal specific site that satisfies at least one of the conditions such as that it should be possible to acquire by (eg, data taken by CT, PET, etc.). For example, a portion that has very little motion during surgery, such as the liver and abdominal wall, and a portion that can be obtained from medical image data such as the stomach, esophagus, and gallbladder may be determined as a reference object.

In addition, when a movement of the camera position occurs, a surgical tool in an image photographed by the camera as a reference object may be used as a reference object. In general, when performing robotic surgery, since the camera movement and the movement of the surgical tool do not occur at the same time, when the camera is moved, the surgical tool stopped at a specific position may be used as a reference object.

In addition, the camera position recognition model may repeat the process of resetting the reference position of the camera based on one or more reference points of the reference object when the reference object appears in the surgical image. In other words, if the camera movement is frequent, an error may occur in the camera position as the relative movement accumulates from the reference position. Therefore, the camera position recognition model performs a process of resetting the camera reference position when a new reference object appears in the image in real time. It can be done.

In addition, the camera position recognition model can accumulate the camera position in the virtual body model generated in the same way as the actual patient's body. That is, the camera position recognition model may accumulate the relative position change with respect to the reference position of the camera after matching the actual reference object in the actual patient body with the virtual reference object in the virtual body model. Through this, the camera position recognition model can record in which 3D virtual body model the path the camera moved during surgery.

In addition, in another embodiment, the camera position recognition model calculates the position of each surgical tool relative to the position of the camera as it recognizes the position of the camera. That is, the camera position recognition model calculates the relative positions of one or more surgical instruments from a specific position of the camera.

In addition, in another embodiment, a surgical tool position calculation model is further included. As an embodiment of the surgical tool position calculation model, the position of the first point of the surgical tool relative to the external space of the surgical subject based on the sensing information obtained from the sensing device attached to the surgical tool inserted into the internal space of the surgical subject Calculate and reflect the characteristic information of the surgical tool based on the position of the first point of the surgical tool through a virtual body model generated in accordance with the physical condition of the surgical subject. Calculate the 2 point position. Through this, the surgical tool position calculation model provides the position information of the surgical tool in the actual body internal space of the surgical subject based on the position of the second point of the surgical tool with respect to the virtual body model.

As another embodiment of the surgical tool position calculation model, the surgical tool is received by applying motion data when the surgical tool of the surgical robot system is moved to the virtual body model by matching the coordinate system of the surgical robot system with the coordinate system of the virtual body model. Calculate real-time location. Specifically, the surgical tool position calculation model is based on a reference point (eg, a specific location, such as a belly button of a patient's body, a marker displayed on a patient's body surface, or an identification mark projected by a surgical robot system on the body surface) of a surgical subject. Acquire the coordinate information of the surgical robot including the surgical tool, match the coordinate information of the virtual body model generated in accordance with the body condition of the surgical target with the coordinate information of the surgical robot, and perform the surgery obtained from the coordinate information of the surgical robot The position of the surgical tool in the virtual body model corresponding to the position of the tool is calculated. That is, the surgical tool position calculation model can acquire the real-time position of the surgical tool by applying the surgical tool movement of the surgical robot into the virtual body model as the surgical robot actually matches the coordinate system of the actual patient body and the virtual body model. have.

In addition, in another embodiment, a surgical step recognition model is further included. The surgical step recognition model calculates what detailed surgical step corresponds to a process in which the current surgery is performed or a specific section in the surgical image among the entire surgical procedures. The surgical stage recognition model can calculate the surgical stage of the surgical image in various ways. For example, the surgical stage recognition model stores a progression sequence for each surgical type, and can recognize a specific surgical phase (Phase) in the progression sequence based on the area where the camera is located in the patient's body.

Also, in another embodiment, an event occurring in the surgical image is recognized, and the event includes an event recognition model, which is a non-ideal situation during surgery including bleeding. The event recognition model (eg, bleeding recognition model) may include a bleeding recognition module; And a bleeding amount calculation module.

The bleeding presence recognition module recognizes whether a bleeding area exists in the surgical image based on deep learning-based learning, and recognizes a bleeding position in the image frame. The bleeding presence recognition module may recognize whether a bleeding area is included in a new image by learning a plurality of images including bleeding.

In one embodiment, the presence/absence recognition module may convert each pixel in the surgical image to a specific value based on the feature map including the feature information, and specify the bleeding area based on the specific value of each pixel in the surgical image. . Each pixel in the surgical image may be converted into a specific value based on a predetermined weight according to whether it is an area corresponding to the characteristic of the bleeding based on the feature map (that is, according to the degree of affecting the characteristic of the bleeding). For example, the bleeding presence recognition module applies Gradient CAM (Gradient-weighted Class Activation Mapping) technology that inversely estimates the learning result of recognizing the bleeding area in the surgical image through CNN, so that each pixel of the surgical image is identified by a specific value. Can be converted to The bleeding presence recognition module assigns a high value (eg, a high weight) to pixels recognized as corresponding to the bleeding area in the surgical image based on the feature map, and a low value to the pixel recognized as not corresponding to the bleeding area. Each pixel value of the surgical image may be converted in a manner of giving (eg, a low weight). The presence or absence of the bleeding module may highlight the portion of the bleeding area in the surgical image through the converted pixel value, and thereby segment the bleeding area to estimate the location of the corresponding area. In addition, the bleeding presence recognition module may apply a Grad CAM technology to generate a heat map for each pixel in the surgical image based on the feature map and convert each pixel to a probability value. The bleeding presence recognition module may specify a bleeding area in the surgical image based on the probability value of each converted pixel. For example, the bleeding presence recognition module may determine a pixel area having a large probability value as a bleeding area.

The bleeding amount calculation module is characterized in that it calculates the bleeding amount in the bleeding area based on the location of the bleeding area. In one embodiment, the bleeding amount calculation module may calculate the bleeding amount using pixel information of the bleeding area in the surgical image. For example, the bleeding amount calculation module may calculate the bleeding amount using the number of pixels corresponding to the bleeding area in the surgical image and color information (eg, RGB values) of the pixels.

In another embodiment, when the surgical image is a stereoscopic image, the bleeding amount calculation module acquires depth information of a bleeding area in the surgical image based on a depth map of the surgical image, and bleeds based on the acquired depth information The amount of bleeding can be calculated by estimating the volume corresponding to the region. When the surgical image is a stereoscopic image, since it has three-dimensional stereoscopic sense, that is, depth information, it is possible to grasp the volume in the three-dimensional space for the bleeding area. For example, the bleeding amount calculation module acquires pixel information (eg, the number of pixels, the position of pixels, etc.) of the bleeding area in the surgical image, and calculates a depth value of the depth map corresponding to the obtained pixel information of the bleeding area. Can. The bleeding amount calculation module may calculate the bleeding amount by grasping the volume of the bleeding area based on the calculated depth value.

In another embodiment, when the bleeding amount calculation module includes gauze in the surgical image, the bleeding amount in the bleeding area may be calculated using gauze information. For example, the bleeding amount calculation module may be reflected in calculating the bleeding amount generated in the bleeding area using the number of gauze in the surgical image, color information of the gauze (eg, RGB value), and the like.

In addition, in another embodiment, the operation time calculation / prediction model further includes. The operation time calculation/prediction model includes an operation time prediction module for calculating an expected time to completion of an operation step being performed during operation; And an operation time calculation module that calculates a time when each step is performed after the operation is completed.

For example, the operation time calculation module extracts a time corresponding to a specific operation phase (Phase) after the operation is completed, and calculates the total time required for a specific operation phase (Phase).

In addition, for example, the operation time prediction module, based on the time required to a specific point in time of a particular surgical step and a specific surgical step prior to a specific point in time (eg, predicted reference point) during the operation, Calculate the remaining surgical time for the expected surgical step. Specifically, the surgical time prediction module acquires a predetermined surgical image including a surgical operation for a specific surgical step, and uses a surgical operation time obtained based on the preset surgical image and the preset surgical image. The learning data is generated, and learning is performed based on the learning data to predict the operation time in the specific surgical step.

The surgical analysis layer is a layer for calculating an analysis result based on surgical elements obtained from one or more surgical element recognition models. The surgical analysis layer is formed as an upper layer of the surgical element recognition layer. The surgical analysis layer includes one or more surgical analysis models for obtaining an analysis result based on a combination of surgical recognition results that are a combination of results provided in the one or more surgical element recognition models.

In one embodiment, the surgical analysis layer, a blood loss recognition model for calculating the degree of blood loss based on a combination of surgical recognition results including a type of surgical organ recognized by the surgical element recognition layer, a surgical operation, and an event during the operation. ;. To this end, the blood loss recognition model may be connected to at least one of an event recognition model, a long-term recognition model, and a surgical motion recognition model included in the surgical element recognition layer to receive a combination of surgical recognition results. For example, the blood loss recognition model, when used during surgery, can calculate the blood loss level when an event occurs and provide it to the medical staff.

In addition, in another embodiment, the surgical analysis layer, a long-term damage detection model for calculating the degree of long-term damage based on a combination of surgical recognition results including the operation step and the operation time recognized by the surgical element recognition layer; includes do. The long-term damage detection model can calculate the level of long-term damage by receiving a specific surgical step, a motion performed based on a specific surgical tool, and a surgical time from each surgical element recognition model.

The blood loss recognition model and the organ damage detection model are used to calculate the analysis results in each surgical procedure during or after surgery.

In another embodiment, the surgical analysis layer further includes an optimal surgical plan calculation model for calculating an optimal surgical plan based on a combination of surgical recognition results obtained from the surgical element recognition layer.

In one embodiment, the optimal surgical plan calculation model calculates and provides an optimal surgical plan to be performed later by analyzing the surgical image data previously obtained in real time.

In addition, in another embodiment, the optimal surgical plan calculation model is based on a combination of surgical recognition results for a virtual operation performed on a patient's virtual body model before surgery, and calculates and provides a surgical plan for a real patient.

In addition, in another embodiment, the optimal surgical plan calculation model may include a function of calculating an optimal entry position during minimally invasive surgery. Specifically, when the medical staff performs virtual surgery in the virtual body model, the virtual surgery system generates only the operation part of the surgical tool to perform a surgical simulation (ie, virtual surgery) without affecting the arm portion of the surgical tool. That is, virtual surgery is performed according to the surgical operation pattern of medical personnel without limitations on the characteristics of the patient's body (eg, organ placement, vascular condition, etc.) or the characteristics of surgical instruments. The optimal surgical plan calculation model calculates the optimal surgical tool entry position during actual surgery by considering the internal characteristics of the patient's body or the characteristics of the surgical tool based on the results after performing the virtual surgery. That is, as the candidate region of the entry position of the arm capable of performing the movement of the surgical tool operation unit is continuously calculated, the region capable of intersection is extracted.

As an example, when using a tool A, tool B, or tool C during surgical robot or laparoscopic surgery, the computer does not reach the operating part of tool A when performing a surgical operation (that is, it is operated due to the limitation of the length of the surgical tool) When an additional surgical operation is performed, a body surface area where a point that cannot be reached occurs) may be excluded from the entry range. In addition, the computer may exclude the surface area of the body that collides with the body organ or tissue in the process of performing the surgical operation by entering the tool A from the range of access. In addition, the computer may exclude the corresponding body surface point from the enterable range if the surgical tool is not implemented at a specific location after the surgical tool enters at each body surface point within the enterable range. Through this, the computer can calculate the entry range for the tool A. The computer can calculate the optimal entry position of each surgical tool by separately performing the process of calculating the accessible range for each surgical tool (eg, B tool and C tool). In addition, as described above, the computer may separately perform an entry range calculation process for each function based on the function of the surgical tool, thereby calculating an optimal entry position to which the function of each surgical tool can be applied.

As another example, when several surgical instruments need to enter one optimal entry position, the computer can extract an optimal entry range for each surgical tool and determine a range where a plurality of optimal entry ranges overlap as an optimal entry position. have. For example, when the tool A is changed to the D tool in the course of performing the surgery, the computer may calculate the overlapping region of the entry range for the A tool and the entry range for the D tool as an optimal entry position candidate region. . Since the location where the surgical tool can be entered is limited to a certain number (for example, 3), when changing from A tool to D tool, the same entry position must be used. A position that satisfies all the tool's entry ranges can be determined as the final optimal entry position.

As another example, when the same surgical tool is used multiple times in a virtual body model, all surgical operations are performed at a single surgical tool entry position when the surgical tool is operated in a wide range (ie, range of motion). Since it can be difficult to perform, the computer can divide the range in which the corresponding surgical tool is used (that is, the range of movement) into several groups reachable from a plurality of entry positions on the body surface. For example, when laparoscopic or robotic surgery is performed by generating three entry positions on the body surface, the computer divides the range of motion of the surgical tool into groups of three or less. At this time, the computer divides the range of motion based on whether or not it can be reached from a plurality of accessible ranges selected by other surgical instruments. In addition, a specific surgical tool having a wide range of motion (ie, the first surgical tool) is used simultaneously with another surgical tool (ie, the second surgical tool) and another surgical tool (ie, the second surgical tool) is required to enter When the optimal entry position is determined, the computer determines the range of motion of the first operation tool when it is used together with the second operation tool, where the first operation tool enters the optimal operation position of the second operation tool (that is, the second operation tool is entered). Keyhole). In addition, when the first surgical tool is continuously used in spite of the change of other surgical tools, the computer must be operated by entering the same entry position in consideration of the convenience of the user and the time required for the surgery. Can be set as a group.

Since the medical staff determines the optimal entry position of the surgical tool by reflecting the surgical simulation results performed without considering the entry position of the surgical tool and the long-term jamming of the arm portion of the surgical tool, the medical staff can perform the most convenient surgical operation. .

In addition, in another embodiment, the optimal surgical plan calculation model is based on a result of performing a virtual operation by only a motion part of a surgical tool in which a cancer part has been removed by a medical staff, and selecting an appropriate surgical tool type for use in surgery. It can be calculated and presented.

In addition, in one embodiment, the optimal surgical plan calculation model may perform training in various ways. For example, it is possible to calculate the optimal operation for each surgical type through reinforced learning by acquiring surgical images performed by a plurality of medical staff.

In addition, for example, the optimal surgical plan calculation model does not use the actual surgical data, but performs a virtual operation with a virtually generated surgical procedure and then repeats the process of evaluating whether it is an optimal surgical procedure and then the optimal surgical plan. Can be created.

Specifically, the optimal surgical plan calculation model is optimized by generating a plurality of genes corresponding to the surgical process based on a surgical process consisting of at least one detailed surgical operation, and performing virtual surgery on each of the plurality of genes. Evaluate surgery. Thereafter, the optimal surgical plan calculation model selects at least one gene among a plurality of genes based on the evaluation result, applies a genetic algorithm, applies a genetic algorithm to generate a new gene, and based on the new gene We derive the optimal surgical procedure.

The optimal surgical plan calculation model may calculate fitness by performing virtual surgery on a new gene (child gene). In addition, the optimal surgical plan calculation model determines whether the suitability of a new gene (child gene) meets a preset condition, selects a new gene (child gene) that meets the condition, and applies genetic algorithms such as mating and mutation. can do. By applying this genetic algorithm to a new gene (child gene), a new child gene can be generated again. That is, the computer can repeatedly generate child genes from the parent genes based on the fitness results for evaluating whether the surgery is optimal, and obtain a gene including the optimal surgical process among the finally generated child genes. have. For example, a gene having the highest suitability among child genes can be selected and derived through an optimized surgical procedure.

Each surgical analysis model in the surgical analysis layer may receive a combination of surgical recognition results in various forms.

In one embodiment, the surgical analysis model may be input in the form of connected code data by encoding the result calculated from the surgical element recognition model. That is, each surgical analysis model can acquire and input the code data necessary for analysis in each model after acquiring the code data connecting the codes for the surgical recognition results at each time point.

Further, in another embodiment, the surgical analysis model, for each of a plurality of image frames, based on the surgical recognition information calculated from the plurality of surgical element recognition model, the surgical elements included in the surgical recognition information (surgical elements) between Relational Representation information representing a relationship may be input. The surgical recognition information refers to the surgical element information recognized from the image frame, for example, surgical tools, surgical operations, body parts, bleeding status, surgical stage, surgical time (eg, remaining surgical time, surgical time, etc.), and camera information ( For example, it may include at least one surgical element among camera positions, angles, directions, movements, and the like. For example, the relationship expression may be in a matrix form in which rows and columns are arranged as respective surgical elements, and values for correlations between the surgical elements are applied as matrix values. In addition, as an example, the surgical analysis model may be input after calculating a relationship expression for each frame of a surgical image or a specific unit division image.

In addition, in another embodiment, the surgical solution providing layer; further includes. The surgical solution providing layer includes one or more surgical solution models providing surgical outputs arranged based on one or more analysis results obtained from the surgical analysis layer. In other words, the surgical solution provider layer includes one or more surgical solution models, which are a form of service that medical staff can use immediately.

The surgical solution providing layer is formed as an upper layer of the surgical analysis layer, and receives results calculated from each surgical analysis model in the surgical analysis layer.

In one embodiment, the surgical solution model includes a surgical evaluation model for calculating an evaluation result for surgery based on one or more analysis results obtained from the surgical analysis layer. For example, the surgical evaluation model may receive a blood loss result, an organ damage model calculation result, and a comparison result between an optimal surgical process and an actual surgery to calculate the evaluation of the surgery performed by the medical staff.

In addition, in another embodiment, the surgical solution model further includes a chart generation model that generates a chart for surgery based on one or more analysis results obtained from the surgical analysis layer. The chart generation model can automatically generate a record of the operation performed and the result of performing the operation. For example, the chart generation model combines the surgical elements obtained in the surgical process after the surgery is completed, inputs them into each surgical analysis model, and receives the analysis results of a plurality of surgical analysis models to enter a chart (for example, surgery Record paper) can be automatically generated (created).

In another embodiment, the surgical complexity calculation model for calculating the complexity or difficulty of surgery based on one or more analysis results obtained from the surgical analysis layer; further includes. For example, as the surgical complexity calculation model is used, the medical staff's performance evaluation may be performed in consideration of the difficulty of the surgery performed by each medical staff.

In addition, for example, when evaluating the surgical results, the surgical complexity calculated in the surgical complexity calculation model may be reflected. That is, in the surgical solution providing layer, the surgical evaluation model can be used for surgical evaluation by receiving the complexity calculation result calculated from the surgical complexity calculation model.

Also, as another embodiment, as learning one or more analysis results obtained from the surgical analysis layer, a surgical Q&A model for calculating an answer to a question about the surgery.

A detailed description of the case where the surgical image analysis apparatus according to an embodiment of the present invention is composed of a surgical element recognition layer, a surgical analysis layer, and a surgical solution layer is described.

In one embodiment, the lowest layer may be a surgical element recognition layer that recognizes surgical elements that are the smallest units that must be medically recognized in the medical procedure. The middle layer may be a surgical module layer that can grasp a medical meaning through each surgical element and determine a condition for making a medical diagnosis. The top layer may be a surgical solution layer that recognizes medical problems that may occur throughout the medical procedure and provides solutions to each problem.

The surgical element recognition layer may recognize surgical elements based on various surgical images photographed in the medical surgery process and various surgical related tools used in the medical surgery process. For example, the surgical elements include a surgical phase (Phase), a body part (eg, an organ), an event, an operation time (Time), a surgical instrument (Instrument), a camera (Camera), and a surgical operation (Acton). , And other elements.

In one embodiment, the surgical element recognition layer may recognize at least one surgical element based on the surgical image obtained in the medical surgery process. At this time, it is possible to recognize not only the surgical elements recognized in the surgical image itself, but also using the results of learning the surgical images. Using the above-described embodiments of the present invention, it is possible to effectively recognize surgical elements that are medically recognized minimum units in the medical procedure.

The surgical element recognition layer may individually recognize surgical elements. For example, when an organ is recognized from a surgical image, only the corresponding organ may be recognized, and when a surgical tool is recognized from a surgical image, only the corresponding surgical tool may be recognized.

Also, the surgical element recognition layer may use other surgical elements to recognize one surgical element. That is, the surgical element recognition layer may establish a primitive level relation representing a relationship between each surgical element, and recognize each surgical element using the primitive level relationship. For example, the primitive level relationship lists additional surgical elements necessary to recognize the corresponding surgical element, and the relationship between the surgical elements (eg, state change, position change, shape change, color change, arrangement relationship, etc.) It may include specified information. For example, when an event (eg, an event for bleeding) is recognized from a surgical image, additional surgical elements such as organs, surgical instruments, and surgical operations are further recognized based on a primitive level relationship for the event, The event can be recognized through additional recognized surgical elements.

The surgical analysis layer may grasp specific medical meanings or make specific medical judgments based on each surgical element recognized through the surgical element recognition layer. For example, the surgical analysis layer, using at least one surgical element, bleeding loss evaluation (Blood Loss Estimation), internal body damage detection (Anatomy Injury Detection; for example, organ damage detection), instrument misuse detection (Instrument Misuse Detection) , It is possible to grasp the medical meaning such as the optimal planning procedure (Optimal Planning Suggestion) or to determine the medical condition.

That is, the surgical analysis layer can configure each surgical module according to information necessary in the medical surgery process or a medical problem to be solved (eg, bleeding loss evaluation, internal body damage detection, tool misuse detection, optimal surgical procedure proposal, etc.). have. For example, the surgical analysis layer may constitute a bleeding loss evaluation module for grasping the degree of bleeding in a surgical subject during medical surgery. In addition, an internal damage detection module may be configured to determine how much damage has occurred in a specific organ during surgery. In addition, a surgical tool misuse detection module may be configured to determine whether a surgical tool is misused during surgery. At this time, each surgical module may selectively use at least one of the surgical elements recognized in the lower layer surgical element recognition layer.

In one embodiment, in selectively using at least one of the surgical elements, each surgical module may use a module level relation. The module level relationship may mean that the surgical elements to be recognized in the corresponding surgical module are determined and designated. For example, the module level relationship may be one in which surgical elements to be recognized in the corresponding surgical module are determined based on the degree to which a specific meaning can be recognized from the surgical image (eg, the above-described representative recognition value; SAM).

The surgical solution layer can finally solve a high-level medical problem using a medical meaning or medical judgment identified through the surgical analysis layer. For example, the surgical solution layer, using at least one surgical module, Chart Generation, Complication Estimation, Surgical Performance Assessment, Surgical Q&A System (Q&A using Surgical Bot) System).

At this time, the surgical solution layer may configure each solution module or system according to each medical problem (eg, chart generation, complications determination, surgical performance evaluation, surgical Q&A system, etc.). For example, the surgical solution layer may constitute a chart generating solution module (or system) for recording all information generated during the surgical process or for recording a patient's condition. In addition, a complications determination solution module (or system) for predicting complications that may occur after the surgical procedure may be configured. In this case, each solution module (or system) may selectively use the medical meaning or medical judgment derived from the surgical analysis layer, which is a lower layer.

In an embodiment, in selectively using at least one of the surgical modules of the surgical analysis layer, each solution module (or system) of the surgical solution layer may use a solution level relation. The solution level relationship may mean that a surgical module that needs to be used in a corresponding solution module is determined and designated. For example, the medical staff may use a complications determination solution module to determine a patient's complications. In this case, the complications determination solution module can grasp that a bleeding loss evaluation module and an internal damage detection module are required from a lower level based on the solution level relationship. In addition, the necessary information may be received from the corresponding module of the lower layer to determine the patient's complications and provide the determination result to the medical staff.

As described above, the medical solution model according to the implementation of the present invention is configured to operate for each module from the lowest layer to the highest layer according to the medical problem. Therefore, even if a new medical problem occurs, it is possible to efficiently provide a solution by configuring a new module for each layer or by configuring a new layer.

In addition, the medical solution model according to the practice of the present invention can be applied to various medical operations, and can be effectively used for minimally invasive surgery using a laparoscopic or endoscope, especially a surgical robot.

Referring to Figure 2, a computer-aided surgical analysis method according to an embodiment of the present invention, the computer obtaining a surgical image (S200); A computer inputting the surgical image into one or more surgical element recognition models (S400); A computer acquiring a combination of surgical recognition results calculated in each surgical element recognition model (S600; obtaining a surgical recognition result combination); And obtaining a one or more analysis results by inputting the combination of the surgical recognition results into one or more surgical analysis models (S800; obtaining a surgical analysis result). Hereinafter, detailed description of each step will be described.

The computer acquires the surgical image (S200). The computer may acquire a surgical image in real time while surgery is being performed by a medical staff. In addition, the computer can acquire the entire surgical image stored after the medical staff completes the operation.

For example, when a computer uses a surgical image acquired in real time, the computer utilizes the surgical image to provide necessary information during surgery of a medical staff. In addition, for example, when the computer uses the entire image after the surgery is completed, the computer uses the surgical image to perform a post-mortem analysis of the surgery.

The computer inputs the surgical image into one or more surgical element recognition models (S400). That is, the computer inputs the surgical images into one or more surgical element recognition models in order to recognize the surgical elements included in the surgical images.

The one or more surgical element recognition models are included in the surgical element recognition layer forming the lowest layer in the surgical analysis system. In addition, when the surgical element recognition layer includes a plurality of surgical element recognition models, each surgical element recognition model may be constructed in parallel within the surgical element recognition layer and have a connection relationship with each other. That is, it is possible to receive and use surgical elements (ie, recognition results) calculated from the B surgical element recognition model formed in parallel in the process of performing the A surgical element recognition model. Detailed description of each surgical element recognition model described above is omitted.

The computer acquires a combination of surgical recognition results calculated from each surgical element recognition model (S600; obtaining a surgical recognition result combination). That is, the computer generates data combining each surgical element recognition result calculated from each surgical element recognition model.

The computer inputs the combination of the surgical recognition results into one or more surgical analysis models to obtain one or more analysis results (S800; obtaining a surgical analysis result). The one or more surgical analysis models are included in the surgical analysis layer on the surgical element recognition layer, and are selected according to a user's request. That is, each surgical analysis model in the surgical analysis layer is based on data necessary for analysis, and a connection relationship is established with one or more surgical element recognition models in the surgical element recognition layer, so that a combination of surgical element recognition results required for analysis It can be input to each surgical analysis model. Detailed description of each surgical analysis model described above is omitted.

In another embodiment, as shown in FIG. 3, the computer inputs one or more analysis results into a specific surgical solution model to provide a surgical output arranged based on the analysis results (S1000). . The surgical solution model is included in the surgical solution providing layer, and the surgical solution providing layer is created in the surgical analysis system as an upper layer of the surgical analysis layer. Also, as each surgical solution model is connected to one or more surgical analysis models, one or more surgical analysis results may be input. Detailed description of each surgical solution model described above is omitted.

The surgical analysis method by the computer according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer that is hardware, and stored in a medium.

The above-described program is C, C++, JAVA, machine language, etc., in which a processor (CPU) of the computer can be read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in the computer language of the. Such code may include functional code related to a function defining functions necessary to execute the above methods, and control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include a memory reference-related code as to which location (address address) of the computer's internal or external memory should be referred to additional information or media necessary for the computer's processor to perform the functions. have. In addition, when the processor of the computer needs to communicate with any other computer or server in the remote to execute the functions, the code can be used to communicate with any other computer or server in the remote using the communication module of the computer. It may further include a communication-related code for whether to communicate, what information or media to transmit and receive during communication, and the like.

The storage medium refers to a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers that the computer can access or various recording media on the user's computer. In addition, the medium may be distributed over a computer system connected through a network, and code readable by a computer in a distributed manner may be stored.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains may be implemented in other specific forms without changing the technical spirit or essential features of the present invention. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive.

Claims

A computer acquiring a surgical image;

A computer inputting the surgical image into one or more surgical element recognition models;

A computer acquiring a combination of surgical recognition results calculated from each surgical element recognition model, wherein one or more surgical element recognition models are included in a surgical element recognition layer at a lowest level in the surgical analysis system; And

The computer inputs the combination of the surgical recognition results into one or more surgical analysis models to obtain one or more analysis results, wherein the one or more surgical analysis models are included in the surgical analysis layer above the surgical element recognition layer. It is selected according to, surgical analysis result acquisition step; including, surgical analysis method using a computer.
According to claim 1,

Each surgical analysis model in the surgical analysis layer,

Based on the data required for analysis, characterized in that the connection relationship is established with one or more surgical element recognition models in the surgical element recognition layer, a computerized surgical analysis method.
According to claim 1,

The surgical element recognition model,

An organ recognition model that recognizes an organ in the surgical image;

A surgical tool recognition model for recognizing the movement of the surgical tool and the surgical tool in the surgical image; And

Recognizing an event occurring within the surgical image, the event is a non-ideal situation during surgery that includes bleeding, an event recognition model; including, surgical analysis method by a computer.
According to claim 1,

The surgical analysis layer,

A blood loss recognition model for calculating a degree of blood loss based on a combination of surgical recognition results including an operation organ type, a surgical operation, and an event occurring during the operation recognized by the surgical element recognition layer; And

Includes; a long-term damage detection model for calculating the degree of long-term damage based on a combination of surgical recognition results including the operation step and the operation time recognized by the surgical element recognition layer;

The blood loss recognition model and the organ damage detection model

Computer-aided surgical analysis method, which is used to calculate the analysis results during or after surgery.
According to claim 4,

When analyzing the results of the surgery after the surgery is completed,

The surgical analysis layer,

Based on the surgical tool recognized by the surgical element recognition layer, the organ in which the operation is performed with the surgical tool, the detailed surgical step during the entire operation performed with the surgical tool, and the event generated in the detailed surgical step, the use of the wrong surgical tool A surgical analysis method using a computer, further comprising; a detection model for misusing a surgical tool.
According to claim 1,

The surgical analysis layer,

Further comprising; an optimal surgical plan calculation model for calculating an optimal surgical plan based on a combination of surgical recognition results obtained from the surgical element recognition layer;

The optimal surgical plan calculation model, characterized in that by providing the optimal surgical plan to be performed later by analyzing the surgical image data acquired in real time, computer-aided surgical analysis method.
According to claim 1,

The surgical analysis layer,

Further comprising; an optimal surgical plan calculation model for calculating an optimal surgical plan based on a combination of surgical recognition results obtained from the surgical element recognition layer;

The optimal surgical plan calculation model is based on a combination of surgical recognition results for virtual surgery performed on a patient's virtual body model before surgery, and provides a surgical plan for a real patient by calculating and providing a surgical plan for a computer. Method of analysis.
According to claim 1,

The computer inputs one or more analysis results into a specific surgical solution model, providing a surgical output arranged based on the analysis results; further comprising, surgical analysis method by a computer.
The method of claim 8,

The surgical solution model,

A surgical evaluation model for calculating an evaluation result for surgery based on one or more analysis results obtained from the surgical analysis layer;

A chart generation model that generates a chart for surgery based on one or more analysis results obtained from the surgical analysis layer; And

A computerized surgical analysis method further comprising; a surgical complexity calculation model for calculating the complexity or difficulty of surgery based on one or more analysis results obtained from the surgical analysis layer.
The method of claim 8,

Includes a surgical Q&A model that calculates an answer to a question about surgery by learning one or more analysis results obtained from the surgical analysis layer.

The step of providing the surgical output,

A computer-aided surgical analysis method in which an answer is calculated and provided according to input of a question about a specific operation of a medical staff.
A computer-aided surgical analysis program stored in a medium to execute the method of any one of claims 1 to 10 in combination with a hardware computer.
Surgical element recognition layer that includes one or more surgical element recognition models for calculating the surgical recognition result as the surgical image is input, wherein one or more surgical element recognition models are included in the surgical element recognition layer at the lowest level in the surgical analysis system. ; And

A surgical analysis model including one or more surgical analysis models for obtaining an analysis result based on a combination of surgical recognition results, which is a combination of results provided by the one or more surgical element recognition models, wherein the surgical analysis layer is formed as an upper layer of the surgical element recognition layer; Including, surgical image analysis device.
The method of claim 12,

A surgical image analysis device further comprising; a surgical solution providing layer, including one or more surgical solution models that provide a surgical output arranged based on one or more analysis results obtained from the surgical analysis layer.