KR102298412B1 - Surgical image data learning system - Google Patents

Abstract

A server that trains one or more models that provide surgical guide information and stores at least one model learned in the server, recognizes a surgical image obtained during surgery using the stored model, and responds to the recognized surgical image A client for providing surgical guide information, wherein the client transmits first learning data obtained from a surgical image obtained during the operation to the server, and receives information for updating the stored model from the server, , for updating the stored model based on the received information, a surgical image data learning system is disclosed.

Description

Surgical image data learning system {SURGICAL IMAGE DATA LEARNING SYSTEM}

The present invention relates to a surgical image data learning system.

In the course of surgery, the development of technologies capable of providing information for assisting a surgeon's operation is required. In order to provide information to assist the operation, it must be able to recognize the operation.

Therefore, it is required to develop a technology that allows a computer to recognize a surgical action from a surgical image.

In addition, recently, deep learning has been widely used in the analysis of medical images. Deep learning is defined as a set of machine learning algorithms that attempt high-level abstractions (summarizing key contents or functions in large amounts of data or complex data) through a combination of several nonlinear transformation methods. Deep learning can be viewed as a field of machine learning that teaches computers to think in a broad framework.

The problem to be solved by the present invention is to provide a surgical image data learning system.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Surgical image data learning system according to an aspect of the present invention for solving the above-described problems, a server for learning one or more models providing surgical guide information, and at least one model learned from the server, and storing the stored Recognizing a surgical image obtained during surgery using a model, and comprising a client providing surgical guide information corresponding to the recognized surgical image, wherein the client is first learning data obtained from the surgical image acquired during the operation to the server, receive information for updating the stored model from the server, and update the stored model based on the received information.

In addition, the one or more models include a first model for recognizing a surgical image by segmentation and a second model for calculating an optimized surgical method, and the first learning data includes the first model and the second model. At least one model used to train at least one of the models and stored in the client may include the first model.

In addition, the client segments the surgical image acquired during the operation into one or more surgical steps, recognizes the divided one or more surgical steps, and transmits the recognition result to the server as the first learning data. can

In addition, the client may assign a preset code to each of the divided one or more surgical steps, and transmit the preset code assigned to each of the divided one or more surgical steps to the server.

In addition, the client compares the model stored in the client with the model stored in the server, and when the model stored in the server is a more updated model than the model stored in the client, updating the model stored in the client from the server may receive information for, and update a model stored in the client based on the received information.

Also, the server may acquire one or more surgical images and information for dividing the one or more surgical images, and generate second learning data by using the acquired information.

In addition, the server receives a learning condition, extracts a dataset corresponding to the input learning condition from the second learning data, and performs learning on the at least one model using the extracted dataset can do.

In addition, the server generates a test data set corresponding to the input learning condition, performs a learning test on the at least one model using the generated test data set, and performs the learning test result feedback can be provided.

In addition, the server receives one or more new second training data, generates a dataset including at least one new second training data, and uses the generated dataset for the at least one model. learning can be performed.

In addition, the server acquires a test dataset, performs a learning test on the at least one model learned using the generated dataset using the test dataset, and performs the learning test result feedback can be provided.

In addition, the server generates the dataset including the new second training data and the existing second training data selected according to a predetermined criterion, and uses the generated dataset for the at least one model. learning can be performed.

Surgical image data learning method according to an aspect of the present invention for solving the above-described problems, the server learning one or more models that provide surgical guide information, information on the learned at least one model to the client transmitting, receiving, by the client, learning data obtained from a surgical image obtained during surgery, from the client, updating the learned one or more models, and information on the at least one model updated to the client including sending

A server according to an aspect of the present invention for solving the above problems includes a memory for storing one or more instructions and a processor for executing the one or more instructions stored in the memory, wherein the processor executes the one or more instructions By doing so, the steps of: learning one or more models that provide surgical guide information, transmitting information about the learned at least one model to the client, and transmitting the learning data obtained from the surgical image obtained by the client during surgery to the client Receiving from, updating the learned one or more models, and transmitting information on the at least one updated model to the client are performed.

In accordance with an aspect of the present invention for solving the above problems, a computer program stored in a computer-readable recording medium is provided so as to be combined with a computer as hardware to perform the surgical image data learning method according to the disclosed embodiment.

Other specific details of the invention are included in the detailed description and drawings.

According to the disclosed embodiment, by providing a server- and client-based surgical image data learning system, it is possible to quickly process data and provide surgical guide information during surgery, and when surgery is not performed, learning data is transmitted through communication with the server. It has the effect of sending and updating the model stored in the client.

In addition, according to the disclosed embodiment, a scheduler for automatically establishing a learning plan and performing learning without human instruction is provided, thereby facilitating learning and updating of the model.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a schematic diagram of a system capable of performing robotic surgery according to an embodiment disclosed herein.
2 is a diagram illustrating a surgical image data learning system according to an embodiment.
3 is a flowchart schematically illustrating the operation of the surgical image data learning system according to an embodiment.
4 is a flowchart illustrating the operation of the surgical image data learning system according to an embodiment.
5 is a configuration diagram conceptually illustrating a method of scheduling learning according to an embodiment.
6 is a configuration diagram of a server according to an embodiment.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, by way of example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to Figure 1, a schematic diagram of a system capable of performing robotic surgery according to the disclosed embodiment is shown.

Referring to FIG. 1 , the robotic surgery system includes a medical imaging device 10 , a server 20 , and a control unit 30 , a display 32 , and a surgical robot 34 provided in an operating room. According to an embodiment, the medical imaging device 10 may be omitted from the robotic surgery system according to the disclosed embodiment.

In one embodiment, the surgical robot 34 includes a photographing device 36 and a surgical tool 38 .

In one embodiment, the robotic surgery is performed by the user using the control unit 30 to control the surgical robot (34). In one embodiment, the robotic surgery may be performed automatically by the control unit 30 without the user's control.

The server 20 is a computing device including at least one processor and a communication unit.

The control unit 30 includes a computing device including at least one processor and a communication unit. In one embodiment, the controller 30 includes hardware and software interfaces for controlling the surgical robot 34 .

The photographing device 36 includes at least one image sensor. That is, the imaging device 36 includes at least one camera device and is used to photograph a body part of the object, that is, a surgical part. In an embodiment, the imaging device 36 includes at least one camera coupled to a surgical arm of the surgical robot 34 .

In an embodiment, the image captured by the photographing device 36 is displayed on the display 340 .

In one embodiment, the surgical robot 34 includes one or more surgical tools 38 capable of performing cutting, clipping, fixing, grabbing operations of the surgical site, and the like. The surgical tool 38 is used in combination with the surgical arm of the surgical robot 34 .

The control unit 30 receives information necessary for surgery from the server 20 or generates information necessary for surgery and provides it to the user. For example, the controller 30 displays generated or received information necessary for surgery on the display 32 .

For example, the user performs robotic surgery by controlling the movement of the surgical robot 34 by manipulating the controller 30 while viewing the display 32 .

The server 20 generates information necessary for robotic surgery by using medical image data of an object (patient) photographed in advance from the medical imaging device 10 , and provides the generated information to the controller 30 .

The controller 30 provides the information received from the server 20 to the user by displaying it on the display 32 , or controls the surgical robot 34 using the information received from the server 20 .

In one embodiment, the means that can be used in the medical imaging equipment 10 is not limited, for example, CT, X-Ray, PET, MRI, etc., various other medical image acquisition means may be used.

In the disclosed embodiment, the surgical image acquired by the imaging device 36 is transmitted to the controller 30 .

In an embodiment, the controller 30 may segment a surgical image acquired during surgery in real time.

In one embodiment, the controller 30 transmits the surgical image to the server 20 during surgery or after the surgery is completed.

The server 20 may divide and analyze the surgical image.

2 is a diagram illustrating a surgical image data learning system according to an embodiment.

Referring to FIG. 2 , the surgical image data learning system includes a client 100 and a server 200 .

In one embodiment, the client 100 and the server 200 are computing devices including at least one processor.

In an embodiment, the client 100 may be a computing device provided in an operating room (surgery site). For example, the client 100 may correspond to the controller 30 shown in FIG. 1 . As another example, the client 100 may be provided in the operating room as a computing device separate from the control unit 30 , and may be connected to the control unit 30 . In this specification, the connection includes not only a physical connection but also the concept of an electronic connection, and being in a state capable of communicating with each other may also be understood as the concept of a connection. For example, the client 100 may be connected to the controller 30 by wire or wirelessly, and may be in a state capable of mutual communication using short-range wireless communication or network communication.

In one embodiment, the client 100 determines the surgical situation in the operating room, and provides corresponding surgical guide information. The client 100 uses the learned model to determine the surgical situation and provide corresponding surgical guide information.

In an embodiment, the trained model may be a model trained using machine learning (machine learning). Machine learning may mean, but is not limited to, deep learning.

In one embodiment, the client 100 acquires a surgical image, and the server 200 performs learning using the acquired surgical image. Since machine learning is a task that requires a lot of resources, it may be difficult to perform in the client 100 . In addition, since a lot of learning data is required for machine learning, it may be difficult to secure sufficient learning data only with information collected from the client 100 .

Accordingly, the server 200 configures learning data using information collected from different sources, determines a surgical situation based on this, and trains a model capable of providing surgical guide information.

The client 100 should determine the surgical situation in real time and provide necessary surgical guide information. Accordingly, in order to save time required for communication with the server 200 and to prevent errors due to communication abnormalities, information about the model learned by the server 200 is stored in the client 100 in advance.

The client 100 may restore the model learned in the server 200 by using the information obtained from the server 200, and use this to provide surgical guide information in real time.

3 is a flowchart schematically illustrating the operation of the surgical image data learning system according to an embodiment.

The client 100 and the server 200 described with reference to FIG. 2 correspond to the client 100 and the server 200 of FIG. 3 , respectively.

In step S210, the server 200 trains a model for providing surgical guide information by using the learning data obtained from the surgical image.

In the disclosed embodiment, the model for providing the surgical guide information includes a first model for dividing and recognizing a surgical image, and a second model for calculating an optimized surgical method.

In an embodiment, the learning data may refer to the surgical image itself, or may refer to data obtained by analyzing the surgical image. For example, the learning data may include data obtained by dividing a surgical image into preset motion units. In addition, the learning data may include data labeled with a surgical image. The type of training data is not limited thereto.

In step S220 , the server 200 transmits information about the learned model to the client 100 . For example, the information on the learned model may include information on the weight of the learned neural network. In an embodiment, the information provided to the client 100 includes information about the first model, which recognizes a surgical image by dividing it.

In one embodiment, the server 200 transmits information about the learned model to the client 100 when the operation using the client 100 is not performed, that is, when the client 100 is not assisting the operation. . That is, while surgery is being performed, surgery guide information is provided based on information previously stored in the client 100, and when surgery is not performed (for example, before surgery is performed), it is applied to the model learned as a preparation stage for surgery. Information about the information may be transmitted from the server 200 to the client 100 in advance.

Referring to FIG. 4 , the server 200 acquires a medical image including a patient's surgical site from the medical image recording device 10 . The server 200 performs 3D modeling of the surgical site of the patient based on the acquired medical image (step S222).

The 3D modeling information on the patient's surgical site is used to determine the position and shape of each organ according to the angle of the camera during surgery, and to provide information on body parts that are not visible with the camera.

The server 200 transmits the 3D modeling information generated in step S222 to the client 100 (step S224). The client 100 stores the 3D modeling information transmitted from the server 200 .

The client 100 acquires and stores the learned model based on the information transmitted from the server 200 . In step S110, the client 100 provides surgical guide information corresponding to the surgical image acquired during the operation using the model stored in the client 100 during the operation. For example, the client 100 may determine the surgical situation by segmenting and recognizing the surgical image using the stored model, and may provide surgical guide information based on this.

In one embodiment, the server 200 transmits 3D modeling information including the surgical site of the patient to the client 100 in advance before surgery for a specific patient starts. The client 100 stores the 3D modeling information transmitted from the server 200, and during surgery, the 3D modeling information stored in the client 100 and information about the learned model correspond to the surgical image obtained during surgery. Provides surgical guide information.

For example, the surgical guide information may include an optimal surgical method corresponding to the current surgical situation. In addition, the surgical guide information may include information on a body part of the patient that is not confirmed by the camera (eg, hidden by other organs). In addition, the surgical guide information may include information on the location of a body part that should not be damaged in the surgical process, such as blood vessels or nerves. In addition, the surgical guide information may include information for recognizing an event or surgical error situation occurring during surgery, and providing information about the recognized situation or a notification according to the recognized situation.

In step S120 , the client 100 transmits the learning data obtained from the surgical image obtained during the operation to the server after the operation is completed. In an embodiment, the training data includes a surgical image. In an embodiment, the learning data includes information on recognizing the divided surgical image by dividing the surgical image obtained during surgery into one or more surgical steps, and recognizing the divided one or more surgical steps.

In an embodiment, the client 100 segments a surgical image acquired during surgery into one or more surgical steps. The client 100 may divide the surgical image into one or more preset surgical steps using the learned model. For example, the preset surgical stage divides each operation included in the operation based on the position and change of the organ and the position and movement of the surgical tool according to a predetermined criterion, and a standardized name for each divided operation may have been granted.

In an embodiment, a preset code may be assigned to each divided operation. The client 100 gives a preset code to each of the divided one or more surgical steps.

In an embodiment, the client 100 may transmit a preset code assigned to each of the one or more divided surgical steps to the server 100 instead of the surgical image. The server 100 may acquire information about each surgical step and the order of the overall surgical operation based on the received code. In this case, compared to transmitting the surgical image to the server 200, the amount of data transmission is greatly reduced. Therefore, if necessary, it is possible to transmit the learning data to the server 200 in real time even during surgery, that is, while the client 100 provides surgical guide information (step S110) (step S120).

In step S230 , the server 200 updates the learned model based on the information received from the client 100 . For example, the server 200 may re-train or fine-tune the model using information received from the client 100 . In an embodiment, the server 200 trains a second model for calculating an optimized surgical method based on the information received from the client 100 . In an embodiment, the server 200 may train a first model for recognizing a surgical image by dividing it based on the information received from the client 100 . For example, the server 200 may train a model for recognizing the movement of a surgical tool based on information received from the client 100 .

In addition, the server 200 may train the first model for recognizing by dividing the surgical image based on one or more surgical images obtained from the outside and information on dividing the one or more surgical images.

In step S240 , the server 200 transmits the model updated through step S230 to the client 100 . When the operation is not performed, the client 100 receives information about the updated model from the server 200, that is, information for updating the model stored in the client 100, and based on the received information, the client 100 Update the trained model stored in .

Referring to FIG. 4 , the client 100 checks the update status of the model to the server 200 (step S130 ).

For example, the client 100 compares the learned model stored in the client 100 with the learned model stored in the server 200 , and the learned model stored in the server 200 is stored in the client 100 . When the model is updated rather than the learned model, the client 100 receives information for updating the learned model from the server 200, and updates the learned model stored in the client 100 based on the received information. do.

5 is a configuration diagram conceptually illustrating a method of scheduling learning according to an embodiment.

In one embodiment, the scheduler 300 , the database 320 , the learning unit 330 , the test unit 340 , and the model 350 are modules implemented in hardware or software in the server 200 , one or more instructions, respectively. It conceptually illustrates a set of , data stored in a memory, or an operation performed by the server 200 .

In an embodiment, the learning information input unit 310 refers to the server 200 or a computing device capable of communicating directly or indirectly with the server 200 .

For example, the learning information input unit 310 may input information for dividing the surgical image obtained by the server 200 . For example, the learning information input unit 310 may be a computing device in which a doctor views a surgical image and directly inputs information for dividing the surgical image into preset operation units. The source of the surgical video used for learning is not limited. The server 200 may provide various user interfaces through which the learning information input unit 310 can easily input information for dividing a surgical image using an application or a web page.

Information for dividing the surgical image and the surgical image is stored in the database (320).

The scheduler 300 serves to control the learning process of the server 200 . In the past, it was common for a developer to directly set a training dataset and train a model by issuing a training command.

According to the disclosed embodiment, the scheduler 300 automatically generates a training dataset, performs learning, and performs a role of automatically learning the model by testing the learning result.

In an embodiment, the scheduler 300 may set a learning plan and determine whether to plan the next learning according to the progress level of the learning.

For example, the scheduler 300 performs learning based on predetermined learning data, performs a test on the learning result, and when learning of more than a predetermined threshold (eg, 80% or more accuracy) is completed, learning You can finish and plan the next lesson. For example, when the learning of the predetermined learning data is finished, the scheduler 300 may plan and perform the next learning based on the newly acquired learning data.

When predetermined training data is stored in the database 320 , the scheduler 300 extracts a training dataset from the training data stored in the database 320 . The scheduler 300 inputs the extracted learning dataset to the learning unit 330 to perform learning.

In an embodiment, the scheduler 300 receives a predetermined learning condition for learning, extracts a training dataset corresponding to the input learning condition from the training data stored in the database 320, and learns the extracted dataset. It is input to the unit 330 to perform learning.

For example, the learning condition may include, but is not limited to, the learning target gender, age group, and type of disease.

In an embodiment, the scheduler 300 tests the learned model using the test unit 340 when learning is completed. For example, the scheduler 300 generates a predetermined test data set, and performs a learning test on the learned model using the generated test data set. The scheduler 300 provides feedback according to the learning test execution result. In addition, the scheduler 300 determines whether to end the learning and whether to plan the next learning as described above according to the learning result.

For example, if the learning test result accuracy is greater than or equal to a predetermined reference value, it is assumed that the test has passed, and the generation of the trained model 350 may be terminated. It is also possible to plan and execute the next learning on the learned model 350 .

In addition, if the accuracy of the learning test result is less than or equal to a predetermined reference value, it is assumed that the test has not been passed, so that learning is performed again through the learning unit 330 or the learning condition may be reset.

As another example, when additional learning is performed on an already trained model using new data, the accuracy of the previously trained model may be compared with the accuracy of the model on which additional learning has been performed using the new data. When the accuracy of the model on which additional learning is performed is higher, the scheduler 300 may update the trained model.

In an embodiment, when learning is performed based on a predetermined learning condition, the test unit 340 generates a test dataset corresponding to the predetermined learning condition, and uses the generated test dataset Test the model.

In one embodiment, when new learning data is received from the server 200 , the scheduler 300 generates a training dataset including the new training data, and inputs the generated dataset to the learning unit 330 to perform learning. carry out

In an embodiment, the scheduler 300 may perform learning whenever new learning data is received, and may perform learning by generating a learning dataset whenever the learning data is accumulated by a predetermined amount or more. it is not

In an embodiment, the test unit 340 may generate a test dataset for testing a learned model using new learning data, and may test the learned model. If the accuracy of the model trained using the new training data is higher than the accuracy of the existing model, the scheduler 300 may update the learned model.

In an embodiment, the scheduler 300 may perform learning by generating a learning dataset including new learning data, but may perform learning based on a rule for preventing the previously learned result from being forgotten.

For example, when the learning unit 330 performs learning using a deep neural network, when learning is performed using new learning data, the decision boundary of the deep neural network is higher than when learning is performed using existing learning data. make sure it doesn't change.

In addition, the features extracted from the existing learning data using the model learned using the new learning data should be close to the features extracted from the existing learning data using the existing learned model.

In one embodiment, the scheduler 300 generates a training dataset including new training data and existing training data selected according to a predetermined criterion, and uses the generated training dataset to train the learning unit 330 . learn

When selecting the existing training data, the scheduler 300 selects the existing training data that can prevent the model learned by the new training dataset from forgetting the previously learned information.

In one embodiment, the client 100 compares the learned model 350 with the model stored in the client 100 , and when the learned model 350 is an updated model than the model stored in the client 100 , learning It receives information on the model 350 and updates the model stored in the client 100 .

6 is a block diagram of the server 200 according to an embodiment.

The processor 202 may include one or more cores (not shown) and a graphic processing unit (not shown) and/or a connection path (eg, a bus, etc.) for transmitting and receiving signals to and from other components. .

The processor 202 according to an embodiment executes one or more instructions stored in the memory 204, thereby performing the surgical image data learning method described in relation to FIGS. 1 to 5 .

For example, the processor 202 trains one or more models that provide surgical guide information by executing one or more instructions stored in a memory, transmits information about the learned at least one model to a client, and the client Receives learning data obtained from a surgical image obtained during surgery from the client, updates the learned one or more models, and transmits information on the at least one updated model to the client.

On the other hand, the processor 202 is a RAM (Random Access Memory, not shown) and ROM (Read-Only Memory: ROM) for temporarily and/or permanently storing signals (or data) processed inside the processor 202 . , not shown) may be further included. In addition, the processor 202 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processing unit, a RAM, and a ROM.

The memory 204 may store programs (one or more instructions) for processing and controlling the processor 202 . Programs stored in the memory 204 may be divided into a plurality of modules according to functions.

The surgical image segmentation method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to functions defining functions necessary for executing the methods, etc. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer to be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected by a network, and a computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100 : client
200 : server

Claims (14)

a server for training at least one model that provides surgical guide information; and
A client that stores the model received from the server, recognizes a surgical image obtained during surgery using the stored model, and provides surgical guide information corresponding to the recognized surgical image; includes,
The model stored in the client is updated based on the update information received from the server,
The server is
extracting a dataset corresponding to a preset learning condition from the training data related to the surgical image, and performing learning on the model using the extracted dataset,
generating a test data set corresponding to the preset learning condition, performing a learning test on the learned model using the generated test data set, and providing feedback according to a result of the learning test , Surgical image data learning system. According to claim 1,
The at least one model includes a first model for recognizing a surgical image by segmentation and a second model for calculating an optimized surgical method,
The training data is used to train at least one of the first model and the second model,
The client, the first model is stored, surgical image data learning system. According to claim 1,
the client,
comparing the model stored in the client with the model stored in the server,
When the model stored in the server is an updated model than the model stored in the client, receiving information for updating the model stored in the client from the server,
Based on the received information to update the model stored in the client, surgical image data learning system. According to claim 1,
The server is
Obtaining one or more surgical images and information for segmenting the one or more surgical images,
Using the obtained information to generate the learning data, surgical image data learning system. According to claim 1,
The learning data is obtained from the surgical image by the client, surgical image data learning system. 6. The method of claim 5,
the client,
Recognizing the surgical image by segmentation into one or more surgical steps,
Transmitting the recognized result to the server as the learning data, surgical image data learning system. 7. The method of claim 6,
the client,
Giving a preset code to each of the divided one or more surgical steps,
As the learning data, for transmitting the given code to the server, surgical image data learning system. training, by the server, at least one model for providing surgical guide information;
transmitting, by the server, information about the learned model to the client so that the learned model is stored;
updating the learned model by the server; and
Including; transmitting update information on the updated model to the client;
The model stored in the client is updated based on the update information received from the server,
The update step is
extracting, by the server, a dataset corresponding to a preset learning condition from training data related to a surgical image obtained during surgery by the client using the stored model;
performing, by the server, learning on the model using the extracted dataset;
generating, by the server, a test data set corresponding to the preset learning condition;
performing, by the server, a learning test on the learned model using the generated test dataset; and
Providing, by the server, feedback according to the result of the learning test; Containing, surgical image data learning method. A computer program stored in a computer-readable recording medium in combination with a computer, which is hardware, to perform the method of claim 8. a memory storing one or more instructions; and
a processor that executes the one or more instructions stored in the memory;
The processor, by executing the one or more instructions,
training at least one model for providing surgical guide information;
transmitting information about the learned model to a client so that the learned model is stored;
updating the learned model; and
transmitting update information on the updated model to the client; and
The model stored in the client is updated based on the update information received from the server,
The update step is
extracting, by the client, a dataset corresponding to a preset learning condition from training data related to a surgical image acquired during surgery by using the stored model;
performing learning on the model using the extracted dataset;
generating a test dataset corresponding to the preset learning condition;
performing a learning test on the learned model using the generated test dataset; and
A server comprising a; providing feedback according to a result of the learning test. delete delete delete delete

Images