KR20200056855A - Method, apparatus and program for generating a pneumoperitoneum model - Google Patents

Abstract

Disclosed is a method for generating a pneumoperitoneum model. The method comprises the steps of: obtaining medical image data of a patient by a computer; obtaining scan data, which scans a body of the patient in a pneumoperitoneum state; and generating a pneumoperitoneum model by matching the medical image data and the scan data.

Description

Relief model creation method, device and program {METHOD, APPARATUS AND PROGRAM FOR GENERATING A PNEUMOPERITONEUM MODEL}

The present invention relates to a relief model generating method, apparatus and program.

Recently, devices and software are needed to enable medical staff to perform training in a realistic situation. In general, a simulation device for a medical staff is a method in which training is performed after being produced similar to a patient's situation. These simulation devices do not provide various situations that occur to the patient, and there is a problem in that reality is reduced during simulation. In addition, in the case of surgical operation, there was a problem that the medical staff cannot simulate under the same conditions as the actual operation.

In addition, in the surgical process, there is a need to develop technologies capable of providing information for assisting a surgeon's surgery. In order to provide information to assist with surgery, it must be possible to recognize the operation. In the past, in order to design a scenario for optimizing the surgical process, reference was made to medical images taken in advance or consulted by a highly skilled doctor. There were difficulties in getting advice. Therefore, medical imaging or the advice of an experienced doctor was often difficult to use as an auxiliary purpose for optimizing the surgical process for the patient.

In the case of performing a surgical simulation using virtual reality, training must be performed under the same conditions as in an actual surgery in order to perform a surgical simulation as a rehearsal. In particular, in the case of performing minimally invasive surgery (for example, robot surgery or laparoscopic surgery), if the shooting direction of the camera is different during the actual surgery and during the surgical simulation, the medical staff checks the image during the simulation and the actual surgery. Depending on the difference in the visible image, it may not be possible to obtain the effect of performing the training in the same way as the actual operation. That is, since the intraperitoneal structure may be different depending on the position of the camera for confirming the intraperitoneal structure, it is necessary to implement such that the camera enters the body in the same manner as in the actual operation during the operation simulation.

Therefore, in the present invention, the camera inflates the patient's stomach to facilitate the operation by injecting gas into the patient's body in the same manner as in the actual operation (pneumoperitoneum: in order to provide the same image as the actual operation during the operation simulation) The object of the present invention is to provide a relief model generating method, apparatus, and program that implements a virtual body model to which the state is applied and generates the same body surface as in actual surgery.

In addition, the present invention creates a virtual body model that generates a virtual body model capable of providing the same screen as that of a real operation when the camera enters the inside of the body through the same location on the body surface implemented in the same way as in the actual operation. It is an object to provide a method, an apparatus, and a program.

In addition, the present invention provides a method, apparatus, and program for constructing a learning dataset for learning a relief model that implements a virtual body model to which a relief state is applied in the same manner as in real surgery based on medical image data of a new patient. It is aimed at.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Method for generating a relief model according to an aspect of the present invention for solving the above-described problems, the computer acquiring the medical image data of the patient, acquiring the scan data of the patient's body in the relief (pneumoperitoneum) state And generating a relief model by matching the medical image data to the scan data.

In addition, the medical image data may be characterized in that the 3D modeling data of the patient's body generated based on the medical image of the patient previously taken.

In addition, the medical image of the patient may be characterized in that the medical image taken by tilting the patient's body at a predetermined angle.

The generating of the relief model may include selecting one or more landmarks from the medical image data and the scan data, and selecting one or more landmarks respectively selected from the medical image data from the scan data. And matching each of the one or more landmarks.

In addition, the step of generating the relief model may further include a step of warping the medical image data according to the scan data.

In addition, the step of warping may include selecting one or more landmarks from the medical image data and the scan data, respectively, and warping the medical image data according to the scan data based on the selected one or more landmarks. It can contain.

In addition, the step of generating the relief model may further include the step of non-rigid registration of the medical image data according to the scan data.

In addition, the generating of the relief model may further include acquiring information about the internal organs of the patient based on at least one indicator indicating the physical characteristics of the patient.

An apparatus for generating a relief model according to an aspect of the present invention for solving the above-described problem includes a memory that stores one or more instructions and a processor that executes the one or more instructions stored in the memory, wherein the processor includes the one or more instructions. By executing the instructions, obtaining the medical image data of the patient, obtaining the scan data of the patient's body in a undulating state, and matching the medical image data to the scan data to generate a relief model. It can be done.

In accordance with an aspect of the present invention for solving the above-described problems, a computer program stored in a recording medium readable by a computer is provided to be combined with a computer that is hardware, and to perform a relief model generating method according to the disclosed embodiment.

Other specific matters of the present invention are included in the detailed description and drawings.

Through the embodiments of the present invention, the body model generated based on the general medical image data of the patient is implemented by the medical image data as it is implemented in a relief state during minimally invasive surgery (eg, robot surgery, laparoscopic surgery). It is possible to match the position and orientation of the camera with the virtual body model in the actual patient's body.

In addition, while the body model generated based on the patient's general medical image data is implemented in a undulating state during minimally invasive surgery (for example, robotic surgery or laparoscopic surgery), learning data for implementing the relief algorithm can be obtained. That is, as the body model based on the medical image data of the existing patient is transformed into undulating body surface scanning data before surgery, learning data for learning the relief algorithm is acquired, and new patients are obtained through the learning algorithm based on the learning data. The medical image data can be implemented in the same manner as the relief state during surgery.

Therefore, the medical staff can perform the surgery while viewing the same image as the simulation at the time of actual surgery as the simulation is performed before the surgery in the same camera view direction as at the time of minimally invasive surgery. Can prepare for unexpected situations. In addition, as the virtual camera enters the virtual camera in the same position and direction in the virtual body model that has the same body surface as the patient's body that is undulated during the actual surgery, the screen that the medical staff actually sees through the camera and the virtual camera on the virtual body model Since the direction of the gaze is the same, it is possible to perform surgical assistance by transmitting accurate information about the location where the medical staff is performing minimally invasive surgery through a virtual body model.

In addition, as the 3D body model is implemented using medical image data photographed in the same posture as the actual surgical posture, the organ disposition of the virtual body model can be implemented in the same manner as the organ disposition inside the body during actual surgery. Through this, the image information provided through the camera entering the same direction from the same location can be implemented identically.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart briefly illustrating a method of generating a relief model according to an embodiment.
2 is an exemplary view using an auxiliary device for obtaining a medical image reflecting an actual surgical posture according to an embodiment of the present invention.
FIG. 3 is a view showing medical image data of a patient and scan data of a patient's body in an upset state.
4 is a flowchart illustrating an example in which a computer performs matching.
5 is a diagram illustrating an example of performing point-based matching.
6 is a diagram illustrating an example of performing warping.
7 is a view showing an example of non-rigid matching.
8 is a configuration diagram of an apparatus according to an embodiment.
9 is a schematic diagram of a system capable of performing robotic surgery according to the disclosed embodiment.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and are common in the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, “comprises” and / or “comprising” does not exclude the presence or addition of one or more other components other than the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and “and / or” includes each and every combination of one or more of the components mentioned. Although "first", "second", etc. are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless explicitly defined.

The term "part" or "module" as used in the specification refers to a hardware component such as software, FPGA or ASIC, and "part" or "module" performs certain roles. However, "part" or "module" is not meant to be limited to software or hardware. The "unit" or "module" may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, “part” or “module” means components, processes, functions, attributes, such as software components, object-oriented software components, class components and task components. Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays and variables. The functionality provided within components and "parts" or "modules" can be combined into a smaller number of components and "parts" or "modules" or into additional components and "parts" or "modules" Can be further separated.

In the present specification, 'computer' includes all of various devices capable of performing arithmetic processing and providing a result to a user. For example, the computer is a desktop PC, a notebook (Note Book), as well as a smart phone (Smart phone), tablet PC, cellular phone (Cellular phone), PC phone (PCS phone; Personal Communication Service phone), synchronous / asynchronous The mobile terminal of the International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), and a Personal Digital Assistant (PDA) may also be applicable. In addition, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. Also, the computer may be a server that receives a request from a client and performs information processing.

In this specification, "image" may mean multi-dimensional data composed of discrete image elements (eg, pixels in a 2D image and voxels in a 3D image). For example, the image may include a medical image of an object acquired by the CT imaging device.

As used herein, "object" may be a person or animal, or a part or all of a person or animal. For example, the subject may include at least one of organs such as the liver, heart, uterus, brain, breast, and abdomen, and blood vessels.

In the present specification, “user” may be a medical expert, a doctor, a nurse, a clinical pathologist, a medical imaging expert, or the like, but may be a technician repairing a medical device, but is not limited thereto.

In the present specification, "medical image data" is a medical image photographed by a medical imaging device, and includes all medical images capable of realizing a body of an object as a 3D model. The "medical image data" may include computed tomography (CT) images, magnetic resonance imaging (MRI), positron tomography (PET) images, and the like.

In this specification, "virtual body model" refers to a model generated in conformity with an actual patient's body based on medical image data. The "virtual body model" may be generated by modeling medical image data in three dimensions as it is, or it may be corrected as in actual surgery after modeling.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, a method of generating a relief model, which can be performed by a computer (for example, a control unit 30, a server 100, or a computing device provided in an operating room) will be described. For example, the computing device provided in the control unit 30, the server 100, or the operating room is obtained from a scanning device, based on a user's body appearance, medical image data of a patient obtained from the medical imaging device 10 A method of matching (eg, a 3D model generated from medical images) is performed.

Specifically, in the case of performing minimally invasive surgery (for example, laparoscopic surgery or robotic surgery), other locations are identified while checking some ranges inside the body through a camera entering the body through a trocar penetrating the body. Surgery is performed with surgical instruments that have entered through one or more trocars inserted in. In order to provide a space for the surgical tool to move, a gas (for example, carbon dioxide) is injected into the body (for example, a space between the abdominal walls when performing abdominal surgery) during minimally invasive surgery.

The medical staff wants to prepare for the variables that may occur during the actual surgery by simulating the surgery in advance before the actual surgery. In order to practice actual surgery, simulation must be performed in the same environment as actual surgery. Since minimally invasive surgery is performed by checking only the camera (ie, endoscope) inserted inside the body, if the actual surgery is performed after practicing with images displayed in a completely different position or direction during simulation, the actual surgery to the medical staff is performed. Because the video provided in the poem is completely different, it is impossible to obtain a practice effect at all. In particular, even if the virtual body model is modeled in the same manner as the patient's body state during physical surgery, when the camera enters in a different position or direction, it is impossible to obtain a practice effect by performing exercises while viewing completely different images.

When the relief state is applied to the patient's body, as the shape of the body surface (e.g., the abdominal surface) is deformed, even if the same position as the undulated body surface is assigned to the undulated 3D body model, the angle at which the camera is inserted Will be different. Therefore, when creating a virtual body model for performing a surgical simulation, it is necessary to implement a body surface to which a relief state is applied in the same way as in an actual surgery.

1 is a flowchart briefly illustrating a method of generating a relief model according to an embodiment.

Each step illustrated in FIG. 1 is performed in time series by a computer such as a control unit 30, a server 100, or a computing device provided in an operating room, which will be described later. Hereinafter, for convenience of description, it is described that each step described in FIG. 1 and below is performed by a computer. However, at least some or all of the steps described in FIG. 1 and below may be performed by the control unit 30, the server 100, or a computing device provided in the operating room, and the subject is not limited thereto.

In step S110, the computer acquires the medical image data of the patient.

In one embodiment, the medical image data includes medical images obtained from the above-described medical image photographing equipment 10, and the type is not limited. For example, the medical image data may include CT image data of a patient's body, and may include 3D modeling image data generated from CT image data.

The patient's 3D medical image obtained by the medical image capturing apparatus (for example, CT system) is used to generate a 3D modeling image by a medical image capturing apparatus (for example, CT system) or an external computer. For example, a CT system or an external computer may generate a 3D modeled image by rendering a 3D medical image of a patient.

In one embodiment, the CT system may include a gantry, a table, an X-ray generator, and an X-ray detector. The gantry may include an X-ray generator and an X-ray detector. The patient may be placed on the table, and the table may move in a predetermined direction (eg, at least one of up, down, left, and right) during CT imaging. As the table moves, the CT system acquires a plurality of body cross-section images used for generating 3D body modeling data.

In addition, in one embodiment, the medical staff performs a predetermined pre-processing operation or requests the medical imaging apparatus to obtain a 3D medical image having a shape of an internal organ of a patient similar to that of an actual operation. In general, the shape or arrangement of the internal organs included in the 3D image of the patient obtained through the CT system while the patient is lying in a horizontal state may differ from the shape or arrangement of the internal organs of the patient during surgery. That is, in the actual operation, the patient's surgical posture may be different from the reclined posture depending on the type of surgery or the surgical site, so the patient's body organs may be subject to different gravitational effects and thus the arrangement state may be different. That is, the shape of the internal organs of the patient may be changed due to the influence of gravity depending on the angle at which the patient is laid during surgery.

In one embodiment, the apparatus for obtaining a medical image may acquire a 3D medical image of a patient by tilting a table or a gantry at a predetermined angle. For example, the CT system can acquire a 3D medical image of the patient by tilting the table 15 degrees (ie, the patient also tilts 15 degrees). In another embodiment, as shown in FIG. 2, the medical staff can actually By using an assisting device capable of implementing the same long-term arrangement as in surgery, the actual surgical site in the general gantry can be implemented in the same state as in actual surgery. That is, when surgery is to be performed by inclining 15 degrees in reverse, the medical staff may place an auxiliary device on the lower body side of the patient when taking a medical image of the patient. In one embodiment, the medical image data of the patient obtained in step S110 is the patient May mean a 3D medical image obtained from a CT image taken at a predetermined angle (for example, 15 degrees), but is not limited thereto.

In step S120, the computer acquires scan data that scans the patient's body in a pneumoperitoneum state.

In one embodiment, the relief state refers to a state in which gas is injected into a patient's body for surgery. The computer acquires scan data (eg, data on the appearance of the body obtained by scanning the skin) that scans the patient's body in a undulating state.

Referring to FIG. 3, the medical image data 200 of the patient and the scan data 300 that scan the patient's body in an upset state are illustrated.

According to an embodiment, the medical image data 200 may be obtained by tilting the patient's body to obtain an image in a state similar to the relief state. However, as shown in the scan data 300 shown in FIG. 3, the body in an undulating state is different from the medical image data 200 such as a stomach swelling due to gas.

Accordingly, in step S130, the computer matches the medical image data 200 to the scan data 300 to generate a relief body model. For example, the scan data 300 may be obtained by shooting with a 3D scanner in a direction vertically or vertically on the floor after injecting carbon dioxide into the patient's body before surgery.

In one embodiment, the computer transforms the appearance of the medical image data 200 according to the scan data 300. In addition, when the organ configuration is modified according to the relief state implementation, the computer may perform a matching such that the internal organ configuration structure included in the medical image data 200 is also modified.

In addition, as an example, before performing matching with the scan data 300, the computer may perform a pre-processing process of the medical image data 200. For example, the computer acquires medical image data modeled in three dimensions from the medical image photographing apparatus or generates three-dimensional modeling data based on two-dimensional image data of a continuous frame obtained by the medical image photographing apparatus, The process of separating the body surface from the 3D modeling data is performed. That is, the computer generates a skin model or a body surface model separated from a 3D body model in which relief is not reflected. In the matching process with the scan data 300, the computer may directly use a 3D body model as medical image data, or may use a body surface model (or skin model).

In one embodiment, when CT image data is used as medical image data, in order to generate a body surface model, the computer performs a process of separating the body surface based on the brightness value. Specifically, in the CT image, since the skin has a low brightness value, the computer calculates a threshold value of -300 HU, automatically extracts a point within the threshold range, calculates it as a seed point, and uses the region expansion method The abdominal region is divided and the inside is filled using a hole filling technique. Thereafter, the computer transforms the skin surface data using a marching cube algorithm to generate surface data for the skin surface in the divided region.

In addition, when it is desired to perform a matching process using a body surface model, the computer may additionally perform a process of deleting unnecessary parts that are not used for matching with scan data. That is, when performing abdominal undulation, the computer performs a process of removing unnecessary dorsal areas not related to undulation formation from the entire skin surface data.

Specifically, in the case of an image taken by tilting the patient at a specific angle, it is difficult to remove unnecessary parts based on the same position in the entire frame, so the point corresponding to the back part in the top frame based on the sagittal plane And a point corresponding to the back portion in the bottom frame is input from the user to generate a straight line, and an area to be removed in each frame can be calculated.

4 is a flowchart illustrating an example in which a computer performs matching.

In step S132, the computer performs point-based registration.

In one embodiment, the computer selects one or more landmarks from the medical image data 200 and the scan data 300, respectively. For example, the landmark may be selected from characteristic parts such as ribs, belly button, and pelvis.

In one embodiment, the landmark may be selected by the user or may be automatically selected by the computer. For example, the computer may automatically select the landmark through image recognition based on information on the feature of the landmark. In addition, the computer may automatically acquire the landmark from the image using the model trained based on the training data labeled with the landmark location.

5 is a diagram illustrating an example of performing point-based matching.

Referring to FIG. 5, medical image data 210 to be matched and scan data 310 to be used as a reference for registration are illustrated. In the embodiment illustrated in FIG. 5, the scan data 310 is illustrated as white dots and means a portion overlapped with the medical image data 210.

Referring to FIG. 5, one or more landmarks (purple dots) selected from the medical image data 210 and one or more landmarks (red dots) selected from the scan data 310 are illustrated.

In one embodiment, the computer matches each of the one or more landmarks selected from the medical image data 210 with each of the one or more landmarks selected from the scan data 310.

For example, the computer may perform point-based matching by moving one or more landmarks selected from the medical image data 210 in the direction of one or more landmarks selected from the scan data 310.

In one embodiment, the computer may move the internal organs included in the medical image data 210 as one or more landmarks included in the medical image data 210 are moved in the direction of one or more landmarks included in the scan data 310. The position, shape and structure can also be changed accordingly.

In an embodiment, the computer may transform, rotate, or change the medical image data 210 in the x, y, and z-axis directions as a point-based matching method.

In step S134, the computer may warp the medical image data 200 according to the scan data 300.

For example, the computer may warp the medical image data 200 according to the scan data 300 to a thin plate spline (TPS).

6 is a diagram illustrating an example of performing warping.

Referring to FIG. 6, the computer may warp medical image data 220 according to the scan data 320.

In one embodiment, the medical image data 220 shown in FIG. 6 may be obtained according to a result of performing point-based matching on the medical image data 210 shown in FIG. 5 according to the scan data 310. .

In one embodiment, the computer selects one or more landmarks from the medical image data 220. In one embodiment, the landmark selected from the medical image data 220 may include at least part or all of the landmark selected from the medical image data 210 of FIG. 5.

For example, the computer may select one or more landmarks from the ribs, belly button, and pelvis, and warp the medical image data 220 according to the scan data 320 based on the selected landmark. The computer may evaluate the degree of change in the body according to the relief, and accordingly, may match the medical image data 220 to the scan data 320 using a warping technique.

In step S136, the computer may perform non-rigid registration of the medical image data 200 according to the scan data 300. For example, a computer can perform B-Spline non-rigid matching.

The computer may evaluate the degree of change of the body according to the relief, and thus perform B-Spline non-rigid matching on the medical image data 200.

7 is a view showing an example of non-rigid matching.

Referring to FIG. 7, an example of performing non-rigid matching according to the scan data 330 with respect to the medical image data 230 is illustrated.

In one embodiment, the medical image data 230 may be obtained as a result of warping the medical image data 220 of FIG. 6.

The computer may match the medical image data 230 to the scan data 330 by performing non-rigid matching on the medical image data 230.

In one embodiment, the computer includes a patient's gender and fat distribution in addition to the above-described matching method, in order to predict a shape in which the appearance (ie, the body surface) of the body changes according to the relief state in performing matching. Information on the physical characteristics of the patient.

In one embodiment, the computer learns a model capable of acquiring physical characteristics after relief from medical image data before and after the patient's relief by using one or more indicators representing the patient's physical characteristics and medical image data before and after the patient's relief. I can do it. Depending on the embodiment, the learning data may further include body scan data.

That is, the computer uses the trained model to reflect the position, shape, and structure of the patient's internal organs after relief from the patient's pre-relief medical image data, one or more indicators representing the patient's body characteristics, and the patient's post-relief scan data. Models can be acquired.

Each learning process can also be used to advance each matching method according to the flow chart shown in FIG. 4, and can also be used to determine the internal organ condition of the patient from information on the patient's body characteristics as described above. Can be.

In addition, in another embodiment, the computer may include information on which body surfaces of a plurality of patient-specific 3D body models are transformed into scan data 300 (eg, landmarks in a 3D body model before relief are on the scan data). By accumulating vector data moving to the corresponding landmark), a statistical model or a learning model for realizing a relief state is implemented. Through this, the computer can transform the 3D body model generated on the basis of the medical image data of the new patient into a virtual body model to which relief is applied by applying it to a statistical model or a learning model for realizing the relief state.

In addition, in another embodiment, the computer may include body information (eg, height, weight, BMI, etc.) of each patient in data used when implementing the statistical model or learning model. Through this, the computer can implement a relief model (ie, a virtual body model reflecting the relief state) by reflecting the characteristics of the body surface change according to the patient's body condition.

In addition, in another embodiment, when the scan data and registration are performed using the body surface model (ie, the skin model) (ie, the relief model is a virtual body surface model to which relief is applied), the computer is configured with the relief model. A blood vessel or organ model (ie, a model implemented in three dimensions for organ and blood vessel placement based on medical imaging data) is matched. Through this, the computer can complete a 3D virtual body model capable of surgical simulation. At this time, when there is no change in the organ placement state due to relief, the blood vessel or organ model may be applied as it is modeled based on the medical image data photographed by reflecting the surgical posture.

8 is a configuration diagram of an apparatus according to an embodiment.

The processor 402 may include one or more cores (not shown) and a connection path (eg, a bus) for transmitting and receiving signals to and from a graphic processing unit (not shown) and / or other components. .

The processor 402 according to an embodiment performs the relief model generation method described with reference to FIGS. 1 to 7 by executing one or more instructions stored in the memory 404.

For example, the processor 402 may obtain medical image data of a patient by executing one or more instructions stored in a memory, obtain scan data of the patient's body that is in an ups and downs state, and receive the medical image data. A step of generating a relief model by matching the scan data may be performed.

Meanwhile, the processor 402 temporarily and / or permanently stores a signal (or data) processed inside the processor 402 (RAM: Random Access Memory, RAM) and ROM (Read-Only Memory). , Not shown). In addition, the processor 402 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processing unit, RAM, and ROM.

Programs (one or more instructions) for processing and controlling the processor 402 may be stored in the memory 404. Programs stored in the memory 404 may be divided into a plurality of modules according to functions.

9 is a schematic diagram of a system capable of performing robotic surgery according to the disclosed embodiment.

According to FIG. 9, the robotic surgical system includes a medical imaging device 10, a server 100, and a control unit 30, a display 32 and a surgical robot 34 provided in the operating room. Depending on the embodiment, the medical imaging device 10 may be omitted from the robotic surgery system according to the disclosed embodiment.

In one embodiment, the surgical robot 34 includes an imaging device 36 and a surgical tool 38.

In one embodiment, robotic surgery is performed by a user controlling the surgical robot 34 using the control unit 30. In one embodiment, robotic surgery may be performed automatically by the control unit 30 without user control.

The server 100 is a computing device including at least one processor and a communication unit.

The control unit 30 includes a computing device including at least one processor and a communication unit. In one embodiment, the control unit 30 includes hardware and software interfaces for controlling the surgical robot 34.

The imaging device 36 includes at least one image sensor. That is, the imaging device 36 includes at least one camera device, and is used to photograph an object, that is, a surgical site. In one embodiment, the imaging device 36 includes at least one camera coupled with the surgical arm of the surgical robot 34.

In one embodiment, the image captured by the imaging device 36 is displayed on the display 32.

In one embodiment, the surgical robot 34 includes one or more surgical tools 38 capable of performing cutting, clipping, fixing, grabbing, etc. of the surgical site. The surgical tool 38 is used in combination with the surgical arm of the surgical robot 34.

The control unit 30 receives information necessary for surgery from the server 100 or generates information necessary for surgery to provide to the user. For example, the control unit 30 displays information required for surgery, which is generated or received, on the display 32.

For example, the user performs robot surgery by controlling the movement of the surgical robot 34 by operating the control unit 30 while looking at the display 32.

The server 100 generates information necessary for robotic surgery using medical image data of an object previously photographed from the medical image photographing equipment 10 and provides the generated information to the control unit 30.

The control unit 30 provides information to the user by displaying the information received from the server 100 on the display 32 or controls the surgical robot 34 using the information received from the server 100.

In one embodiment, the means that can be used in the medical imaging equipment 10 is not limited, for example, CT, X-Ray, PET, MRI, and other various medical imaging means can be used.

In the disclosed embodiment, the surgical image obtained from the imaging device 36 is transferred to the control unit 30. Although not shown, a scanning device capable of scanning the patient's body may be further provided in the operating room. For example, the scanning equipment may include equipment capable of scanning the appearance of the patient's skin, that is, the body.

In one embodiment, the controller 30 may segment a surgical image obtained during surgery in real time.

In one embodiment, the control unit 30 transmits the surgical image to the server 100 during or after surgery.

The server 100 may analyze by dividing the surgical image.

The server 100 learns and stores at least one model for dividing and analyzing surgical images. In addition, the server 100 learns and stores at least one model for generating an optimized surgical process.

The server 100 uses training data to train at least one model, and the training data includes, but is not limited to, surgical images and labeling information for the surgical images.

The learning data management method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a hardware computer, and stored in a medium.

The above-described program is C, C ++, JAVA, machine language, etc., in which a processor (CPU) of the computer can be read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in the computer language of. Such code may include functional code related to a function or the like that defines necessary functions for executing the above methods, and control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include a memory reference-related code as to which location (address address) of the computer's internal or external memory should be referred to additional information or media necessary for the computer's processor to perform the functions. have. Also, when the processor of the computer needs to communicate with any other computer or server in the remote to execute the functions, the code can be used to communicate with any other computer or server in the remote using the communication module of the computer. It may further include a communication-related code for whether to communicate, what information or media to transmit and receive during communication, and the like.

The storage medium refers to a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short moment, such as a register, cache, memory, or the like. Specifically, examples of the medium to be stored include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers that the computer can access or various recording media on the computer of the user. In addition, the medium is distributed in a computer system connected by a network, so that the computer readable code may be stored in a distributed manner.

Through the embodiments of the present invention, the body model generated based on the general medical image data of the patient is implemented by the medical image data as it is implemented in a relief state during minimally invasive surgery (eg, robot surgery, laparoscopic surgery). It is possible to match the position and orientation of the camera with the virtual body model in the actual patient's body.

In addition, while the body model generated based on the patient's general medical image data is implemented in a undulating state during minimally invasive surgery (for example, robotic surgery or laparoscopic surgery), learning data for implementing the relief algorithm can be obtained. That is, as the body model based on the medical image data of the existing patient is transformed into undulated body surface scanning data before surgery, learning data for learning the relief algorithm is acquired, and new patients are obtained through the relief algorithm learned based on the learning data. The medical image data can be implemented in the same manner as the relief state during surgery.

Therefore, the medical staff can perform the surgery while viewing the same image as the simulation at the time of actual surgery as the simulation is performed before the surgery in the same camera view direction as at the time of minimally invasive surgery. Can prepare for unexpected situations. In addition, as the virtual camera enters the virtual camera in the same position and direction in the virtual body model that has the same body surface as the patient's body that is undulated during the actual surgery, the screen that the medical staff actually sees through the camera and the virtual camera on the virtual body model Since the direction of the gaze is the same, it is possible to perform surgical assistance by transmitting accurate information about the location where the medical staff is performing minimally invasive surgery through a virtual body model.

In addition, as the 3D body model is implemented using medical image data photographed in the same posture as the actual surgical posture, the organ disposition of the virtual body model can be implemented in the same manner as the organ disposition inside the body during actual surgery. Through this, the image information provided through the camera entering the same direction from the same location can be implemented identically.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but a person skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive.

400: relief model generating device
402: processor
404: memory

Claims (10)

A computer obtaining medical image data of a patient;
Obtaining scan data that scans the patient's body in a pneumoperitoneum state; And
Generating a relief model by matching the medical image data to the scan data; The relief model generation method comprising a. According to claim 1,
The medical image data,
Method for generating a relief model, characterized in that it is 3D modeling data of the patient's body generated based on the medical image of the patient previously taken. According to claim 2,
The medical image of the patient,
A method of generating a relief model, characterized in that the patient's body is a medical image taken at a predetermined angle. According to claim 1,
The step of generating the relief model,
Selecting one or more landmarks from the medical image data and the scan data, respectively; And
Matching each of the one or more landmarks selected from the medical image data according to each of the one or more landmarks selected from the scan data; The relief model generation method comprising a. According to claim 1,
The step of generating the relief model,
Warping the medical image data according to the scan data; Further comprising, relief model generation method. The method of claim 5,
The step of warping,
Selecting one or more landmarks respectively from the medical image data and the scan data; And
Warping the medical image data according to the scan data based on the selected one or more landmarks; The relief model generation method comprising a. According to claim 1,
The step of generating the relief model,
Non-rigid registration of the medical image data according to the scan data; Further comprising, relief model generation method. According to claim 1,
The step of generating the relief model,
Obtaining information about the patient's internal organs based on at least one indicator indicating the patient's body characteristics; Further comprising, relief model generation method. A memory for storing one or more instructions; And
And a processor executing the one or more instructions stored in the memory.
The processor executes the one or more instructions,
Obtaining medical image data of the patient;
Obtaining scan data that scans the patient's body in a undulating state; And
Generating a relief model by matching the medical image data to the scan data; To perform, the device. A computer program stored in a recording medium readable by a computer to perform the method of claim 1 in combination with a computer that is hardware.

Images