JP7182127B2 - ROBOT SURGERY SUPPORT DEVICE, INFORMATION OUTPUT METHOD, AND PROGRAM - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to a robotic surgery assistance device, a robotic surgery assistance method, and a program for assisting robotic surgery by a surgery assistance robot.

2. Description of the Related Art Conventionally, when minimally invasive robotic surgery is performed by a surgical assistance robot, a port is drilled into the body of a patient undergoing surgery for inserting forceps. The position of the port is generally determined according to the surgical procedure, but the optimum position has not yet been established. Japanese Patent Laid-Open No. 2002-200000 discusses Port Placement Planning. Specifically, the surgical port placement system of U.S. Pat. receiving a given parameter set for a given surgical procedure and, based on the given parameter set and a surgical port placement model, at least one port for a given patient for the given surgical procedure; Plan your location.

Also, once the port location is planned, the port is positioned on the patient's body surface for drilling the port at the planned port location. A rhomboid sheet (Simplifying Patient Positioning and Port Placement) (see Non-Patent Document 1) is known as a device for positioning a port in non-robotic laparoscopic surgery. In brain surgery, the Leksell Stereotactic System (registered trademark) (see Non-Patent Document 2) is known as a brain stereotaxic system. The device of Non-Patent Document 2 has a fixture for fixing bones as a reference and a ruler (scale).

U.S. Patent Application Publication No. 2014/0148816

Cestari A, Buffi NM, Scapaticci E, Lughezzani G, Salonia A, Briganti A, Rigatti P, Montorsi F, Guazzoni G., ``Simplifying patient positioning and port placement during robotic-assisted laparoscopic prostatectomy'', Eur Urol. 2010 Mar;57 (3):530-3, PMID: 19963314, DOI: 10.1016/j.eururo.2009.11.028 "Leksell Stereotactic System", [online], Elekta Neuroscience, [searched on October 1, 2018], Internet <URL: http://www.elekta.co.jp/products/stereotactic.html>

The device described in Non-Patent Document 1 only designates a two-dimensional position on the body surface of the abdomen of the subject, and it is difficult to designate a three-dimensional position on the subject. Also, this device cannot be finely adjusted for each subject. Also, the device is not intended for the precision required for robotic surgery. In addition, since this device assumes a non-pneumoperitoneum state, an error occurs when a pneumoperitoneum occurs after marking. In addition, the device described in Non-Patent Document 2 is fixed to the skull of the subject, so it is difficult to apply to anything other than brain surgery. This device is relatively large in size relative to the subject, making it difficult to handle the device easily. This device has many degrees of freedom for adjustment, and it takes time to place the device on the subject. Also, this device does not follow the subject's breathing or body movements.

The present disclosure has been made in view of the circumstances described above, and provides a robotic surgery support apparatus, a robotic surgery support method, and a device that enable designation of a three-dimensional position of a port in a subject using a device with high operability. Offer a program.

One aspect of the present disclosure is a robotic surgery assisting apparatus that assists minimally invasive robotic surgery by a surgery assisting robot, comprising a processing unit, wherein the processing unit includes three-dimensional pixels obtained by imaging a subject. obtaining 3D data of the subject including pixel values of and positioning the plurality of ports at planned locations for the insertion of an endoscopic camera or end effector perforated into the body surface of the subject. obtains jig information about the positioning jig including the shape of the positioning jig to be positioned, obtains reference position information on the subject for aligning the subject and the positioning jig, and obtains in virtual space , by installing the positioning jig in accordance with the reference position and performing a simulation for determining the positions of the plurality of ports perforated in the body surface of the subject in the 3D data, deriving planned positions of the plurality of ports on the body surface , and planning the plurality of ports on the positioning jig based on the jig information and the planned positions of the plurality of ports on the body surface; It is a robotic surgery support device that outputs information indicating a position .

One aspect of the present disclosure is an information output method in a robotic surgery assisting apparatus that assists minimally invasive robotic surgery by a surgery assisting robot, the information output method including pixel values of three-dimensional pixels obtained by imaging a subject. The shape of a positioning jig that acquires 3D data of a subject and positions the plurality of ports at the planned positions of the plurality of ports for inserting an endoscope camera or an end effector perforated on the body surface of the subject. to obtain jig information about the positioning jig, obtain information of a reference position on the subject for aligning the subject and the positioning jig, and align with the reference position in a virtual space by installing the positioning jig in the 3D data and performing a simulation for determining the positions of the plurality of ports perforated in the body surface of the subject in the 3D data, the plurality of ports on the body surface of the subject deriving the planned positions of the ports, and generating information indicating the planned positions of the plurality of ports on the positioning jig based on the jig information and the planned positions of the plurality of ports on the body surface of the subject; It is an information output method to output.

One aspect of the present disclosure is a program for causing a computer to execute the information output method.

According to the present disclosure, it is possible to designate the three-dimensional position of the port in the subject using a device with high operability.

FIG. 1 is a block diagram showing a hardware configuration example of a robotic surgery assistance device according to a first embodiment; Block diagram showing an example of the functional configuration of a robotic surgery support device FIG. 11 shows examples of MPR cross-sectional images of the abdomen before and after pneumoperitoneum simulation is performed. Diagram for explaining an example of measuring the port position of an existing port FIG. 11 is a diagram showing a first arrangement plan example of the positions of ports installed on the body surface of a subject; FIG. 11 is a diagram showing a second arrangement plan example of the positions of ports installed on the body surface of the subject; FIG. 11 is a diagram showing a third arrangement plan example of port positions to be installed on the body surface of the subject; A diagram showing an example of the positional relationship between a subject, a port, a trocar, and a robot arm during robotic surgery. Flowchart showing an example of a port position simulation procedure using a robotic surgery support device Flowchart showing an operation example when calculating a port position score by a robotic surgery support device Diagram showing an example of working area defined based on port position A diagram showing an example of a positioning jig in the first output example of parameters A diagram showing an example of an axial section of the abdomen of a subject on which a positioning jig is installed in the first output example of parameters. Flowchart showing an example of operation by the robotic surgery support device in the first output example of parameters A diagram showing an example of a state of a subject on which a positioning jig is installed in the first output example of parameters, viewed from the ventral side. A diagram showing an example of a positioning jig in a second output example of parameters FIG. 11 is a diagram showing an example of an axial cross-section of the abdomen of the subject on which the positioning jig is installed in the second output example of parameters; Flowchart showing an operation example of the robotic surgery support device in the second output example of parameters A diagram showing an example of a state of a subject on which a positioning jig is installed in a second output example of parameters, viewed from the ventral side. A diagram showing an example of a positioning jig in the third output example of parameters A diagram showing an example of an axial cross-section of the abdomen of a subject on which a positioning jig is installed in the third output example of parameters. Flowchart showing an operation example of the robotic surgery support device in the third output example of parameters FIG. 11 is a diagram showing an example of the appearance of the subject on which the positioning jig is installed in the third output example of parameters, viewed from the ventral side; A diagram showing an example of a positioning jig in the fourth output example of parameters A diagram showing an example of an axial cross-section of the abdomen of a subject on which a positioning jig is installed in the fourth output example of parameters. A diagram showing an example of a state of the subject on which the positioning jig is installed in the fourth output example of parameters, viewed from the ventral side. Classification diagram of the features of the positioning jigs of the comparative example and the present embodiment A diagram showing an example of the positional relationship between the subject and each device in the operating room

Embodiments of the present disclosure will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration example of a robotic surgery support device 100 according to the first embodiment. The robotic surgery support device 100 may support robotic surgery by the surgery support robot 300, and may perform preoperative simulation, intraoperative simulation, and intraoperative navigation, for example.

The surgical assistance robot 300 includes a robot operating terminal, a robot main body, and an image display terminal.

The robot operating terminal includes a hand controller and foot switch operated by the operator. The robot operation terminal operates a plurality of robot arms AR provided on the robot main body according to the operator's operation of the hand controller or foot switch. Also, the robot operation terminal includes a viewer. The viewer may be a stereo viewer and may display a three-dimensional image by fusing the images captured by the endoscope. It should be noted that a plurality of robot operation terminals may exist, and robot surgery may be performed by a plurality of operators operating the plurality of robot operation terminals.

The robot body includes a plurality of robot arms AR for performing robotic surgery and end effectors EF (forceps, instruments) as surgical instruments attached to the robot arms AR.

The robot main body of the surgical assistance robot 300 has four robot arms AR, a camera arm to which an endoscope camera is attached, and an end effector EF to be operated by the right hand controller of the robot operation terminal. A first end effector arm, a second end effector arm to which an end effector EF operated by a left hand controller of a robot operation terminal is attached, and a third end effector arm to which a replacement end effector EF is attached. include. Each robot arm AR has a plurality of joints, and has a motor and an encoder corresponding to each joint. Each robot arm AR has at least 6 degrees of freedom, preferably 7 or 8 degrees of freedom, and may operate in three-dimensional space and be movable in each direction in three-dimensional space. The end effector EF is an instrument that actually comes into contact with a treatment target within the subject PS in robotic surgery, and enables various treatments (for example, grasping, cutting, peeling, and suturing).

End effectors EF may include, for example, grasping forceps, dissecting forceps, electrocautery, and the like. As for the end effector EF, a plurality of separate end effectors EF that are different for each role may be prepared. For example, in robotic surgery, a procedure may be performed in which two end effector EFs hold or pull tissue and one end effector EF cuts the tissue. The robot arm AR and end effector EF may operate based on instructions from the robot operation terminal.

The image display terminal has a monitor, a controller for processing an image captured by the camera of the endoscope, and displaying the image on the viewer or monitor, and the like. The monitor is viewed, for example, by a robotic surgical assistant or a nurse.

The surgery support robot 300 receives the hand controller and foot switch of the robot operation terminal operated by the operator, controls the operation of the robot arm AR and the end effector EF of the robot main body, and performs various treatments on the subject PS. perform surgery. In robotic surgery, laparoscopic surgery may be performed within the subject PS.

In robotic surgery, the body surface of the subject PS may be perforated with a port PT, and pneumoperitoneum may be performed through the port PT. In a pneumoperitoneum, carbon dioxide may be pumped to distend the peritoneal cavity of the subject PS. A trocar TC may be installed at the port PT. The trocar TC has a valve and keeps the inside of the subject PS airtight. Also, air (for example, carbon dioxide) is continuously introduced into the subject PS in order to maintain an airtight state.

An end effector EF (a shaft of the end effector EF) is inserted through the trocar TC. The valve of the trocar TC is opened when the end effector EF is inserted, and the valve of the trocar TC is closed when the end effector EF is removed. An end effector EF is inserted from the port PT via the trocar TC, and various treatments are performed according to the surgical procedure. Robotic surgery may be applied not only to laparoscopic surgery targeting the abdomen, but also to arthroscopic surgery involving areas other than the abdomen.

As shown in FIG. 1 , the robotic surgery support device 100 includes a communication unit 110 , a user interface (UI) 120 , a display 130 , a processor 140 and a memory 150 . Note that the UI 120 , the display 130 , and the memory 150 may be included in the robotic surgery support device 100 or may be provided separately from the robotic surgery support device 100 .

A CT (Computed Tomography) apparatus 200 is connected to the robotic surgery support apparatus 100 via a communication unit 110 . The robotic surgery support apparatus 100 acquires volume data from the CT apparatus 200 and processes the acquired volume data. The robotic surgery support apparatus 100 may be configured by a PC (Personal Computer) and software installed in the PC. The robotic surgery support device 100 may be configured as part of the surgery support robot 300 .

A surgery support robot 300 is connected to the robotic surgery support apparatus 100 via a communication unit 110 . The robotic surgery support device 100 may, for example, provide various data, information, and images to the surgery support robot 300 to support robotic surgery. The robotic surgery support apparatus 100 acquires, for example, information on the mechanism and operation of the surgery support robot 300 from the surgery support robot 300 and data obtained before, during, or after the surgery with the robot, and stores the acquired information and data. Various analyzes and analyzes may be performed based on this. Analysis results and analysis results may be visualized.

A measuring instrument 400 may be connected to the robotic surgery support apparatus 100 via the communication unit 110 . The measuring instrument 400 may measure information (for example, body surface position of the subject PS) regarding the subject PS (for example, a patient) to be operated by the surgical assistance robot 300 . The measuring instrument 400 may measure the position of the port PT provided on the body surface of the subject PS. Gauge 400 may be, for example, depth sensor 410 . The depth sensor 410 may be included in the surgical assistance robot 300 (for example, a robot main body), or may be installed on the ceiling of an operating room where robotic surgery is performed. Moreover, the operation unit of the measuring instrument 400 may receive an input of the result of manual measurement. For manual measurements, for example, patient information and port locations on the body surface may be measured with a ruler or tape measure.

Further, the robotic surgery support apparatus 100 may be connected to the CT apparatus 200 or, instead of being connected to the CT apparatus 200, connected to a device capable of capturing various images. The device may be an angiography device, an ultrasound device, or the like. This device may be used to check the internal state of the subject PS before and during robotic surgery.

The CT apparatus 200 irradiates a living body with X-rays and captures an image (CT image) by utilizing differences in absorption of X-rays by tissues in the body. The subject PS may be, for example, a human body or a living body. Note that the subject PS may not be a human body or a living body. For example, it may be an animal or a surgical training phantom.

A plurality of CT images may be captured in time series. The CT apparatus 200 generates volume data containing information on arbitrary points inside the living body. Arbitrary locations inside the living body may include various organs (eg, brain, heart, kidney, large intestine, small intestine, lung, chest, mammary gland, prostate, lung). A pixel value (CT value, voxel value) of each pixel (voxel) in the CT image is obtained by capturing the CT image. The CT apparatus 200 transmits volume data as CT images to the robotic surgery support apparatus 100 via a wired line or a wireless line.

Specifically, the CT apparatus 200 includes a gantry (not shown) and a console (not shown). The gantry includes an X-ray generator (not shown) and an X-ray detector (not shown), and detects X-rays that have passed through the subject PS by taking an image at a predetermined timing instructed by the console. , to obtain X-ray detection data. The x-ray generator includes an x-ray tube (not shown). The console is connected to the robotic surgery support device 100 . A console obtains a plurality of X-ray detection data from the gantry and generates volume data based on the X-ray detection data. The console transmits the generated volume data to the robotic surgery support device 100 . The console may include an operation unit (not shown) for inputting patient information, imaging conditions for CT imaging, imaging conditions for administration of a contrast agent, and other information. This operation unit may include input devices such as a keyboard and a mouse.

The CT apparatus 200 can also acquire a plurality of three-dimensional volume data by continuously capturing images and generate a moving image. Data of a moving image made up of a plurality of three-dimensional volume data is also called 4D (four-dimensional) data.

The CT apparatus 200 may capture CT images at each of a plurality of timings. The CT apparatus 200 may capture a CT image with the subject PS being contrasted. The CT apparatus 200 may capture a CT image while the subject PS is not contrast-enhanced.

In the robotic surgery support device 100, the communication unit 110 communicates various data and information with other devices. The communication unit 110 may communicate various data with the CT apparatus 200 , the surgery support robot 300 and the measuring instrument 400 . The communication unit 110 performs wired communication and wireless communication. The communication unit 110, the CT apparatus 200, the surgical assistance robot 300, and the measuring instrument 400 may be connected by wire or wirelessly.

The communication unit 110 may acquire various types of information for robotic surgery from the surgical assistance robot 300 . This various information may include, for example, kinematics information of the surgical assistance robot 300 . The communication unit 110 may transmit various information for robotic surgery to the surgical assistance robot 300 . This various information may include, for example, information (such as images and data) generated by the processing unit 160 .

The communication unit 110 may acquire various types of information for robotic surgery from the measuring instrument 400 . For example, it may include positional information on the body surface of the subject PS measured by the measuring instrument 400 and information on the port positions perforated on the body surface of the subject PS.

The communication unit 110 may acquire volume data from the CT apparatus 200 . The acquired volume data may be immediately sent to the processor 140 for various processing, or may be stored in the memory 150 and then sent to the processor 140 for various processing when necessary. Also, the volume data may be acquired via a recording medium or a recording medium.

Volume data captured by the CT apparatus 200 may be sent from the CT apparatus 200 to an image data server (PACS: Picture Archiving and Communication Systems) (not shown) and stored. The communication unit 110 may acquire volume data from this image data server instead of acquiring it from the CT apparatus 200 . Thus, the communication unit 110 functions as an acquisition unit that acquires various data such as volume data.

UI 120 may include a touch panel, pointing device, keyboard, or microphone. The UI 120 accepts arbitrary input operations from the user of the robotic surgery support apparatus 100 . Users may include physicians, radiologists, or other Paramedic Staff. The doctor may include an operator who operates the robot operation terminal to manually perform the robot surgery, and an assistant who assists the robot surgery in the vicinity of the subject PS.

The UI 120 accepts operations such as specifying a region of interest (ROI) in volume data and setting brightness conditions. Regions of interest may include regions of various tissues (eg, blood vessels, bronchi, organs, bones, brain, heart, legs, neck, blood flow). The tissue may broadly include tissue of the subject PS, such as diseased tissue, normal tissue, organ, and organ. In addition, the UI 120 may accept operations such as specification of a region of interest and setting of brightness conditions in volume data or an image based on the volume data (for example, a three-dimensional image or a two-dimensional image to be described later).

The display 130 may include an LCD (Liquid Crystal Display) and displays various information. Various information may include a three-dimensional image or a two-dimensional image obtained from volume data. A three-dimensional image may include a volume rendering image, a surface rendering image, a virtual endoscopic image (VE image), a virtual ultrasound image, a CPR (Curved Planar Reconstruction) image, and the like. A volume rendering image is a RaySum image (also simply referred to as a "SUM image"), a MIP (Maximum Intensity Projection) image, a MinIP (Minimum Intensity Projection) image, an Average image, or a Raycast image. may include Two-dimensional images may include axial images, sagittal images, coronal images, MPR (Multi Planer Reconstruction) images, and the like. Three-dimensional images and two-dimensional images may include color fusion images.

The memory 150 includes primary storage devices such as various ROMs (Read Only Memories) and RAMs (Random Access Memories). The memory 150 may include secondary storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives). The memory 150 may include a tertiary storage device such as a USB memory or an SD card. The memory 150 stores various information. The various information may include information acquired via the communication unit 110, information and images generated by the processor 140, setting information set by the processor 140, and various programs. Information acquired via the communication unit 110 includes, for example, information from the CT apparatus 200 (for example, volume data), information from the surgical assistance robot 300, information from the measuring instrument 400, and information from an external server. good. The memory 150 is an example of a non-transitory recording medium on which programs are recorded.

The projection unit 170 projects visible light (for example, laser light) toward the subject. The projection unit 170 displays various information (for example, port position information) on the body surface of the subject PS (for example, the body surface of the abdomen) by projecting visible light. Visible light, that is, information displayed on the body surface of the subject PS is confirmed by a user (for example, an assistant).

The processor 140 may include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit). The processor 140 functions as a processing unit 160 that performs various processes and controls by executing programs stored in the memory 150 .

FIG. 2 is a block diagram showing a functional configuration example of the processing unit 160. As shown in FIG.

The processing unit 160 includes an area extraction unit 161 , an image generation unit 162 , a deformation simulation unit 163 , a port position processing unit 164 , a display control unit 166 and a projection control unit 167 .

The processing unit 160 controls each unit of the robotic surgery support device 100 . Note that each unit included in the processing unit 160 may be implemented as different functions by one piece of hardware, or may be implemented as different functions by a plurality of pieces of hardware. Also, each unit included in the processing unit 160 may be realized by a dedicated hardware component.

The region extraction unit 161 may perform segmentation processing on the volume data. In this case, the UI 120 receives an instruction from the user, and information on the instruction is sent to the area extracting section 161 . The region extracting unit 161 may perform segmentation processing from the volume data by a known method based on the instruction information to extract a region of interest (segment). Alternatively, the region of interest may be manually set by detailed instructions from the user. Further, when an observation target is predetermined, the region extracting unit 161 may perform segmentation processing from the volume data without a user instruction to extract a region of interest including the observation target. The extracted regions may include regions of various tissues (eg, blood vessels, bronchi, organs, bones, brain, heart, legs, neck, blood flow, mammary glands, breast, tumors). The observed subject may be a subject undergoing robotic surgery.

The image generation section 162 may generate a three-dimensional image or a two-dimensional image based on the volume data acquired by the communication section 110 . The image generation unit 162 may generate a three-dimensional image or a two-dimensional image based on the designated area or the area extracted by the area extraction unit 161 from the volume data acquired by the communication unit 110 .

The deformation simulation unit 163 performs processing related to deformation in the subject PS to be operated. For example, the deformation simulation unit 163 may perform a pneumoperitoneum simulation to virtually perform pneumoperitoneum on the subject PS. A specific method of pneumoperitoneum simulation may be a known method, such as the method described in Reference Non-Patent Document 1, for example. That is, the deformation simulation unit 163 performs a pneumoperitoneum simulation based on the volume data (volume data before pneumoperitoneum (non-pneumoperitoneum state)) acquired from the communication unit 110 or the region extracting unit 161. Volume data (volume data of a pneumoperitoneum state) may be generated. The pneumoperitoneum simulation enables the user to assume a pneumoperitoneum state of the subject PS and observe a virtual pneumoperitoneum state without actually performing a pneumoperitoneum on the subject PS. Among the pneumoperitoneum states, the pneumoperitoneum state estimated by the pneumoperitoneum simulation may be referred to as a virtual pneumoperitoneum state, and the actual pneumoperitoneum state may be referred to as a real pneumoperitoneum state.

(Reference non-patent document 1) Takayuki Kitasaka, Kensaku Mori, Yuichiro Hayashi, Yasuhito Suenaga, Makoto Hashizume, and Jun-ichiro Toriwaki, “Virtual Pneumoperitoneum for Generating Virtual Laparoscopic Views Based on Volumetric Deformation”, MICCAI (Medical Image Computing and Computer- Assisted Intervention), 2004, P559-P567

The pneumoperitoneum simulation may be a large deformation simulation using the finite element method. In this case, the deformation simulation unit 163 may segment the body surface including the subcutaneous fat of the subject PS and the abdominal internal organs of the subject PS. Then, the deformation simulation unit 163 may model the body surface as two layers of finite elements of skin and body fat, and model the abdominal viscera as finite elements. The deformation simulation unit 163 may optionally segment lungs and bones and add them to the model. The deformation simulation unit 163 may provide a gas region between the body surface and the internal organs of the abdomen, and expand (expand) the gas region (pneumoperitoneum space) according to virtual gas injection.

FIG. 3 is a diagram showing image examples of MPR cross-sections of the abdomen before and after performing a pneumoperitoneum simulation. The image G11 shows the state before the pneumoperitoneum simulation is performed, in which the abdomen of the subject PS is not inflated (non-pneumoperitoneum state). The image G12 shows a state after the pneumoperitoneum simulation is performed, in which the abdomen of the subject PS is inflated (virtual pneumoperitoneum state), and has a pneumoperitoneum space KS. In robotic surgery, the subject PS is operated on in a pneumoperitoneum state, so the deformation simulation unit 163 performs a pneumoperitoneum simulation on the volume data obtained by imaging in a non-pneumoperitoneum state to simulate a virtual pneumoperitoneum state. volume data may be derived.

The deformation simulation unit 163 may virtually deform an observation target such as an organ or a lesion in the subject PS. The observation target may be a surgical target operated on by an operator. The deformation simulation unit 163 may simulate, for example, how an organ is pulled, pushed, or cut by the end effector EF. Also, the deformation simulation unit 163 may simulate movement of an organ due to postural change, for example.

The port position processing unit 164 acquires information on a plurality of ports PT provided on the body surface of the subject PS. Information on the port PT may include identification information on the port PT, information on the position (port position) on the body surface of the subject PS where the port PT is punctured, information on the size of the port PT, and the like. Information on multiple ports may be held in the memory 150 or an external server as a template. Information on a plurality of ports may be defined by a technique. Information for multiple ports may be intended for use in preoperative planning.

The port position processing unit 164 may acquire information on a plurality of port positions from the memory 150 . The port location processing unit 164 may acquire information on multiple port locations from an external server via the communication unit 110 . The port position processing unit 164 may receive designation of port positions of a plurality of ports PT via the UI 120 and acquire information on the plurality of port positions. Information on a plurality of port positions may be information on a combination of a plurality of port positions.

The port position processing unit 164 acquires kinematics information of the surgical assistance robot 300 . Kinematics information may be held in memory 150 . The port position processing unit 164 may acquire kinematics information from the memory 150 . The port position processing unit 164 may acquire kinematics information from the surgical assistance robot 300 or an external server via the communication unit 110 . Kinematics information may differ for each surgical assistance robot 300 .

The kinematics information may include, for example, shape information about the shape of instruments (for example, robot arm AR, end effector EF) for performing robotic surgery provided in the surgical assistance robot 300 and motion information about the motion. This shape information includes at least one of the length and weight of each part of the robot arm AR and the end effector EF, the angle of the robot arm AR with respect to a reference direction (for example, horizontal plane), the mounting angle of the end effector EF with respect to the robot arm AR, and the like. may contain departmental information. This motion information includes, for example, the movable range of the robot arm AR and the end effector EF in the three-dimensional space, the position, velocity, and acceleration of the robot arm AR when operating the robot arm AR, and the robot arm AR when operating the end effector EF. may include at least some information such as position, velocity, acceleration, etc. for the .

In kinematics, the movable range of the other robot arm is defined together with the movable range of the own robot arm. Therefore, the surgery support robot 300 can avoid interference between a plurality of robot arms AR during surgery by having each robot arm AR of the surgery support robot 300 operate based on kinematics.

The port position processing unit 164 acquires surgical procedure information. A surgical procedure indicates a method of surgical operation for the subject PS. A surgical procedure may be specified via the UI 120 . A surgical procedure may define each procedure in robotic surgery. Depending on the treatment, the end effector EF required for the treatment may be determined. Therefore, the end effector EF to be attached to the robot arm AR may be determined according to the surgical procedure, and which type of end effector EF is attached to which robot arm AR may be determined. Also, depending on the treatment, a minimum area required for the treatment or a recommended area that is recommended to be reserved for the treatment may be determined.

The port position processing unit 164 acquires information on the target area. A target region may be a region that includes an object (eg, tissue (eg, blood vessels, bronchi, organs, bones, brain, heart, foot, neck)) on which robotic surgery is to be performed. The tissue may broadly include tissue of the subject PS, such as diseased tissue, normal tissue, organ, and organ.

The port position processing unit 164 may acquire information on the position of the target area from the memory 150 . The port position processing unit 164 may acquire information on the position of the target area from an external server via the communication unit 110 . The port position processing unit 164 may receive designation of the position of the target area via the UI 120 and acquire information on the position of the target area.

The port location processor 164 may perform port location simulation. The port position simulation may be a simulation for determining whether or not desired robotic surgery can be performed on the subject PS by the user operating the UI 120 . In the port position simulation, the user may operate the end effector EF inserted from each port position in virtual space while assuming surgery, and determine whether or not the target region to be operated can be accessed. In other words, in the port position simulation, the movable part (for example, the robot arm AR and the end effector EF) of the surgical assistance robot 300 related to the robotic surgery moves to the target area to be operated while receiving manual operation of the surgical assistance robot 300 by the user. It may be determined whether access to is possible without problems. The port position processing unit 164 may obtain planning information of port positions through port position simulation.

In the port position simulation, the above access is possible based on the volume data of the subject PS, the combination of multiple acquired port positions, the kinematics of the surgical assistance robot 300, the surgical procedure, the volume data of the virtual pneumoperitoneum state, etc. It may be determined whether The port position processing unit 164 may determine whether or not the target region is accessible at each port position while changing a plurality of port positions on the body surface of the subject PS, and may sequentially perform the port position simulation. The port position processing unit 164 may designate information on a finally preferable (for example, optimal) combination of port positions according to user input via the UI 120 . This may allow the port location processor 164 to plan multiple port locations to drill. Details of the port position simulation will be described later.

The port position processing unit 164 may derive (for example, calculate) a port position score indicating the degree of suitability for robotic surgery using a plurality of port positions provided on the body surface of the subject PS. In other words, a port location score based on a combination of multiple port locations indicates the value of the combination of multiple port locations for performing robotic surgery. The port position score may be calculated based on a combination of a plurality of port positions, the kinematics of the surgical assistance robot 300, the surgical procedure, the virtual pneumoperitoneum state volume data, and the like. A port location score is derived for each port location. Details of the port location score will be described later.

The port location processor 164 may adjust port locations based on the port location scores. In this case, the port position processing unit 164 may adjust the port position based on the amount of change in the port position score accompanying movement of the port position. The details of port position adjustment will be described later.

Thus, the port location processor 164 may derive multiple port locations to be drilled according to the port location simulation. Also, the port location processor 164 may derive a plurality of port locations to be drilled based on the port location score.

The display control unit 166 causes the display 130 to display various data, information, and images. The display control section 166 may display the three-dimensional image or the two-dimensional image generated by the image generating section 162 . The display control unit 166 may display an image showing information of a plurality of ports PT (for example, information of port positions) generated by the image generation unit 162 .

The projection control unit 167 controls projection of visible light by the projection unit 170 . The projection control unit 167 may control, for example, the frequency of visible light, the amount of light, the position at which visible light is projected, and the time (timing) at which visible light is projected.

The projection control unit 167 causes the projection unit 170 to project visible light toward the subject PS, and displays various information on the body surface of the subject PS (for example, the body surface of the abdomen). The projection control unit 167 may project a laser beam toward the body surface of the subject PS to mark a specific position on the body surface. This specific position may be, for example, the position of the port to be perforated, or the position where the observation target (for example, the affected part) exists on the volume data in the normal direction from this specific position on the body surface. That is, the projection control section 167 may be a laser pointer that indicates the port position.

Further, the projection control unit 167 causes the projection unit 170 to project visible light onto the body surface of the subject PS, and superimposes information (for example, information about port positions) that supports robotic surgery on the body surface of the subject PS. can be displayed. Information to be superimposed may be character information, graphic information, or the like. That is, the projection control unit 167 may assist the user in robotic surgery using augmented reality (AR) technology.

FIG. 4 is a diagram for explaining an example of measuring the port position of the perforated port PT1. Measurement of the port position may be three-dimensional measurement. In FIG. 4, a subject PS (for example, a patient) is placed lying down on a bed BD.

The depth sensor 410 may include a light emitting unit that emits infrared rays, a light receiving unit that receives infrared rays, and a camera that captures images. The depth sensor 410 may detect the distance from the depth sensor 410 to the subject PS based on the infrared rays emitted by the light emitting unit to the subject PS and the reflected light reflected and received by the subject PS. . The depth sensor 410 may detect the top, bottom, left, and right of the subject from the captured image captured by the camera. Thereby, the depth sensor 410 may acquire information on the three-dimensional position (three-dimensional coordinates) of each position (for example, the port position of the perforated port PT1) on the body surface of the subject PS.

Depth sensor 410 may have a processor and internal memory. The internal memory may hold information on the shape of the trocar TC. The depth sensor 410 refers to the shape information of the trocar TC held in the internal memory, detects (recognizes) the trocar TC installed in the port PT drilled in the body surface of the subject PS, and detects the trocar TC in three dimensions. A position may be detected (measured).

Moreover, a predetermined mark may be attached to the surface of the trocar TC. The depth sensor 410 may detect (recognize) the trocar TC by image recognition by capturing an image of a predetermined mark on the trocar TC as a feature point. Thereby, the depth sensor 410 can improve the recognition accuracy of the trocar TC, and can improve the measurement accuracy of the three-dimensional position of the trocar TC.

Further, the depth sensor 410 may not include an infrared sensor (light emitting unit and light receiving unit), but may include a stereo camera and measure the three-dimensional position of the trocar TC by image processing. In this case, the depth sensor 410 recognizes the trocar TC by object recognition in the image captured by the stereo camera, detects (recognizes) the position of the trocar TC on the body surface of the subject, and measures the distance to the trocar TC. By calculating, the three-dimensional position of the trocar TC may be measured.

The depth sensor 410 detects each position on the body surface of the subject PS and the position of the trocar TC within the reachable range of the infrared rays emitted from the infrared sensor and the range capable of being imaged by the camera (see range A1 in FIG. 4). can be measured.

Note that the deformation simulation unit 163 of the robotic surgery support apparatus 100 obtains from the depth sensor 410 the information of each position on the body surface of the subject PS in the state of the actual pneumoperitoneum, that is, the body of the subject PS in the state of the actual pneumoperitoneum. Information on the shape of the table may be obtained. Further, the deformation simulation unit 163 extracts the contour (corresponding to the body surface) of the subject PS based on the volume data of the subject PS in the non-pneumoperitoneum state, and extracts the contour of the subject PS in the non-pneumoperitoneum state. Information on each position on the body surface, that is, information on the shape of the body surface in the non-pneumoperitoneum state of the subject PS may be acquired.

The deformation simulation unit 163 calculates the difference between each position on the body surface of the subject PS in the actual pneumoperitoneum state and each position on the body surface of the subject PS in the non-pneumoperitoneum state, that is, the actual pneumoperitoneum of the subject PS. A difference between the shape of the body surface in the abdominal state and the shape of the body surface in the non-pneumoperitoneum state of the subject PS may be calculated. Thereby, the deformation simulation unit 163 can recognize the pneumoperitoneum condition for bringing the subject PS into the actual pneumoperitoneum state.

As the pneumoperitoneum condition, there may be a parameter indicating the amount of air supplied during pneumoperitoneum. The insufflation volume may be gas injection volume, gas pressure, or intraperitoneal gas volume. Lung volume, pulmonary function, cardiac function, age, sex, weight, medical history, and other factors may be used by the physician to determine the insufflation volume during the pneumoperitoneum simulation. A pneumoperitoneum condition may include a parameter indicative of the ease of stretching of the body tissue of the subject. For example, if the subject has experienced childbirth, the subject's skin is more likely to stretch, and the subject's pneumoperitoneum will be greater even if the amount of air supplied is the same. Pneumoperitoneum conditions may include waist circumference, subcutaneous fat, surgical history, age, sex, weight, medical history, and other parameters that affect the subject's ability to stretch body tissue. Further, the easiness of stretching of the body tissue of the subject may be a parameter for estimating the stiffness of organs and blood vessels. In addition, the stretchability of the body tissue of the subject may vary depending on the location on the body surface. The pneumoperitoneum condition and the pneumoperitoneum simulation may be represented by a large deformation simulation using the finite element method. Pneumoperitoneum conditions and pneumoperitoneum simulations may be expressed by the finite volume method, the level set method, the Lattice Boltzmann methods, the CIP method (Constrained Interpolation Profile Scheme), or a combination thereof.

Further, the deformation simulation unit 163 may correct the simulation method and simulation result of the pneumoperitoneum simulation based on the difference between the actual pneumoperitoneum state and the virtual pneumoperitoneum state obtained by the pneumoperitoneum simulation. That is, the deformation simulation unit 163 may correct the simulation method and the simulation results of the pneumoperitoneum simulation based on the actual pneumoperitoneum conditions. The deformation simulation section 163 may hold this correction information in the memory 150 . Further, the deformation simulation unit 163 may receive the scavenging amount from the pneumoperitoneum device via the communication unit 110, and correct the simulation method and simulation result of the pneumoperitoneum simulation. Thereby, the robotic surgery support device 100 can improve the accuracy of the pneumoperitoneum simulation.

Further, the port position processing unit 164 may correct the port position simulation result based on the pneumoperitoneum simulation result corrected based on the difference between the real pneumoperitoneum state and the virtual pneumoperitoneum state obtained by the pneumoperitoneum simulation. The port position processing section 164 may hold this correction information in the memory 150 . Thereby, the robotic surgery support device 100 can improve the accuracy of the port position simulation.

Next, a display example of the port position will be described.

The deformation simulation unit 163 performs a pneumoperitoneum simulation on volume data obtained in a non-pneumoperitoneum state (for example, preoperative CT imaging) to generate volume data in a virtual pneumoperitoneum state. The image generation unit 162 may generate a volume rendering image by volume rendering the volume data of the virtual pneumoperitoneum state. The image generator 162 may surface render the volume data of the pneumoperitoneum state to generate a surface rendered image.

The deformation simulation unit 163 performs a pneumoperitoneum simulation on volume data obtained in a non-pneumoperitoneum state (for example, preoperative CT imaging), and deforms each point of the volume data in the non-pneumoperitoneum state to represent a destination to which the pneumoperitoneum moves. information may be generated. The image generator 162 may create virtual pneumoperitoneum volume data by applying deformation information to volume data obtained in a non-pneumoperitoneum state (for example, preoperative CT imaging). The image generator 162 may generate a surface from volume data obtained in a non-pneumoperitoneum state (for example, preoperative CT imaging) to generate a surface rendering image. The image generation unit 162 may generate a surface rendering image in a virtual pneumoperitoneum state by applying deformation information to a surface generated from volume data obtained in a non-pneumoperitoneum state (for example, preoperative CT imaging). . The deformation information should include at least information expressing the movement of the port position due to the pneumoperitoneum. The image generation unit 162 segments bones from volume data obtained in a non-pneumoperitoneum state (for example, preoperative CT imaging), excludes bones from deformation information assuming that they do not move due to pneumoperitoneum, and detects movements of other tissues. may be created as deformation information.

The display control unit 166 may cause the display 130 to superimpose the port positions derived by the port position processing unit 164 on the volume rendering image or surface rendering image of the virtual pneumoperitoneum state.

The projection control unit 167 may project visible light onto the port position derived by the port position processing unit 164 on the body surface of the subject PS (for example, a patient), point the port position with the visible light, and visualize the port position. . This allows the user to perform a treatment such as perforation on the port position while confirming the port position on the body surface of the subject PS.

Further, the projection control section 167 may project visible light onto the subject PS and display information indicating the port position derived by the port position processing section 164 on the body surface of the subject PS (for example, a patient). In this case, the projection control unit 167 may superimpose and display information indicating the port position (for example, identification information of the port, an arrow indicating the port position) on the subject PS using AR technology. Thereby, the user can refer to the guide information by the visible light, and perform a treatment such as perforation at the port position while confirming the information regarding the port position on the body surface of the subject PS.

The deformation information will now be described in detail.

The deformation simulation unit 163 uses the deformation information to generate virtual pneumoperitoneum volume data from the volume data in the non-pneumoperitoneum state. Based on a plurality of volume data (CT images) obtained before and after pneumoperitoneum, motion (deformation) of each part included in the volume data is detected to generate deformation information. In this case, the deformation simulation unit 163 performs motion analysis (deformation analysis) on deformation of a plurality of volume data based on volume data under a plurality of pneumoperitoneum conditions (which may also include a non-pneumoperitoneum state). Get deformation information at . A specific method of deformation analysis is described in Reference Patent Document 1 and Reference Patent Document 2, for example. These are techniques in the example of non-rigid body registration, but can be read as motion analysis (deformation analysis) and motion analysis information (deformation information).
(Reference Patent Document 1: US Pat. No. 8,311,300)
(Reference Patent Document 2: Japanese Patent No. 5408493)

As deformation information, the deformation simulation unit 163 may acquire information regarding the amount of movement of an arbitrary point in the volume data and information regarding the speed. When applying the method of Reference Patent Document 1, the deformation simulation unit 163 divides the volume data into two-dimensional lattice nodes (k, l), and divides the two-dimensional lattice node (k, l, t) in phase t of the two-dimensional lattice. When the coordinates are (x, y), based on the difference of a plurality of nodes (k, l, t) obtained by changing the value of phase t, the amount of movement related to the grid point of node (k, l) information may be calculated. Further, the deformation simulation unit 163 may calculate speed information by time-differentiating the movement amount information. Information on the amount of movement and speed may be represented by a vector.

When the deformation simulation unit 163 interpolates the deformation information of the two-dimensional lattice for each point of the entire volume data, the deformation information of each point of the volume data is obtained. By applying the deformation information of this predetermined point to each point of the area including the observation site, the deformation information of each point of the area including the observation site is obtained.

Further, when the method of Reference Patent Document 2 is applied, the deformation simulation unit 163, among the volume data (before and after the pneumoperitoneum) arranged in time series, the volume data tk-1 and its time information tk-1, and the volume data tk And the deformation information may be generated based on the time information tk. Deformation information may refer to information on corresponding relationships of corresponding positions or corresponding objects on a plurality of volume data, and information on the process of movement and change of positions and objects. A pixel of each volume data serves as an index indicating a position at an arbitrary time between time k−1 and time k. In addition, this allows the deformation simulation unit 163 to obtain more accurate deformation information by taking into account the movement of organs due to respiration and heartbeat.

Note that the deformation simulation unit 163 may perform deformation analysis using a known method without being limited to the method described in Reference Patent Document 1. Alternatively, deformation analysis may be performed using other known registration techniques. The robotic surgery assisting apparatus 100 can grasp to which position an arbitrary position in the subject has moved before and after the pneumoperitoneum by performing deformation analysis of each point and observation site using deformation information.

The deformation information may be anything as long as it is the result of the deformation simulation, and the deformation information may express where at least one point moves due to the pneumoperitoneum. Also, the deformation information may be expressed directly or indirectly. For example, the deformation information may be expressed as a set of a grid before deformation and a grid after deformation. For example, deformation information may be expressed as a motion vector of at least one point.

A plurality of deformation information and virtual pneumoperitoneum states can exist by changing the pneumoperitoneum conditions of the pneumoperitoneum simulation. As the pneumoperitoneum condition, there may be a parameter indicating the amount of air supplied during pneumoperitoneum. The insufflation volume may be gas injection volume, gas pressure, or intraperitoneal gas volume. Lung volume, pulmonary function, cardiac function, age, gender, weight, pre-existing conditions, and other factors that the physician uses to determine the insufflation volume may be used in determining the insufflation volume during the pneumoperitoneum simulation. A pneumoperitoneum condition may include a parameter indicative of the ease of stretching of the body tissue of the subject. For example, if the subject has experienced childbirth, the subject's skin is more likely to stretch, and the subject's pneumoperitoneum will be greater even if the amount of air supplied is the same. Pneumoperitoneum conditions may include waist circumference, subcutaneous fat, surgical history, age, sex, weight, medical history, and other parameters that affect the stretchability of the subject's body tissue. Further, the easiness of stretching of the body tissue of the subject may be a parameter for estimating the stiffness of organs and blood vessels. In addition, the stretchability of the body tissue of the subject may vary depending on the location on the body surface. The pneumoperitoneum condition and the pneumoperitoneum simulation may be represented by a large deformation simulation using the finite element method. Pneumoperitoneum conditions and pneumoperitoneum simulations may be expressed by the finite volume method, the level set method, the Lattice Boltzmann methods, the CIP method (Constrained Interpolation Profile Scheme), or a combination thereof.

Next, specific examples of standard port positions will be described. Standard port location arrangements are also applicable to this embodiment.

FIG. 5A is a diagram showing a first placement plan example of port positions to be installed on the body surface of the subject PS. FIG. 5B is a diagram showing a second layout plan example of port positions to be installed on the body surface of the subject PS. FIG. 5C is a diagram showing a third arrangement plan example of port positions to be installed on the body surface of the subject PS. The placement of multiple port locations may be planned according to the surgical procedure, for example. In FIGS. 5A to 5C, the physique of the subject PS and the position and size of a lesion to be observed are not considered.

It should be noted that the plurality of port positions shown in FIGS. 5A to 5C are port positions to be drilled. A slight error may occur between the position of the port to be drilled and the position of the actually drilled port, for example, an error of about 25 mm.

The ports PT provided on the body surface of the subject PS include a camera port PTC into which the camera CA is inserted, an end effector port PTE into which the end effector EF is inserted, an auxiliary port PTA into which forceps held by the assistant are inserted, may be included. A plurality of ports PT may exist for each type (eg, camera port PTC, end effector port PTE, auxiliary port PTA), and the size of each port PT may be the same or different for each type. For example, the end effector port PTE into which an end effector EF for suppressing an organ or an end effector EF that moves in a subject PS in a complicated manner is inserted is more likely than the end effector port PTE into which an end effector EF as an electric scalpel is inserted. can also be large. The arrangement position of the auxiliary port PTA may be planned relatively freely.

In FIG. 5A, many end effector ports PTE and auxiliary ports PTA are linearly arranged in the right direction and the left direction of the subject PS with the port position of the camera port PTC as a reference (apex). .

In FIG. 5B, a number of end effector ports PTE and auxiliary ports PTA are arranged linearly across the position of the umbilical hs. A camera port PTC is also arranged near the navel hs.

In FIG. 5C, a number of end effector port PTEs and auxiliary ports PTA are arranged in a straight line. The position of the navel hs is slightly deviated from the position on this straight line. A camera port PTC is also arranged near the navel hs.

In the existing plan, many ports PT are often arranged in a straight line because it is easy for the user to locate the port position and there is a sense of security. Of the plurality of ports PT, the camera port PTC need not be arranged at the center of the body surface of the subject PS.

FIG. 6 is a diagram showing an example of the positional relationship among the subject PS, port PT, trocar TC, and robot arm AR during robotic surgery.

The subject PS is provided with one or more ports PT. A trocar TC is arranged in each of the ports PT. An end effector EF is connected to (for example, inserted through) the trocar TC, and work (treatment) by the end effector EF within the subject is possible. The port location is fixedly placed and does not move intraoperatively. Therefore, the position of the trocar TC placed at the port position also does not move. On the other hand, the robot arm AR and the end effector are controlled based on the operation of the robot operation terminal, and the robot arm AR moves according to the intraoperative treatment. Therefore, the positional relationship between the robot arm AR and the trocar TC changes, and the angle of the trocar TC with respect to the body surface of the subject and the angle of the end effector EF attached to the trocar TC change. Note that, in FIG. 6, the monitor held by the assistant is also shown as an end effector.

Next, the operation of the robotic surgery support device 100 will be described.

First, the port position simulation procedure will be described. FIG. 7 is a flow chart showing an example of a port position simulation procedure.

First, the port position processing unit 164 acquires volume data including the subject PS, for example, via the communication unit 110 (S11). The port position processing unit 164 acquires kinematics information of the surgical assistance robot 300, for example, via the communication unit 110 (S12). The deformation simulation unit 163 executes a pneumoperitoneum simulation (S13) and generates volume data of the virtual pneumoperitoneum state of the subject PS.

The port position processing unit 164 acquires surgical procedure information (S14). The port position processing unit 164 acquires and sets the positions (initial positions) of a plurality of ports PT according to the acquired technique (S14). In this case, the port position processing unit 164 may set the positions of a plurality of ports PT using three-dimensional coordinates.

The port position processing unit 164 acquires information on the position of the target area (S15).

The port position processing unit 164 determines whether or not each end effector EF inserted from each port PT can access the target area based on the positions of the plurality of ports and the position of the target area acquired in S14. (S16) Whether or not each end effector EF can access the target area may correspond to whether or not each end effector EF can reach all positions in the target area. In other words, it indicates whether or not the end effector EF (a plurality of end effector EFs as necessary) can perform the robotic surgery according to the acquired surgical procedure. It shows that it is possible.

If at least one of the end effectors EF cannot access at least part of the target area, the port position processing unit 164 determines the port position of at least one port PT included in the plurality of ports PT to be drilled, It moves along the body surface of the subject PS (S17). In this case, the port position processing unit 164 may move the port position based on user input via the UI 120 . The ports PT to be moved include at least the port PT into which the end effector EF was inserted that was inaccessible to at least part of the target area.

If each end effector EF can access the target area, the processing unit 160 ends the port position simulation process of FIG.

In this way, the robotic surgery support apparatus 100 performs the port position simulation to determine whether or not the target area can be accessed using the acquired port positions. It can be determined whether or not the surgical assistance robot 300 used can perform a robotic surgery. If the target area is inaccessible using multiple port locations, whether at least a portion of the port locations are changed via the UI 120 and the target area is accessible using the modified multiple port locations. may be determined again. The robotic surgical assist device 100 can plan a combination of multiple port locations accessible to the target area to multiple port locations to be drilled. In this way, the robotic surgery support device 100 can manually adjust the port positions and plan the port positions.

Next, an example of calculation of the port position score will be described.

It may be assumed that the plurality of port positions are determined, for example, according to the surgical procedure, and arranged at arbitrary positions on the body surface of the subject PS. Therefore, various combinations of port positions may be assumed. One end effector EF attached to the robot arm AR can be inserted into the subject PS from one port PT. Therefore, a plurality of end effectors EF attached to a plurality of robot arms AR can be inserted into the subject PS from a plurality of ports PT.

A reachable range within the subject PS through the port PT by one end effector EF is a working area (individual working area WA1) in which work (treatment in robotic surgery) can be performed by one end effector EF. Therefore, the overlapping area of the individual working areas WA1 of the plurality of end effectors EF becomes the working area (whole working area WA2) in which the plurality of end effectors EF can simultaneously reach within the subject PS via the plurality of ports PT. . Since surgical procedures require a predetermined number (eg, three) of end effectors EF to operate simultaneously, the entire working area WA2, which is simultaneously reachable by a predetermined number of end effectors EF, is taken into account.

In addition, since the position in the subject PS that can be reached by the end effector EF differs depending on the kinematics of the surgical assistance robot 300, it is taken into account in deriving the port position, which is the position where the end effector EF is inserted into the subject PS. In addition, since the position of the entire working area WA2 to be secured in the subject PS differs depending on the surgical procedure, it is taken into account in deriving the port position corresponding to the position of the entire working area WA2.

The port location processing unit 164 may calculate a port location score for each combination of a plurality of acquired (assumed) port locations. The port position processing unit 164 may plan a combination of port positions that achieves a port position score that satisfies a predetermined condition (for example, the maximum port score) among a plurality of hypothesized combinations of port positions. That is, multiple port locations included in a combination of planned port locations may be planned for multiple port locations to be drilled.

Note that the relationship between the port position and the operation of the movable part of the surgical assistance robot 300 may satisfy the relationships described in References 2 and 3, for example.
(Reference non-patent document 2): Mitsuhiro Hayashibe, Naoki Suzuki, Makoto Hashizume, Kozo Konishi, Asaki Hattori, “Robotic surgery setup simulation with the integration of inverse-kinematics computation and medical imaging”, computer methods and programs in biomedicine, 2006, P63-P72
(Reference non-patent document 3) Pal Johan From, “On the Kinematics of Robotic-assisted Minimally Invasive Surgery”, Modeling Identification and Control, Vol.34, No.2, 2013, P69-P82

FIG. 8 is a flow chart showing an operation example when the robotic surgery support apparatus 100 calculates a port position score.

Before the processing of FIG. 8, similarly to S11 to S14 of the port position simulation shown in FIG. , and the acquisition of information on the technique is performed in advance. Also, the kinematics information may include information on each end effector EF attached to each robot arm AR according to the surgical procedure. Note that the initial value of the port location score is zero. A port position score is an evaluation function (evaluation value) that indicates the value of a combination of port positions. The variable i is an example of work identification information, and the variable j is an example of port identification information.

The port position processing unit 164 generates a work list works, which is a list of works work_i using each end effector EF, according to the surgical procedure (S21). Work work_i includes information for each end effector EF to work in a surgical procedure according to the surgical procedure. Work work_i may include, for example, grasping, cutting, suturing, and the like. The work may include independent work by a single end effector EF and cooperative work by a plurality of end effectors EF.

The port position processing unit 164 plans the minimum region least_region_i, which is the minimum required region for performing the work work_i included in the work list works, based on the surgical procedure and volume data of the virtual pneumoperitoneum (S22). The minimum area may be defined as a three-dimensional area on the subject PS. The port location processing unit 164 generates a minimum region list Least_regions, which is a list of minimum regions least_region_i (S22).

The port position processing unit 164 selects a recommended region, which is a region recommended for performing work work_i included in the work list works, based on the surgical procedure, the kinematics of the surgical support robot 300, and the volume data of the virtual pneumoperitoneum state. Plan effective_region_i (S23). The port location processing unit 164 generates a recommended region list effective_regions, which is a list of recommended regions effective_region_i (S23). The recommended area may include the minimum space (minimum area) for working and the recommended space for the end effector EF to operate, for example.

The port position processing unit 164 acquires information on the port position list ports, which is a list of a plurality of port positions port_j (S24). Port locations may be defined in three-dimensional coordinates (x, y, z). The port location processing unit 164 may, for example, accept user input via the UI 120 and obtain a port location list ports including one or more port locations designated by the user. The port position processing unit 164 may acquire the port position list ports held as a template in the memory 150 .

Based on the surgical procedure, the kinematics of the surgical support robot 300, the virtual pneumoperitoneum state volume data, and the acquired plurality of port positions, the port position processing unit 164 performs each work work_i via each port position port_j. A port work area region_i, which is a workable area for each end effector EF, is planned (S25). A port workspace may be defined in a three-dimensional area. The port location processing unit 164 generates a port work area list regions, which is a list of port work areas region_i (S25).

The port position processing unit 164 subtracts the minimum region least_region_i from the port work region region_i for each work work_i to calculate a subtraction region (subtraction value) (S26). The port position processing unit 164 determines whether or not the subtraction region is an empty region (the subtraction value is a negative value) (S26). , region_i (region that the end effector EF cannot reach via the port PT) exists or not.

If the subtraction region is an empty region, the port position processing unit 164 calculates a volume value Volume_i that is the product of the recommended region effective_region_i and the port work region region_i (S27). Then, the port position processing unit 164 totals the volume values Volume_i calculated for each work work_i to calculate the total value Volume_sum. The port position processing unit 164 sets the total value Volume_sum as the port position score (S27).

That is, when the subtraction area is an empty area, there is no area not covered by the port work area in the minimum area, and it is preferable that this port position list ports (a combination of port positions port_j) is selected. A value for each work work_i is added to the port location score so that this port location list is likely to be selected. Also, by planning the port position score based on the volume Volume_i, the larger the minimum area and the port work area, the larger the port position score, and this port position list ports becomes more likely to be selected. Therefore, the port position processing unit 164 can easily select a combination of port positions that has a large minimum area and a large port working area and facilitates each procedure in surgery.

On the other hand, if the subtraction area is not an empty area, the port location processing unit 164 sets the port location score for the port location list ports to the value 0 (S28). In other words, there is an area not covered by the port work area in at least a part of the minimum area, and there is a possibility that the target work work_i cannot be completed. Therefore, it is preferable to select this port position list Posts. do not have. Therefore, the port location processing unit 164 sets the port location score to a value of 0 and excludes this port location list Posts from the selection candidates so that it is difficult for it to be selected. In this case, the port location processing unit 164 sets the overall port location score to a value of 0 even if it becomes an empty area when another work work_i is performed using the same port location list ports.

Note that the port position processing unit 164 may repeat each step in FIG. 8 for all work work_i and calculate the port position score taking into account all work work_i.

In this way, by deriving the port position score, the robotic surgery support apparatus 100 performs a robotic surgery using a plurality of port positions provided on the body surface of the subject PS. can be understood to what extent is appropriate. The individual working area WA1 and the entire working area WA2 are affected by the arrangement positions of a plurality of ports to be punched. Even in this case, the surgery support robot 300 takes into account the score (port position score) for each combination of a plurality of port positions, for example, the combination of a plurality of port positions where the port position score is equal to or greater than the threshold th1 (for example, the maximum). can be derived, and port positions can be set to facilitate robotic surgery.

In addition, by appropriately securing the working area based on the port position score, the user can secure a wide field of view inside the subject that cannot be directly viewed in robotic surgery, and a wide port working area, which can prevent unexpected situations. easier to deal with.

In addition, in robotic surgery, the position of the perforated port remains unchanged, but the robot arm AR, to which the end effector inserted into the port position is mounted, is movable within a predetermined range. Therefore, port location planning is important in robotic surgery, as robot arms AR can interfere with each other depending on the planned port location. Also, since the positional relationship between the surgical assistance robot 300 and the subject PS is basically not changed during surgery, the planning of the port positions is important.

FIG. 9 is a diagram showing an example of working areas determined based on port positions. Individual working area WA1 is an individual working area corresponding to each port position port_j. The individual working area WA1 may be an area within the subject PS that is reachable by an individual end effector. The area where each individual working area WA1 overlaps is the overall working area WA2. The entire working area WA2 may correspond to the port working area region_i. By using the port position score, the robotic surgery assisting apparatus 100 can optimize each port position and derive a suitable individual working area WA1 and overall working area WA2.

Next, the details of port position adjustment will be described.

The port position processing unit 164 acquires information on a plurality of port positions (candidate positions) based on a template held in the memory 150 or a user instruction via the UI 120, for example. The port position processing unit 164 calculates a port position score when using the plurality of port positions based on the combination of the acquired plurality of port positions.

The port position processing section 164 may adjust the position of the port PT based on the port position score. In this case, the port position processing unit 164 performs port position processing based on the obtained port position score for the plurality of port positions and the port position score when at least one of the plurality of port positions is changed. to adjust the position of the port PT. In this case, the port position processing unit 164 may take into consideration minute movements and differentiation of the port positions along each direction (x direction, y direction, z direction) in the three-dimensional space.

Note that the x direction may be along the horizontal direction with respect to the subject PS. The y direction may be the front-rear direction (thickness direction of the subject PS) with respect to the subject PS. The z direction may be the vertical direction (body axis direction of the subject PS) with respect to the subject PS. The x-direction, y-direction, and z-direction may be three directions defined by DICOM (Digital Imaging and COmmunications in Medicine). The x-direction, y-direction, and z-direction may be directions other than these, and may be directions that are not based on the subject PS.

For example, the port position processing unit 164 may calculate the port position score F(ports) for a plurality of port positions according to (Equation 1) and calculate the differential value F' of F.

F(port_j(x+Δx, y, z)) - F(port_j(x, y, z))
F(port_j(x, y+Δy, z)) - F(port_j(x, y, z)) ・・・(Formula 1)
F(port_j(x, y, z+Δz)) - F(port_j(x, y, z))

That is, the port position processing unit 164 calculates the port position score F in the case of the port position F(port_j(x+Δx, y, z)), and in the case of the port position F(port_j(x, y, z)) is calculated, and the difference is calculated. This difference value indicates the change in the port position score F with respect to a small change in the x direction in the port position F(port_j(x, y, z)), ie, the differential value F' of F in the x direction.

In addition, the port position processing unit 164 calculates the port position score F in the case of the port position F(port_j(x, y+Δy, z)), and in the case of the port position F(port_j(x, y, z)) is calculated, and the difference is calculated. This difference value indicates the change in the port position score F with respect to a small change in the y direction in the port position F(port_j(x, y, z)), that is, the differential value F' of F in the y direction.

In addition, the port position processing unit 164 calculates the port position score F in the case of the port position F(port_j(x, y, z+Δz)), and in the case of the port position F(port_j(x, y, z)) is calculated, and the difference is calculated. This difference value indicates the change in the port position score F with respect to a small change in the z direction in the port position F(port_j(x, y, z)), ie, the differential value F' of F in the z direction.

The port position processing unit 164 calculates the maximum value of the port position score based on the differential value F' in each direction. In this case, the port position processing unit 164 may calculate the port position with the maximum port position score according to the re-descent method based on the differential value F'. The port position processing unit 164 may adjust the port position and optimize the port position so that the calculated port position is the position to be drilled. Note that the port position does not have to be the port position where the port position score is the maximum, for example, it may be the position where the port position score is equal to or greater than the threshold value th1, as long as the port position score is improved (increased).

The port position processing unit 164 applies such port position adjustments to other port position adjustments included in a plurality of port position combinations, or to port position adjustments in other combinations of a plurality of port positions. You may apply. Thereby, the port position processing unit 164 can plan a plurality of ports PT whose respective port positions are adjusted (for example, optimized) at the port positions to be drilled. In this way, the robotic surgery support device 100 can automatically adjust the port positions and plan the port positions.

It should be noted that a plurality of port positions (coordinates of the port positions) may have an error of about a predetermined length (for example, 25 mm) between the planned drilling position and the actual drilling position, and the planning accuracy of the port positions is at most 3 mm. It is believed that there is. Therefore, the port position processing unit 164 selects a plurality of port positions included in a combination of port positions for each predetermined length on the body surface of the subject PS as the scheduled perforation positions in a round-robin manner. A score may be calculated for each. That is, the planned perforation positions may be arranged in a grid of a predetermined length (for example, 3 mm) on the body surface of the subject PS. Further, when the number of ports assumed on the body surface (for example, the number of grid intersections) is n, and the number of ports included in the combination of port positions is m, the port position processing unit 164 performs n m port locations may be sequentially selected from the port locations and combined, and the port location score for each combination may be calculated. In this way, when the grid is not excessively fine, such as a lattice with 3 mm intervals, it is possible to suppress the calculation load of the port position processing unit 164 from becoming excessive, and the port position scores of all combinations can be calculated. .

Note that the port position processing section 164 may adjust a plurality of port positions according to a known method. The port position processing unit 164 may plan the port positions to be drilled to a plurality of port positions included in the combination of port positions after adjustment. Known methods of port alignment may include techniques described in references 4, 5 and 3 below.

(Reference non-patent document 4) Shaun Selha, Pierre Dupont, Robert Howe, David Torchiana, “Dexterity optimization by port placement in robot-assisted minimally invasive surgery”, SPIE International Symposium on Intelligent Systems and Advanced Manufacturing, Newton, MA, 28- 31, 2001
(Reference non-patent document 5) Zhi Li, Dejan Milutinovic, Jacob Rosen, “Design of a Multi-Arm Surgical Robotic System for Dexterous Manipulation”, Journal of Mechanisms and Robotics, 2016
(Reference Patent Document 3) US Patent Application Publication No. 2007/0249911

Next, the positioning jig 50 for positioning the port PT will be described. The positioning jig 50 is an example of a three-dimensional positioning device.

During or before robotic surgery, port locations may be planned and the positioning fixture 50 may be used to position the ports PT at the planned port locations. Then, the positioned position may be marked, and the port PT may be drilled at the marked position.

Port placement planning of the positions (port placements) of the ports PT may include determining the port positions in the surgical plan and defining the port positions on the simulation. Port positioning may also include measuring and positioning the body surface to place the port PT according to the port placement in the surgical plan. Positioning of the port includes actually measuring and marking the piercing position of the port PT on the body surface using the positioning jig 50, or omitting the marking and piercing the port. good. Perforation of the port PT may include specifically perforating the body surface of the subject PS and inserting the trocar TC. Marking may include marking the port location with a marker or the like prior to drilling the port PT.

The robotic surgery support device 100 of this embodiment adds a function to preoperative simulation and outputs parameters for using the positioning jig 50 . This parameter is an example of port position specifying information for specifying planned positions of a plurality of ports PT using the positioning jig 50 . Examples of parameter output include first to fourth output examples. The positioning jigs 50A, 50B, 50C, and 50D described in the first to fourth output examples are examples of the positioning jig 50. FIG.

(First output example of parameters)
FIG. 10 is a diagram showing an example of the positioning jig 50A in the first output example of parameters. The positioning jig 50A is a plate-shaped template made of an elastic material. The elastic material may be, for example, stainless steel or elastic resin. The positioning jig 50A may be prepared in advance for each surgical procedure. The shape of the positioning jig 50A may be determined according to the surgical technique. Note that the shape of the positioning jig 50A shown in FIG. 10 is an example, and other shapes may be used.

The positioning jig 50A includes a marker Mhs, a port identification marker Pid, a landmark LM, and a scale SC.

The marker Mhs has a marker for alignment with the navel hs of the subject PS. The navel hs is an example of a reference position for aligning the subject PS and the positioning jig 50A. Note that the reference position may be a position other than the navel hs. The information indicating the marker portion Mhs is a semicircular mark in FIG. 10, but may be a mark of another shape, may be indicated by a symbol, or may be indicated by other information.

The port identification mark Pid is identification information for identifying the positioned port PT. Port identifications may include camera port PTC, end effector port PTE identification (e.g., ports A, B, C, D, E), assistant port PTA identification (e.g., "assist"). .

The landmark LM is a point captured as a reference point by a position measurement camera (for example, the camera of the depth sensor 410). The information indicating the landmark LM may be marks of various shapes. One or a plurality of landmarks LM may be present, and five are present in FIG.

The scale SC may have graduations arranged at predetermined intervals, or may have information indicating the length (for example, numerical information corresponding to the numbers on the graduations) at predetermined positions. The positioning jig 50A may specify the position of the port PT by the value of the scale SC (position on the scale SC).

FIG. 11 is a diagram showing an example of an axial section of the abdomen of the subject PS on which the positioning jig 50A is installed. As shown in FIG. 11, the positioning jig 50A is curved along the body surface of the subject PS due to its elasticity.

FIG. 12 is a flowchart showing an operation example of the robotic surgery support apparatus 100 in the first output example of parameters.

First, the port position processing unit 164 acquires reference position information for setting the positioning jig 50A using the volume data of the virtual pneumoperitoneum state (S31). The reference position may be a three-dimensional position, and may be the position of the navel hs of the subject PS.

The port position processing unit 164 acquires jig information regarding the positioning jig 50A. The jig information may be held in the memory 150 or acquired from an external server via the communication unit 110 . The jig information may include information on the shape and size of the positioning jig 50A. The shape of the positioning jig 50A may exhibit a standard shape that is not bent or stretched. The jig information may include information on the material of the positioning jig 50A (for example, information on elastic resin) and information on parameters related to the material (for example, information on the elastic modulus and the amount of deformation due to pneumoperitoneum). The jig information may include model numbers of the positioning jigs 50A, which exist in a plurality of types.

The port position processing unit 164 executes jig installation simulation for installing the positioning jig 50A in accordance with the reference position (S32). In the jig installation simulation, the port position processing unit 164 installs the positioning jig 50A by aligning the marker portion Mhs of the positioning jig 50A with the reference position in the virtual space. A state in which the positioning jig 50A is installed on the subject PS can be confirmed via the display 130 or the like. When the subject PS is in a non-pneumoperitoneum state, the acquired shape data of the positioning jig 50A may be used. When the subject PS is in a non-pneumoperitoneum state, the acquired shape data of the positioning jig 50A may be used as shape data in which the amount of deformation or extension due to the pneumoperitoneum is added. The port position processing unit 164 associates the position on the subject PS with the position on the positioning jig 50A by jig installation simulation.

The port position processing unit 164 performs port position simulation or port position adjustment taking into account the positioning jig 50A (S33). In this case, the port position processing unit 164 plans (for example, calculates) the port position to be drilled based on the installation position and shape of the positioning jig 50A in the port position simulation or port position adjustment taking into account the positioning jig 50A. do. For example, the port position processing unit 164 determines the port position by excluding a range on the body surface of the subject PS that overlaps with the positioning jig 50A and a range that cannot be reached by the positioning jig 50A (positioning impossible range). you can plan. Thus, in this port position simulation or port position adjustment, the port positions can be limited to positions that can be set with the positioning jig 50A.

The port position processing unit 164 outputs parameters indicating the position of each planned port position in the positioning jig 50A (S34). This parameter may be of the form "port A: 13 mm", "port assist: 107 mm", and so on. In this case, the port identification mark Pid is "A", and the position of port A may be specified at the position of the scale SC for this port with a value of 13 mm. The port identification mark Pid is "assist", and the position of the port assist may be specified at the position of the scale SC for this port with a value of 107 mm. That is, a value specifically indicating the position of the port PT may be output as a parameter.

In S34, the port location processor 164 may output the parameters to the display 130, for example. That is, the display 130 may display a value specifically indicating the position of the port PT related to the parameter. This display may be confirmed by the user. Then, the user positions the positioning jig 50A in real space by aligning it with the reference position and the marker part Mhs, and punches the position of the body surface indicated by the value on the scale SC of the positioning jig 50A. A port PT may be installed in the specimen PS.

FIG. 13 is a diagram showing an example of a state in which the subject PS on which the positioning jig 50A is installed is viewed from the ventral side (negative side in the y direction). The positioning jig 50A is aligned with the subject PS with reference to the marker Mhs and placed on the body surface. In the scale SC of the installed positioning jig 50A, the detailed positions of the ports A, B, C, D, E, and assist are identified by the values of the scale SC as derived parameters. Then, the user can punch the port PT positioned by the positioning jig 50A using an instrument for punching the port PT.

Thus, according to the first output example of the parameters, the robotic surgery support device 100 is a positioning jig capable of positioning the port PT at a planned port position on the three-dimensional coordinates with high accuracy. Parameters relating to tool 50A can be output. Therefore, the user can position the port PT to be drilled at the port position indicated by the parameter, for example, using the positioning jig 50A having a predetermined shape according to the surgical technique.

Further, the port position processing unit 164 may analogize how much the positioning jig 50 bends or expands and contracts due to the pneumoperitoneum based on the pneumoperitoneum simulation result, and generates and outputs parameters. In this case, the port position processing unit 164 may perform port position simulation and port position adjustment by limiting the range of port positions that can be set in the positioning jig 50 . For example, the port position processing unit 164 excludes port positions where the installation position of the positioning jig 50 overlaps the port position, excludes port positions far from the installation range of the positioning jig 50, Port locations may be planned by excluding port locations that are not within the scale SC of . The template shape of the positioning jig 50 may be determined by the surgical procedure.

By limiting the possible values of the parameters, the robotic surgery support apparatus 100 can limit the calculation range for deriving the parameters, and can speed up the calculation. In addition, the robotic surgery support apparatus 100 can reduce the possibility that unintended parameters are output, and can suppress the occurrence of a situation in which the positioning jig 50 cannot position the port position.

The positioning jig 50A in the first output example of the parameters may not be prepared in advance, but may be generated based on the output parameters (shape data) as in the second output example.

(Second output example of parameters)
In the second parameter output example, descriptions of items similar to those in the first parameter output example are omitted or simplified.

FIG. 14 is a diagram showing an example of the positioning jig 50B in the second parameter output example. The positioning jig 50B may be generated each time by a laser processing machine or a 3D printer. The positioning jig 50B may be formed by processing a plurality of plates (plate members) and connecting the plurality of plates with a tape tp or the like. The positioning jig 50B may be grooved so that one plate is processed and one plate is partially bent. A single plate positioning jig 50B may be bent by grooving. The plate or plates may be made of rigid material or other material, for example. The rigid material may be stainless steel, rigid resin, polypropylene, polyethylene, rigid vinyl chloride, and the like. One or more of the plates may have elastic material on part thereof.

The positioning jig 50B is not prepared in advance, but is generated based on shape data and pattern data as parameters.

The positioning jig 50B includes a marker portion Mhs, a port identification marker Pid, a landmark LM, and a port position indication portion. The port position indication part indicates the port position by the shape and label of the positioning jig 50B. The port position indicator may have a projection 51, a notch, and a port position indicator 51A that indicate the port position. Further, the port position indicator may indicate a range of port positions by the protrusion 51B instead of pointing the port position by one point by the protrusion 51. FIG. The protrusion 51B may have a trapezoidal shape by removing a part of the triangular vertices. The range of port locations may be the same as or correspond to the range of errors allowed in drilling the ports. This margin of error may be defined, for example, based on how much the port location score drops when moving each planned port location.

FIG. 15 is a diagram showing an example of an axial section of the abdomen of the subject PS on which the positioning jig 50B is installed. As shown in FIG. 15, the positioning jig 50B is bent along the body surface of the subject PS.

FIG. 16 is a flow chart showing an operation example of the robotic surgery support apparatus 100 in the second parameter output example.

First, the port position processing unit 164 acquires reference position information for setting the positioning jig 50A using the volume data of the virtual pneumoperitoneum state (S41). The reference position may be a three-dimensional position, and may be the position of the navel hs of the subject PS.

The port position processing unit 164 performs port position simulation or port position adjustment, and derives (for example, calculates) the port position to be drilled (S42). That is, the port location processing unit 164 plans port locations. Here, the reference position is determined in S41. The port position processing unit 164 virtually aligns the marker Mhs of the positioning jig 50 with the acquired reference position, and based on the derived port position in the subject PS, Derive (eg, calculate) port locations.

The port position processing unit 164 outputs parameters (for example, shape data) indicating the shape of the positioning jig 50B that can indicate the derived port positions (S43). Each port position is indicated by a port position indicating portion of the positioning jig 50B. Further, the parameters indicating the shape of the positioning jig 50B include information on the combination of plates (partial members of the positioning jig 50B) having port position indication portions that indicate the positions of the respective ports, shape data of each plate, may include a parameter indicating That is, the positioning jig 50B may be generated from one plate-shaped member, or may be generated by combining a plurality of plate-shaped members.

The output parameters may be input to a 3D printer or laser processing machine. The 3D printer or laser processing machine may generate the positioning jig 50B based on the input parameters (for example, the shape data of the positioning jig 50B, the combination of the plates and the shape of each plate). The 3D printer or laser processing machine may generate each plate based on the shape data of each plate, combine the plates based on the information on the combination of each plate, and generate the positioning jig 50B. The shape data of the positioning jig 50B may also include data of the size of the positioning jig 50B. Then, the user installs the positioning jig 50B by aligning it with the reference position and the marker Mhs in real space, and determines the shape (for example, projections 51, 51B, notches) and pattern (for example, the port position) of the positioning jig 50B. A port PT may be placed in the subject PS by drilling the location indicated by the indicator mark).

Note that the communication unit 110 of the robotic surgery support device 100 and the communication unit of the 3D printer or laser processing machine may exchange shape data and the like through communication. Also, shape data and the like may be transferred via a storage medium.

FIG. 17 is a diagram showing an example of a state in which the subject PS on which the positioning jig 50B is installed is viewed from the ventral side (negative side in the y direction). A positioning jig 50B is generated based on the shape data as derived parameters. The shape data takes into account the shape of the port position indication portion that indicates each port A, B, C, D, E and the auxiliary port PTA and the position of the port position indication portion in the positioning jig 50B. As a result, each port position indicator in the positioning jig 50B can specify each port A, B, C, D, and E and the auxiliary port PTA to specify the port position. Therefore, the user can punch the port PT positioned by the positioning jig 50A using an instrument for punching the port PT.

As described above, according to the second output example of the parameter, the robotic surgery support apparatus 100 can use the positioning jig 50B prepared in advance to specify the port position, even if the configuration of the positioning jig 50B ( For example, the port position can be specified by shape, pattern, color). In this case, the robotic surgery support apparatus 100 provides the shape data and pattern data of the positioning jig 50B indicated by the parameters to other devices (for example, 3D printers and laser processing machines), thereby performing positioning based on the shape data and pattern data. A jig 50B can be generated.

In addition, the planned position of the port is derived from the results of port position simulation and port position adjustment, and the shape of the positioning jig 50B is derived according to the planned position of the port. 50B may be generated. In this case, the positioning jig 50B may be generated and disposable for each operation. Also, the positioning jig 50B may be generated for each subject PS and discarded. As a result, the positioning jig 50B is kept clean when the positioning jig 50B is used, so that the robotic surgery assisting apparatus 100 can reduce the labor and time required for sterilizing the positioning jig 50B with an autoclave or the like.

The positioning jig 50B of the second output example of parameters may be prepared in advance as in the second output example instead of being generated based on the output parameters (shape data). That is, parameters for specifying the port position may be derived using the positioning jig 50B prepared in advance, and the port position may be indicated in detail by the positioning jig 50B using the derived parameters.

Also, the positioning jig 50 does not have to be generated from scratch based on the shape data. In this case, a plurality of plates (parts of the positioning jig 50) are prepared, and the port position processing unit 164 determines which plates are to be combined to generate the positioning jig 50B based on the derived shape data. may decide. That is, the port position processing unit 164 may output, as a parameter, selection information for selecting a plate-shaped member for generating the positioning jig 50B from among the plurality of plate-shaped members. Even in this case, the positioning jig 50B can be generated by combining the plates.

(Third output example of parameters)
In the third parameter output example, descriptions of items similar to those in the first and second parameter output examples are omitted or simplified.

FIG. 18 is a diagram showing an example of the positioning jig 50C in the first output example of parameters.

The positioning jig 50C has a stretchable region D1 made of a stretchable material and non-stretchable regions D2 formed of a non-stretchable material and formed at both ends of the stretchable region D1. A non-stretchable material is less stretchable than a stretchable material. The elastic material may be, for example, cloth, elastic resin, rubber. Stretchable resins may include vinyl (eg, polyvinyl chloride). The stretchable material may be an elastic material. That is, the stretchable region D1 may have stretchability and elasticity. A non-stretchable material may be a rigid material. In other words, the elastic region D1 may not have elasticity and may have hardness. The positioning jig 50C can be used like a scroll with the stretchable region D1 wound or unfolded. It is rolled up and stored when not in use. When used, it is deployed and utilized for positioning the port PT.

The positioning jig 50C may be translucent and may be made of a transparent material or a translucent material. When the positioning jig 50C has translucency, even if the positioning jig 50C is installed on the body surface of the subject PS, the user can visually observe the body surface and inside of the subject PS through the positioning jig 50C. Therefore, workability using the positioning jig 50C is improved, positioning accuracy is improved, and safety during positioning can be considered.

The positioning jig 50C may have a portion with a hole, and the portion with the hole may be the port position specifying portion. Moreover, the perforated portion may be configured to include a large number of perforations. Also, the perforated portion may be composed of one perforation. If the positioning jig 50C has a perforated portion, when the positioning jig 50C is placed on the body surface of the subject PS, the user can touch the body surface of the subject PS from above the positioning jig 50C. can be marked.

The positioning jig 50C may be prepared in advance. The positioning jig 50C may be commonly prepared in advance for each surgical technique or regardless of the surgical technique. The positioning jig 50C may be prepared in advance for each surgical procedure or for each surgical procedure. The shape of the positioning jig 50C may be a shape other than a rectangle.

The positioning jig 50C includes a marker portion Mhs and a port identification marker Pid in the elastic region D1. Further, the positioning jig 50C is provided with a port position designating portion for designating a planned port position based on derived parameters (described later). The positioning jig 50C may be punched or marked with locations corresponding to the planned port locations in the elastic region D1. That is, the port position specifying portion may be the opening of the elastic region D1 or the marking portion Mp. FIG. 18 illustrates the marking portion Mp.

Also, after the port position designating portion (for example, opening or marking portion Mp) of the positioning jig 50C is provided, the position of the port position designating portion can be used as a port position by using this port position designating portion as a mark. In other words, the user can easily drill holes at the planned port positions by positioning using the port position designating portion of the positioning jig 50C. In this case, the port PT can be installed on the body surface by cutting or tearing part of the elastic region D1. Therefore, once the positioning jig 50C is installed, it becomes unnecessary to remove the positioning jig 50C until the end of surgery, and the robotic surgery support apparatus 100 can reduce the burden on the operator and the assistant.

FIG. 19 is a diagram showing an example of an axial section of the abdomen of the subject PS on which the positioning jig 50C is installed. As shown in FIG. 19, the positioning jig 50A is bent along the body surface of the subject PS due to its stretchability. When positioning the positioning jig 50A on the body surface of the subject PS, the user may hold and pull the non-stretchable region D2 of the positioning jig 50A to deploy the positioning jig 50A. The presence of the non-stretchable region D2 allows the user to operate while holding the non-stretchable region D2, and the stretchable region D1 can be stably installed.

FIG. 20 is a flow chart showing an operation example of the robotic surgery support apparatus 100 in the third parameter output example.

First, the port position processing unit 164 acquires reference position information for setting the positioning jig 50C using the volume data of the virtual pneumoperitoneum state (S51). The reference position may be a three-dimensional position, and may be the position of the navel hs of the subject PS.

The port position processing unit 164 acquires jig information regarding the positioning jig 50C. The jig information may be held in the memory 150 or acquired from an external server via the communication unit 110 . The jig information may include shape data of the positioning jig 50C. The shape data of the positioning jig 50C may indicate a standard shape that is not bent or stretched. The jig information may include information on the material of the positioning jig 50C (for example, information on cloth) and information on parameters related to the material (for example, information on the elastic modulus, expansion ratio, deformation amount and expansion amount due to pneumoperitoneum).

The port position processing unit 164 executes jig installation simulation for installing the positioning jig 50C in accordance with the reference position (S52). In the jig installation simulation, the port position processing unit 164 installs the positioning jig 50C by aligning the marker Mhs with the reference position in the virtual space. A state in which the positioning jig 50C is installed on the subject PS can be confirmed via the display 130 or the like. When the subject PS is in a non-pneumoperitoneum state, the acquired shape data of the positioning jig 50C may be used. When the subject PS is in a non-pneumoperitoneum state, the acquired shape data of the positioning jig 50C may be used as shape data in which the amount of deformation and extension caused by the pneumoperitoneum is added. The port position processing unit 164 associates the position on the subject PS with the position on the positioning jig 50C by jig installation simulation.

The port position processing unit 164 performs port position simulation or port position adjustment taking into account the positioning jig 50C (S53). In this case, the port position processing unit 164 plans (for example, calculates) the port position to be drilled based on the installation position and shape of the positioning jig 50C in the port position simulation or port position adjustment taking into account the positioning jig 50C. do.

Note that when the positioning jig 50C is used, marking and perforation can be performed even if the elastic region D1 of the positioning jig 50A and the planned port position overlap. In addition, the positioning jig 50A can cover the entire abdomen of the subject PS that can be perforated. As a result, there are almost no positions where positioning by the positioning jig 50C is not possible, so there are almost no restrictions on port position simulation or port position adjustment by the positioning jig 50C.

The port position processing unit 164 outputs parameters indicating the position of each planned port position in the positioning jig 50A (S54). This parameter may be of the form "port A, 45°, +30 mm", "port assist, 120°, -45 mm", and so on. That is, the parameter may indicate the position of the port in polar form. In this case, the port identification mark Pid is "A", and the position of the port A may be identified at a position +30 mm from the reference position in the direction of 45° from the reference position (reference direction). Further, the port identification mark Pid may be "assist", and the position of the port assist may be identified at a position −45 mm from the reference position in the direction of 120° from the reference position (reference direction). That is, a value specifically indicating the position of the port PT may be output as a parameter.

In S54, the port location processing unit 164 may output the parameters to the display 130, for example. That is, the display 130 may display a value specifically indicating the position of the port PT related to the parameter. This display may be confirmed by the user. Then, the user may generate the marking portion Mp and the opening at the position of the positioning jig 50C corresponding to the position of the port PT related to the parameters. In real space, the user sets the positioning jig 50C by aligning it with the reference position and the marking portion Mhs, drills through the marking portion Mp and the opening of the positioning jig 50C, and connects the port PT to the subject PS. may be installed.

At S54, the parameters may be output (eg, sent) to, for example, a printer for printing information on the positioning jig 50C. The printer may acquire parameter information from the robotic surgery support apparatus 100 and print the marking portion Mp at the position of the positioning jig 50C corresponding to the position of the port PT related to the parameter. Then, in real space, the user positions the positioning jig 50C by aligning it with the reference position and the marking portion Mhs, performs a hole through the marking portion Mp of the positioning jig 50C, and sets the port PT in the subject PS. You can

In S54, the parameters may be output (eg, transmitted) to, for example, a drilling machine for drilling holes in the positioning jig 50C. The drilling machine may acquire parameter information from the robotic surgery support apparatus 100, drill the position of the positioning jig 50C corresponding to the position of the port PT related to the parameter, and generate an opening. The user may position the positioning jig 50C in real space by aligning it with the reference position and the marker Mhs, drill holes through the opening of the positioning jig 50C, and set the port PT in the subject PS. .

FIG. 21 is a diagram showing an example of a state in which the subject PS on which the positioning jig 50C is installed is viewed from the ventral side (negative side in the y direction). The positioning jig 50C is aligned with the subject PS using the marker Mhs as a reference and placed on the body surface. In the installed positioning jig 50C, the detailed positions of the ports A, B, C, D, E, and assist are specified by the derived parameters indicating the position of each port (for example, information in polar coordinate format). , a port position specifying portion (for example, an opening, a marking portion Mp) is provided. Then, the user can use an instrument for punching the port PT to punch the port PT (the position of the port position specifying portion) positioned by the positioning jig 50A.

Thus, according to the third parameter output example, the robotic surgery support apparatus 100 can output the parameters related to the positioning jig 50C capable of highly accurately positioning the port PT at the planned port position on the three-dimensional coordinates. . Therefore, the user can use the positioning jig 50C to position the port PT to be drilled at the port position indicated by the parameter.

Further, by outputting the parameters in the polar coordinate format, the robotic surgery support apparatus 100 can output the position and direction (angle) relative to the reference position in an intuitive and easy-to-understand manner for the user.

The positioning jig 50C of the third output example of the parameters may not be prepared in advance, but may be generated based on the output parameters (shape data) as in the second output example.

(4th parameter output example)
In the fourth parameter output example, descriptions of items similar to those in the first to third parameter output examples are omitted or simplified. The fourth parameter output example is a variation of the third parameter output example.

FIG. 22 is a diagram showing an example of the positioning jig 50D in the fourth output example of parameters.

The positioning jig 50D is made of a stretchable material. The positioning jig 50D goes around the body surface of the subject PS and may be used like a bellyband. The positioning jig 50D may be formed in a cylindrical shape and arranged so as to revolve around the subject PS, or may be formed in a plane, wound so as to revolve around the subject PS, and may be attached to a pin or the like. May be zipped. The positioning jig 50D may be translucent and may be made of a transparent material or a translucent material.

The positioning jig 50D may be prepared in advance. The positioning jig 50D may be commonly prepared in advance for each surgical procedure or regardless of the resin. The positioning jig 50D may be prepared in advance for each surgical procedure or for each surgical procedure. The shape of the positioning jig 50D may be a shape other than a rectangle.

The positioning jig 50D includes a marker part Mhs and a port identification marker Pid. Further, the positioning jig 50D is provided with a port position specifying section for specifying a planned port position based on the derived parameters. The locating fixture 50D may be drilled or marked with planned port locations. That is, the port position specifying portion may be the opening of the positioning jig 50D or the marking portion Mp. FIG. 22 illustrates the marking portion Mp. The user can easily drill holes in the planned port positions by positioning using the port position designating portion of the positioning jig 50D.

FIG. 23 is a diagram showing an example of an axial section of the abdomen of the subject PS on which the positioning jig 50D is installed. As shown in FIG. 23, the positioning jig 50D is bent along the body surface of the subject PS due to its stretchability. Also, the positioning jig 50D is arranged to circle around the body surface of the subject PS. Therefore, the port PT can be positioned not only on one side of the body such as the abdominal side, but also on the side and the back at the same time.

The operation example in the fourth output example of parameters is the same as the operation example in FIG. 20 shown in the third output example of parameters, so the explanation is omitted.

FIG. 24 is a diagram showing an example of a state in which the subject PS on which the positioning jig 50D is installed is viewed from the ventral side (negative side in the y direction). The positioning jig 50D is aligned with the subject PS using the marker Mhs as a reference and placed on the body surface. In the installed positioning jig 50D, the detailed positions of the ports A, B, C, D, E, and assist are specified by the derived parameters indicating the position of each port (for example, information in polar coordinate format). , a port position specifying portion (for example, an opening, a marking portion Mp) is provided. Then, the user can use an instrument for punching the port PT to punch the port PT (the position of the port position specifying portion) positioned by the positioning jig 50D.

Thus, according to the fourth parameter output example, the robotic surgery support apparatus 100 can output the parameters related to the positioning jig 50D capable of positioning the port PT at the planned port position on the three-dimensional coordinates with high accuracy. . Therefore, the user can use the positioning jig 50D to position the port PT to be drilled at the port position indicated by the parameter.

Further, by outputting the parameters in the polar coordinate format, the robotic surgery support apparatus 100 can output the position and direction (angle) relative to the reference position in an intuitive and easy-to-understand manner for the user.

The positioning jig 50D in the fourth output example of the parameters may not be prepared in advance, but may be generated based on the output parameters (shape data) as in the second output example.

Next, a comparative example and an output example of each parameter of this embodiment will be considered.

FIG. 25 is a classification diagram of the features of the positioning jigs of the comparative example and each output example of the present embodiment. In FIG. 25, the presence/absence of each feature of the first to fourth output examples of the parameters of Comparative Examples 1 and 2 and the present embodiment is shown in a table format. Comparative Example 1 is an example using the jig described in Non-Patent Document 2 (Leksell Stereotactic System (registered trademark)). Comparative Example 2 is an example using a jig used in general plastic surgery. As features of the jig, information such as whether it is elastic, whether it can be bent, whether it can be stretched, whether it can be cut, whether it has a scale, whether it can be reused, and whether it can be generated for each subject PS (for example, a patient) is exemplified. ing. Note that the features of the jig may have features other than these.

FIG. 25 illustrates that the positioning jig 50A of the first parameter output example has elasticity, has scale SC, and is reusable. It illustrates that the positioning jig 50B of the second output example of the parameters is bendable and can be generated for each subject PS (for example, patient). The positioning jig 50C of the third output example of parameters is elastic, bendable, stretchable, cuttable, and has a scale SC. The positioning jig 50D of the fourth output example of parameters is elastic, bendable, stretchable, cuttable, and has a scale SC.

Note that the positioning jig 50 may have the characteristics of the positioning jigs 50A to 50D in the first to fourth output examples of the parameters described above in combination. For example, the positioning jig 50C in the third parameter output example may have the scale SC.

Next, variations of this embodiment will be described.

The port position processing unit 164 may adjust the port position based on the position of the actually drilled port (pre-drilled port).

FIG. 26 is a diagram showing an example of the positional relationship between the subject PS and each device in the operating room. For example, the port position processing unit 164 may acquire a captured image captured by a camera (depth camera) included in the depth sensor 410 via the communication unit 110 . In this captured image, the subject PS, the bed BD on which the subject PS is placed, the positioning jig 50, and the trocar TC inserted into the perforated port PT may be reflected. A display 130 may display this captured image and allow monitoring by the user. Also, the port position processing unit 164 may extract feature points of the subject PS by image processing. The port position processing unit 164 may sequentially correct the port arrangement plan based on the positional relationships among the feature points of the subject PS, the bed BD, the positioning jig 50, the trocar TC, and the like. Position adjustments may be made.

In other words, the port position processing unit 164 may perform port position simulation and port position adjustment based on the positions of already drilled ports (pre-drilled ports). In this case, the position of the perforated port becomes a fixed position, and the position of the port that has not yet been perforated (unperforated port) becomes a variable position, and the position of the non-perforated port is adjustable. As a result, the robotic surgery support apparatus 100 can plan subsequent port placement based on the actually drilled pre-perforated ports.

Information on the position of the perforated port may be input via an operation unit (not shown) of the measuring instrument 400 and notified to the robotic surgery support apparatus 100 . Information on the position of the perforated port may be input via the UI 120 of the robotic surgery support device 100 .

The scale SC of the positioning jig 50 may be curved. In the positioning jig 50, the scale SC is formed in a curved shape, so that the outline of the dermatological tissue, the outline of the bone, and the like can be easily used in the preoperative simulation. For example, it is possible to position the port using the scale SC along the outline of ribs and the outline of the organ of the curved subject PS. Also, the scale SC may be provided one-dimensionally (that is, linearly). The one-dimensional arrangement of the scale SC facilitates the use of the scale SC. The scale SC may be represented in polar form (ie distance from a reference point and angle relative to a reference direction). The scale SC may be presented in xy coordinate format (ie, values of orthogonal x and y axes).

The reference position for installing the positioning jig 50 may be a position other than the navel hs. For example, the reference position may be a bone of the subject PS (for example, the ribs, the gap between the ribs, the left and right upper ends of the pelvis), and the position of the bed BD on which the subject PS is placed. The reference position may be the position of the planned or actually drilled camera port, the reference position being the position of the first drilled port (pre-drilled port position, e.g. camera port) good too.

Also, a plurality of reference positions may be provided. For example, the plurality of reference positions may include the position of the inferior xiphoid process and the umbilical hs. One or more reference positions may be positions having features that can be recognized from the body surface of the subject PS, or positions of instruments installed in the operating room and fixed to the subject PS.

The positioning jig 50 may be attached with a tape measure or a handle. A fixture may be attached to the positioning jig 50, and the positioning jig 50 may be attached to a predetermined position using the fixture.

The number of planned port locations may be the same as the number of robot arms AR, less than the number of robot arms AR, or more than the number of robot arms AR. If the number of planned port locations is greater than the number of robot arms AR, one port planned location can be kept in reserve. For example, if the position of the port to be perforated changes during surgery or if it is difficult to perform treatment via a pre-perforated port, the positioning jig 50 is used to perforate the previously prepared port position. position. For example, the port location processor 164 may plan port locations only for those ports that require precise positioning.

There may be a plurality of pneumoperitoneum procedures used in the pneumoperitoneum simulation, and may include, for example, three types of open method, closed method, and direct method.

In the open method, a first port is opened on the body surface of the subject PS, as in laparotomy. Once safe penetration of the first port is confirmed, a first trocar (Hasson Trocar) is inserted and pneumoperitoneum is initiated. The Open method is highly safe, but it takes a long time to perform the procedure and is relatively invasive, so it is performed when adhesions are expected to occur in the subject's PS.

In the Closed method, a needle called a Veress needle (pneumoperitoneum needle) is pierced into the body surface of the subject PS, the first port is perforated, gas is sent from the Veress needle, and the subject PS is pneumoperitoneum. A first trocar is later inserted into the first port. The Closed method is a blind operation, and thus has a high degree of difficulty. The Closed method is a quick procedure, but can damage organs.

In the Direct method, the body surface of the subject PS is suddenly pierced by the first trocar and pneumoperitoneum is created. The Direct method is a quick procedure, but can damage organs. It can damage organs, and the degree of damage can be greater at that time. However, unlike the Closed method, a camera is attached to the first trocar and the holes are drilled, so work is performed with a secured field of view.

Therefore, in this embodiment, there may be a port PT that is measured, positioned, marked and installed before pneumoperitoneum and a port PT that is measured, positioned, marked and installed after pneumoperitoneum. For example, in the Direct method and the Open method, a first port (usually a camera port) that is drilled first is installed in front of the pneumoperitoneum, and gas is sent through the first port for pneumoperitoneum. In the Closed method, the first port may be placed after pneumoperitoneum.

The positioning jig 50 may be made of a material such as a resin plate, cloth, vinyl, paper, or the like.

In this way, the robotic surgery support device 100 can output parameters relating to the positioning jig 50 capable of positioning the port PT at a planned port position on three-dimensional coordinates with high accuracy. Therefore, the user can position the port PT to be drilled at the port position indicated by the parameter using the positioning jig 50 having a predetermined shape according to the surgical procedure, for example. Further, the robotic surgery support apparatus 100 provides the shape data and pattern data of the positioning jig 50 indicated by the parameters to other devices (for example, a 3D printer and a laser processing machine), thereby performing positioning jigs based on the shape data and pattern data. Tool 50 can be generated.

This positioning jig 50 may be custom made for positioning at the planned port location on the three-dimensional coordinates. Therefore, by using the generated positioning jig 50, the user can perform high-precision positioning at the planned port position on the three-dimensional coordinates. In addition, the robot surgery support apparatus 100 uses the positioning jig 50 that can be moved and installed before or during surgery, instead of a jig that is fixedly installed, so that the port PT can be rapidly positioned and perforated. I can assist.

Further, according to the robotic surgery support device 100, the planned position of the port PT can be specified in a three-dimensional space. Also, at the time of drilling the port PT, the surgical assistance robot 300 is often not connected (docked) to the bed BD. can be positioned. The positioning jig 50 may be placed on the body surface of the subject PS after or before pneumoperitoneum. The positioning jig 50 can be made in a small size, for example, from a cloth-like or bendable surface material, so that it can be easily installed on the body surface of the subject PS. Further, the robotic surgery support device 100 can be used in a planning system for positioning the port PT using the positioning jig 50. FIG. Also, the positioning jig 50 can be used to confirm the position of the actual drilled port.

Further, the robotic surgery support device 100 can position the port PT using the positioning jig 50, and can position the port PT without fixing the positioning jig 50 on the basis of the bone or the like. Therefore, the user can install the positioning jig 50 in a short time.

Since the subject PS is anesthetized in the pneumoperitoneum, the pneumoperitoneum is included in the operation time. Therefore, it is difficult to install a large-scale jig on the subject PS, but since the positioning jig 50 is small, it can be installed at a desired position on the body surface of the subject PS in a short time. .

Also, when the positioning jig 50 is used for positioning the port PT in a region other than the brain, such as the abdomen, even if the positioning accuracy is somewhat low, it can be tolerated. In this case, by using a simple positioning jig 50, the time required for the installation of the positioning jig 50 and the positioning using the positioning jig 50 can be further shortened.

Even if the port position moves from the port position before the pneumoperitoneum due to the pneumoperitoneum, the user can position the port PT by setting the positioning jig 50 in a short time after the pneumoperitoneum. Moreover, at least a part of the positioning jig 50 can be deformed according to the pneumoperitoneum condition, and the positioning jig 50 can be installed according to the body shape of the subject PS. Depending on the port PT, some are installed before the pneumoperitoneum and some are installed after the pneumoperitoneum. The user can easily install the positioning jig 50 and position the port PT both before and after the pneumoperitoneum. .

The locating jig 50 can be removed after locating, marking and drilling the ports PT. The positioning jig 50 can be kept clean, for example, by being disposable.

The positioning jig 50 may be generated exclusively for the subject PS (for each subject PS) or may be generated commonly for each subject PS. When the positioning jig 50 is generated for each subject PS, it is possible to use a specific positioning jig 50 according to the body shape and surgical procedure of the subject PS. When the positioning jig 50 is generated commonly for each subject PS, one positioning jig 50 can be used many times, improving the efficiency of use.

The positioning jig 50 may be a disposable jig or a reusable jig. If the positioning jig 50 is a disposable jig, the cleanliness of the positioning jig 50 can be easily maintained. If the positioning jig 50 is a jig that can be used repeatedly, one positioning jig 50 can be used many times, improving the efficiency of use.

Various embodiments have been described above with reference to the drawings, but it goes without saying that the present disclosure is not limited to such examples. It is obvious that a person skilled in the art can conceive of various modifications or modifications within the scope described in the claims, and these also belong to the technical scope of the present disclosure. Understood.

In the first embodiment, the volume data as the captured CT image is transmitted from the CT apparatus 200 to the robotic surgery support apparatus 100 as an example. Alternatively, the volume data may be transmitted to a server or the like on the network and stored in the server or the like so as to be temporarily accumulated. In this case, the communication unit 110 of the robotic surgery support device 100 may acquire volume data from a server or the like via a wired line or wireless line, or via an arbitrary storage medium (not shown). You may

In the first embodiment, volume data as a captured CT image is transmitted from the CT apparatus 200 to the robotic surgery support apparatus 100 via the communication unit 110 as an example. This includes the case where the CT apparatus 200 and the robotic surgery support apparatus 100 are substantially combined into one product. It also includes the case where the robotic surgery support apparatus 100 is treated as the console of the CT apparatus 200 .

In the first embodiment, an image is captured by the CT device 200 and volume data including information on the inside of the living body is generated, but an image may be captured by another device to generate volume data. . Other devices include Magnetic Resonance Imaging (MRI) devices, Positron Emission Tomography (PET) devices, Angiography devices, or other modality devices. Also, the PET device may be used in combination with other modality devices.

In the first embodiment, the surgical assistance robot 300 is connected to the robotic surgical assistance device 100, but it does not have to be connected. This is because it is sufficient if information on the kinematics of the surgical assistance robot 300 is acquired in advance. Alternatively, the surgery support robot 300 may be connected after the port is perforated. Also, it may be connected to only some of the devices constituting the surgical assistance robot 300 . Also, the robotic surgery support device 100 itself may be a part of the surgery support robot 300 .

In the first embodiment, the surgery support robot 300 is intended for minimally invasive surgery. can be Also, the surgery support robot 300 may be a surgery support robot that supports endoscopic surgery.

In the first embodiment, the robotic surgery support apparatus 100 plans the port positions based on the volume data of the virtual pneumoperitoneum state of the subject PS, but the present invention is not limited to this. This is because, for example, the observation target may undergo robotic surgery without pneumoperitoneum in the respiratory organ or neck. In other words, the robotic surgery support apparatus 100 may plan port positions based on the volume data of the non-pneumoperitoneum state.

In the first embodiment, the human body was exemplified as the subject PS, but the body of an animal may also be used.

According to the present disclosure, a program that realizes the functions of the robotic surgery assistance device of the first embodiment is supplied to the robotic surgery assistance device via a network or various storage media, and read and executed by a computer in the robotic surgery assistance device. Programs are also covered.

As described above, the robotic surgery support device 100 of the embodiment supports minimally invasive robotic surgery by the surgery support robot 300 . The processing unit 160 may acquire 3D data of the subject PS (for example, volume data in a non-pneumoperitoneum state or a virtual pneumoperitoneum state). The processing unit 160 may derive planned positions of a plurality of ports PT to be drilled in the body surface of the subject PS in the 3D data. Based on the planned positions of the plurality of ports PT and the positions of the positioning jig 50 for positioning the plurality of ports PT at the planned positions of the plurality of ports PT, the processing unit 160 determines the position of the positioning jig 50 with respect to the subject PS. may derive the planned positions of multiple ports PT in . The processing unit 160 may output port position specifying information that specifies planned positions of the plurality of ports PT when the positioning jig 50 is installed along the body surface of the subject P. FIG.

As a result, the robotic surgery support apparatus 100 uses the positioning jig 50 as a device (jig) for positioning the port PT that can be easily handled, and designates the three-dimensional position of the port PT in the subject PS. We can make it possible. In addition, unlike the jig described in Non-Patent Document 2, which is fixedly installed for the head, the positioning jig 50 has high mobility, so that various postures of the subject PS during surgery can be accommodated. can respond flexibly to Further, even when the position and size of the body surface of the subject PS change due to pneumoperitoneum or the like, the positioning jig 50 can flexibly respond. Further, by being installed along the body surface of the subject, the positioning jig 50 follows the respiration and body movement of the subject PS.

Moreover, the shape data of the positioning jig 50 may be included in the port position specifying information.

Thereby, the robotic surgery support apparatus 100 can output (for example, transmit or output to an external storage medium) the shape data of the positioning jig 50 and provide it to, for example, an external device. An external device (for example, a 3D printer, a laser processing machine) can acquire the shape data and generate the positioning jig 50 based on the shape data. This shape data is shape data such that, for example, the port position specifying unit of the positioning jig 50 specifies the planned positions of the plurality of ports PT. Therefore, the three-dimensional planned positions of the plurality of ports PT can be designated by the shape of the positioning jig 50 itself.

The port position specifying information may include pattern data of the positioning jig 50 .

Thereby, the robotic surgery support apparatus 100 can output (for example, transmit or output to an external storage medium) the pattern data of the positioning jig 50 and provide it to an external device, for example. Pattern data may include patterns including graphics, symbols, characters, and the like. Pattern data may include color information. An external device (for example, a 3D printer, a laser processing machine) can acquire the pattern data and generate the positioning jig 50 based on the pattern data. This pattern data is, for example, a port position specifying portion (for example, a marking portion Mp) of the positioning jig 50, and is pattern data specifying planned positions of a plurality of ports PT. Therefore, three-dimensional planned positions of a plurality of ports PT can be designated by the pattern of the positioning jig 50 .

Also, the positioning jig 50 may have a scale SC for positioning the planned positions of the plurality of ports PT on the subject PS in the real space. The port position specifying information may be information indicating the planned position of the positioning jig 50 on the scale SC.

Thereby, the robotic surgery support apparatus 100 can provide information on the position (for example, value) on the scale SC corresponding to the derived planned position. Thus, the user can use the scale SC to measure the position of the provided scale SC values to locate the planned port location.

The positioning jig 50 may also have an indicator (for example, a port position indicator) for indicating the planned positions of the plurality of ports PT in the subject PS in the real space. The port position specifying information may be information indicating the planned position indicated by the indicating portion of the positioning jig 50 .

Thereby, the robotic surgery support apparatus 100 can provide information on the indicator (for example, projection, notch, mark) corresponding to the derived planned position. The positioning jig 50 can indicate the derived planned position by means of the indicating section. Therefore, the user can easily recognize the planned position of the port PT by checking the indication part of the positioning jig 50 .

Also, the positioning jig 50 may have two or more scales SC.

Thereby, the robotic surgery support device 100 can support positioning of a plurality of ports PT using two or more scales SC of the positioning jig 50 . This allows the user to use independent optimal scale SCs for two or more ports PT. This allows the user to receive simple parameters for two or more ports PT and independent scales SC. Also, the robotic surgery support device 100 can provide port location information for each of the two or more ports PT.

Moreover, the positioning jig 50 may have stretchability.

As a result, the robotic surgery support apparatus 100 uses the positioning jig 50 that expands and contracts according to the expansion and contraction of the subject PS, even when the part of the body surface on which the positioning jig 50 is arranged expands and contracts in the subject PS. , can assist in locating the three-dimensional position of the port in the subject PS. Further, the expansion and contraction of the positioning jig 50 allows the positioning jig 50 to follow the respiration and body movement of the subject.

In addition, the positioning jig 50 may have a stretchable region D1 having stretchability and a non-stretchable region D2 formed at both ends of the stretchable region D1 and having less stretchability than the stretchable region D1. .

Thereby, the robotic surgery support apparatus 100 can easily adjust the shape of the positioning jig 50 along the body surface by using the non-stretchable region D2, and easily fix the position of the positioning jig 50 with respect to the body surface at both ends. can.

Also, the positioning jig 50 may have elasticity.

Thereby, the robotic surgery support apparatus 100 bends the positioning jig 50 along the body surface on which the positioning jig 50 is arranged in the subject PS to position the three-dimensional position of the port PT in the subject PS. I can assist. For example, even if the body surface near the planned port position is uneven, the positioning jig 50 can be installed along the unevenness of the body surface, and accurate measurement of the port position can be achieved. .

Also, the positioning jig 50 may be formed of a bendable plate-like member.

Thereby, the user can use the positioning jig 50 along the body surface of the subject PS while bending it.

The positioning jig 50 may be formed by connecting a plurality of rigid members so as to be freely bendable.

Thereby, the user can secure the strength of the positioning jig 50 and bend the positioning jig 50 so as to follow the body surface of the subject PS.

The positioning jig 50 may have translucency.

As a result, the robotic surgery support apparatus 100 allows the user to check the body surface with the positioning jig 50 placed on the body surface, and to determine the positions of the marking portions Mp and the openings on the positioning jig 50. While confirming, it can assist in easily drilling the port PT.

The 3D data may be 3D data of a virtual pneumoperitoneum state generated by performing a pneumoperitoneum simulation on volume data of the subject PS.

Thereby, the robotic surgery support apparatus 100 can support specification of the three-dimensional position of the port PT in the subject PS using the positioning jig 50 as a device with high operability even when the subject PS is in a pneumoperitoneum state. .

INDUSTRIAL APPLICABILITY The present disclosure is useful for a robotic surgery assisting apparatus, a robotic surgery assisting method, a program, etc. that enable specification of the three-dimensional position of a port in a subject using a device with high operability.

50, 50A, 50B, 50C, 50D Positioning jig 51 Projection 51A Port position indicator 51B Projection 100 Robotic surgery support device 110 Communication unit 120 User interface (UI)
130 display 140 processor 150 memory 160 processing unit 161 area extraction unit 162 image generation unit 163 deformation simulation unit 164 port position processing unit 166 display control unit 167 projection control unit 170 projection unit 200 CT device 300 surgery support robot 400 measuring instrument BD bed EF End effector hs Navel LM Landmark Mhs Marking part Mp Marking part Pid Port identification mark PS Subject PT Port PTA Auxiliary port PTC Camera port PTE End effector port SC Scale TC Trocar tp Tape WA1 Individual working area WA2 Whole working area

Claims (13)

A robotic surgery support device that supports minimally invasive robotic surgery by a surgery support robot,
a processing unit,
The processing unit is
Acquiring 3D data of the subject including pixel values of three-dimensional pixels obtained by imaging the subject;
A jig related to the positioning jig including the shape of the positioning jig for positioning the plurality of ports at the planned positions of the plurality of ports for inserting an endoscope camera or an end effector drilled into the body surface of the subject. get information,
Acquiring information of a reference position on the subject for aligning the subject and the positioning jig ;
In a virtual space, the positioning jig is installed in accordance with the reference position;
deriving planned positions of the plurality of ports on the body surface of the subject by performing a simulation for determining the positions of the plurality of ports perforated on the body surface of the subject in the 3D data;
outputting information indicating the planned positions of the plurality of ports on the positioning jig based on the jig information and the planned positions of the plurality of ports on the plurality of body surfaces ;
Robotic surgery support device. the positioning jig has a scale for positioning planned positions of the plurality of ports on the body surface of the subject in the subject in real space;
The processing unit outputs information indicating a planned position of the positioning jig on the scale.
The robotic surgery support device according to claim 1 . The positioning jig has openings or markings for designating planned positions of the plurality of ports on the body surface of the subject in the subject in real space,
The processing unit outputs information indicating a planned position specified by the opening of the positioning jig or the marking unit.
The robotic surgery support device according to claim 1. The positioning jig has two or more scales,
The robotic surgery support device according to claim 2 . The positioning jig has stretchability,
The robotic surgery support device according to any one of claims 1 to 4 . The positioning jig is
a stretchable first region;
A second region formed at both ends of the first region and having a lower stretchability than the first region,
The robotic surgery support device according to claim 5 . The positioning jig has elasticity,
The robotic surgery support device according to any one of claims 1 to 6 . The positioning jig is formed of a bendable plate-shaped member,
The robotic surgery support device according to any one of claims 1 to 4 . The positioning jig is formed by connecting a plurality of rigid members so as to be freely bendable.
The robotic surgery support device according to any one of claims 1 to 4 . The positioning jig has translucency,
The robotic surgery support device according to any one of claims 1 to 8 . The 3D data is 3D data of a virtual pneumoperitoneum state generated by performing a pneumoperitoneum simulation on the volume data of the subject.
The robotic surgery support device according to any one of claims 1 to 10 . An information output method in a robotic surgery support device for supporting minimally invasive robotic surgery by a surgery support robot,
Acquiring 3D data of the subject including pixel values of three-dimensional pixels obtained by imaging the subject;
A jig related to the positioning jig including the shape of the positioning jig for positioning the plurality of ports at the planned positions of the plurality of ports for inserting the endoscope camera or end effector perforated on the body surface of the subject. get information,
Acquiring information of a reference position on the subject for aligning the subject and the positioning jig ;
In a virtual space, the positioning jig is installed in accordance with the reference position;
deriving planned positions of the plurality of ports on the body surface of the subject by performing a simulation for determining the positions of the plurality of ports perforated on the body surface of the subject in the 3D data;
outputting information indicating the planned positions of the plurality of ports on the positioning jig based on the jig information and the planned positions of the plurality of ports on the body surface of the subject ;
Information output method. A program for causing a computer to execute the information output method according to claim 12 .

Images