JP2021045341A - Arthroscopic surgery support device, arthroscopic surgery support method, and program - Google Patents

Abstract

To provide an arthroscopic surgery support device having a simplified system configuration to make it possible to simply perform surgical navigation.SOLUTION: An arthroscopic surgery support device comprises a processing unit for acquiring 3D data on an analyte; setting a virtual port on the analyte's body surface in the 3D data; acquiring an insertion distance of a surgical instrument inserted into the inside of the analyte from an actual port corresponding to the virtual port; and, on the basis of at least a position of the virtual port and the insertion distance of the surgical instrument, superposing information showing the surgical instrument on a first image obtained by visualizing the 3D data before being displayed on a display unit.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to a arthroscopic surgery support device, a arthroscopic surgery support method, and a program.

Conventionally, a method for controlling the position and orientation of a camera in a laparoscopic surgical navigation system has been studied (see Non-Patent Document 1). In the system of Non-Patent Document 1, a pseudo camera in which a CCD camera is attached to the tip of forceps is used as a laparoscopic camera. By measuring the markers on each surface of the regular hexagonal pedestal attached to the pseudo camera with an external camera, the position and orientation of the tip of the pseudo camera are calculated on the PC.

Masatomo Fumiyama, 5 outsiders, "Examination of camera position / orientation control method in laparoscopic surgery navigation system", [online], Information and System Society Special Program Student Poster Session Proceedings, 2018, IEICE (Electronic Information and Communication) ISS-P-034, [Search on August 9, 1991], Internet <URL: https://www.ieice.org/~iss/jpn/Publications/issposter_2018/data/pdf/ISS- P-034.pdf ＞

When the system of Non-Patent Document 1 is used for surgical navigation, it is installed on the ceiling, for example, to image the state during surgery in addition to the laparoscopic camera in order to derive the position of the laparoscopic camera corresponding to the pseudo camera. Requires an external camera. Therefore, the system for performing surgical navigation becomes large-scale. Further, since the external camera is installed on the ceiling of the operating room, for example, a coordinate system based on the operating room (also referred to as an operating room coordinate system) is used. On the other hand, since the laparoscopic camera is operated in the patient's body based on the port installed in the patient, the coordinate system based on the patient (patient coordinate system) is used. Therefore, when referring to an image using the operating room coordinate system and an image using the patient coordinate system in the intraoperative navigation, mutual alignment is required. Therefore, in surgical navigation, a step of registering coordinates in advance is required. In addition, re-registration is required during the operation according to the movement of the patient.

The present disclosure provides a arthroscopic surgery support device, a arthroscopic surgery support method, and a program that can simplify the system configuration and easily perform surgical navigation in view of the above circumstances.

One aspect of the present disclosure is a arthroscopic surgery support device that supports arthroscopic surgery, comprising a processing unit, the processing unit acquires 3D data of a subject, and the subject in the 3D data. A virtual port is set on the body surface of the body surface, the insertion distance of the surgical instrument inserted inside the subject is obtained from the real port corresponding to the virtual port, and at least the position of the virtual port and the insertion distance of the surgical instrument are obtained. Based on the above, it is a spectroscopic surgery support device having a function of superimposing information indicating the surgical instrument on the first image that visualizes the 3D data and displaying it on the display unit.

According to the present disclosure, the system configuration can be simplified and surgical navigation can be easily performed.

A block diagram showing a hardware configuration example of the medical image processing apparatus according to the first embodiment. Block diagram showing a functional configuration example of a medical image processing device The figure which shows an example of an organ including a target, a surgical instrument used for treatment with respect to a target, a positional relationship of a port into which a surgical instrument is inserted, and a subject coordinate system. The figure which shows an example of the port coordinate system with respect to a port The figure which shows an example of the positional relationship between the tip position and the tip direction of the endoscope, the imaging range, and the forceps. A diagram showing a display example in which a volume rendered image and various information are superimposed and displayed. A diagram showing a display example in which a virtual endoscopic image and various information are superimposed and displayed. Flowchart showing an operation example of a medical image processing device Flowchart showing an example of port position simulation procedure Flow chart showing an operation example when calculating the port position score by the medical image processing device Diagram showing an example of a working area defined based on the virtual port position

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration example of the medical image processing apparatus 100 according to the first embodiment. The medical image processing device 100 includes an acquisition unit 110, a UI 120, a display 130, a processor 140, and a memory 150. The medical image processing apparatus 100 supports arthroscopic surgery (including robotic surgery) by image processing. The medical image processing device 100 performs surgical navigation. Surgical navigation includes, for example, preoperative simulation for preoperative planning (preoperative planning) and intraoperative navigation for intraoperative support.

A CT device 200 is connected to the medical image processing device 100. The medical image processing device 100 acquires volume data from the CT device 200 and processes the acquired volume data. The medical image processing device 100 may be composed of a PC and software mounted on the PC.

The CT device 200 irradiates a subject with X-rays and takes an image (CT image) by utilizing the difference in absorption of X-rays by tissues in the body. The subject may include a living body, a human body, an animal, and the like. The CT apparatus 200 generates volume data including information on an arbitrary location inside the subject. The CT device 200 transmits volume data as a CT image to the medical image processing device 100 via a wired line or a wireless line. For the imaging of CT images, imaging conditions related to CT imaging and contrast conditions related to administration of a contrast medium may be taken into consideration.

Information from the surgical instrument 300 is input to the medical image processing device 100. The surgical instrument 300 is an instrument used in surgery and includes an endoscope (camera), forceps, and the like. The medical image processing apparatus 100 acquires, for example, insertion distance information, orientation information, position information, and a camera image captured by the surgical instrument 300 from the surgical instrument 300, which will be described later, and performs processing.

The acquisition unit 110 in the medical image processing device 100 includes, for example, a communication port, an external device connection port, and a connection port to an embedded device. The acquisition unit 110 acquires the volume data acquired by the CT device 200. The acquired volume data may be immediately sent to the processor 140 for various processing, or may be stored in the memory 150 and then sent to the processor 140 for various processing when necessary. Further, the volume data may be acquired via a recording medium or a recording medium. Further, the volume data may be acquired in the form of intermediate data, compressed data or synogram. Further, the volume data may be acquired from the information from the sensor device attached to the medical image processing device 100. As described above, the acquisition unit 110 has a function of acquiring various data such as volume data.

The UI 120 may include, for example, a touch panel, a pointing device, a keyboard, or a microphone. The UI 120 accepts an arbitrary input operation from the user of the medical image processing device 100. The user may include a surgeon, a doctor, a nurse, a radiologist, a student, and the like.

The UI 120 accepts various operations. For example, it accepts operations such as designation of a region of interest (ROI) and setting of luminance conditions in volume data or an image based on volume data (for example, a three-dimensional image or a two-dimensional image described later). Regions of interest may include regions of various tissues (eg, blood vessels, bronchi, organs, organs, bones, brain). Tissue may include lesioned tissue, normal tissue, tumor tissue, and the like.

The display 130 may include, for example, an LCD and displays various information. The various information may include a three-dimensional image or a two-dimensional image obtained from the volume data. The three-dimensional image may include a volume rendered image, a surface rendered image, a virtual endoscopic image, a virtual ultrasonic image, a CPR image, and the like. The volume rendered image may include a RaySum image, a MIP image, a MinIP image, an average value image, a raycast image, and the like. The two-dimensional image may include an axial image, a sagittal image, a coronal image, an MPR image, and the like.

The memory 150 includes a primary storage device of various ROMs and RAMs. The memory 150 may include a secondary storage device such as an HDD or SSD. The memory 150 may include a USB memory or a tertiary storage device of an SD card. The memory 150 stores various information and programs. The various information may include volume data acquired by the acquisition unit 110, an image generated by the processor 140, setting information set by the processor 140, and various programs. The memory 150 is an example of a non-transient recording medium on which a program is recorded.

Processor 140 may include a CPU, DSP, or GPU. The processor 140 functions as a processing unit 160 that performs various processes and controls by executing a medical image processing program stored in the memory 150.

FIG. 2 is a block diagram showing a functional configuration example of the processing unit 160. The processing unit 160 includes an area processing unit 161, a deformation processing unit 162, a model setting unit 163, a port information processing unit 164, a surgical instrument information processing unit 165, an image generation unit 166, and a display control unit 167. Each unit included in the processing unit 160 may be realized as a different function by one hardware, or may be realized as a different function by a plurality of hardware. Further, each part included in the processing part 160 may be realized by a dedicated hardware component.

The area processing unit 161 acquires the volume data of the subject via, for example, the acquisition unit 110. The area processing unit 161 extracts an arbitrary area included in the volume data. The area processing unit 161 may automatically specify the area of interest and extract the area of interest based on, for example, the pixel value of the volume data. The area processing unit 161 may manually specify the area of interest and extract the area of interest via, for example, the UI 120. Areas of interest may include areas such as organs, bones, blood vessels, affected areas (eg, lesioned tissue or tumor tissue), and the like.

Further, the region of interest may be segmented and extracted including not only a single tissue but also the tissues surrounding the tissue. For example, when the organ of interest is the liver, not only the liver itself, but also blood vessels that connect to the liver or run in or around the liver (eg, hepatic arteries, hepatic veins, portal veins), and bones around the liver (eg, hepatic arteries, hepatic veins, portal veins). For example, the liver, ribs) may be included. Further, the above-mentioned liver body, blood vessels in or around the liver, and bone around the liver may be obtained by segmentation as separate tissues.

The model setting unit 163 sets the model of the organization. The model may be set based on the region of interest and volume data. The model expresses the organization represented by the volume data in a simpler manner than the volume data. Therefore, the amount of data in the model is smaller than the amount of data in the volume data corresponding to the model. The model is the target of deformation processing and deformation operation that imitates various procedures in surgery, for example. The model may be, for example, a bone deformation model. In this case, the model assumes a skeleton in a simple finite element and moves the vertices of the finite element to deform the bone. Tissue deformation can be expressed by following the bone deformation. The model may include an organ model that mimics an organ (eg, liver). The model may have a shape similar to a simple polygon (eg, a triangle) or any other shape. The model may be, for example, a contour line of volume data indicating an organ. The model may be a three-dimensional model or a two-dimensional model. The bone may be expressed not by the deformation of the model but by the deformation of the volume data. This is because bone has a small degree of freedom of deformation and can be expressed by affine transformation of volume data.

The model setting unit 163 may acquire the model by generating the model based on the volume data. Further, a plurality of model templates are predetermined and may be stored in the memory 150 or an external server. The model setting unit 163 may acquire the model by acquiring the template of one model from the templates of a plurality of models prepared in advance from the memory 150 or the external server according to the volume data.

The model setting unit 163 sets the position of the target in the tissue of the subject included in the volume data. Alternatively, the model setting unit 163 sets the position of the target in the model imitating the organization. The target is set within any tissue (eg, an organ). The model setting unit 163 may specify the target position via the U120. Further, the position of the target (for example, the affected part) treated with respect to the subject in the past may be held in the memory 150. The model setting unit 163 may acquire the target position from the memory 150 and set it. The model setting unit 163 may set the target position according to the surgical technique. The surgical method indicates the method of surgery on the subject. The target position may include the position of the area of the target.

The port information processing unit 164 sets the position of the port provided on the body surface of the subject in order to insert the surgical instrument 300 into the subject.

The port provided (planned) on the body surface of the subject in the surgical plan is also referred to as a virtual port. The port that is actually installed (perforated) on the body surface of the subject corresponding to the virtual port is also called a real port. There may be some error between the position of the virtual port and the position of the real port that is actually drilled against this virtual port. Further, the port includes a camera port into which an endoscope is inserted, a forceps port into which forceps are inserted, and the like. For example, a combination of port types may be described such that a virtual port as a camera port and a virtual camera port and a real port as a forceps port are used as a real forceps port.

The port information processing unit 164 determines and sets the position of the virtual port on the body surface of the subject in the 3D data based on the volume data. The port information processing unit 164 may determine the position of the virtual port based on port position simulation, port position score calculation, port position adjustment, and the like, which will be described later. The details of the determination example of this virtual port will be described later. In addition, the position of the virtual port may be a standard port position determined according to the surgical technique. The port information processing unit 164 may acquire standard port position information from the memory 150, or may acquire it from an external device via the acquisition unit 110.

The 3D data may be, for example, volume data of a subject in a non-pneumoperitoneum state described later, or volume data of a subject subjected to a pneumoperitoneum simulation in a virtual pneumoperitoneum state described later. Further, the 3D data may be surface data generated from the volume data. Further, the 3D data may include the above-mentioned model. In addition, the 3D data may include segmentation information of organs involved in surgery.

The deformation processing unit 162 performs processing related to deformation in the subject to be operated on. For example, as a process related to deformation, a pneumoperitoneum simulation may be performed on a virtual subject. The specific method of the pneumoperitoneum simulation may be a known method, for example, the method described in Reference Non-Patent Document 1. That is, the deformation processing unit 162 may perform a pneumoperitoneum simulation based on the volume data in the non-pneumoperitoneum state and generate volume data in the virtual pneumoperitoneum state. By the pneumoperitoneum simulation, the user can virtually observe the pneumoperitoneum state by assuming that the subject is pneumoperitoneum without actually being pneumoperitoneum. Of the pneumoperitoneum states, the pneumoperitoneum state estimated by the pneumoperitoneum simulation may be referred to as a virtual pneumoperitoneum state, and the actual pneumoperitoneum state may be referred to as a real pneumoperitoneum state.

(Reference Non-Patent Document 1) Takayuki Kitasaka, Kensaku Mori, Yuichiro Hayashi, Yasuhito Suenaga, Makoto Hashizume, and Jun-ichiro Toriwaki, “Virtual Pneumoperitoneum for Generating Virtual Laparoscopic Views Based on Volumetric Deformation”, MICCAI (Medical Image Computing and Computer- Assisted Intervention), 2004, P559-P567

The pneumoperitoneum simulation may be a large deformation simulation using the finite element method. In this case, the deformation processing unit 162 may segment the body surface containing the subcutaneous fat of the subject and the abdominal internal organs of the subject via the region processing unit 161. Then, the deformation processing unit 162 may model the body surface into two layers of finite elements of skin and body fat and the abdominal internal organs into finite elements via the model setting unit 163. The deformation processing unit 162 may optionally segment the lungs and bones, for example, and add them to the model. In addition, a gas region may be provided between the body surface and the abdominal internal organs, and the gas region (pneumoperitoneum space) may be expanded (expanded) in response to virtual gas injection. The pneumoperitoneum simulation does not have to be performed.

In addition, tissues such as organs in a subject can be subjected to various deformation operations by a user, imitating various procedures of an operator in surgery. The deformation operation may include an operation of lifting an organ, an operation of turning over, an operation of cutting, and the like. Correspondingly, the deformation processing unit 162 may deform the model corresponding to the tissue such as an organ in the subject as the processing related to the deformation. For example, an organ can be pulled, pushed, or cut by forceps (eg, grasping forceps, peeling forceps, electrocautery), which may be simulated by modifying the model. As the model deforms, so does the target in the model.

The deformation by the deformation operation is performed on the model, and a large deformation simulation using the finite element method may be used. For example, the movement of organs due to repositioning may be simulated. In this case, the elastic force applied to the contact point of the organ or the lesion, the rigidity of the organ or the lesion, and other physical characteristics may be added. The amount of calculation is reduced in the transformation processing for the model as compared with the transformation processing for the volume data. This is because the number of elements in the deformation simulation is reduced. It should be noted that the transformation processing may be performed directly on the volume data without performing the transformation processing on the model.

The surgical instrument information processing unit 165 acquires the operation information of the surgical instrument 300 input via the UI 120. The operation information of the surgical instrument 300 may include information such as the type of operation (for example, movement, rotation), operation position, operation speed, and the like. The surgical instrument information processing unit 165 may directly acquire information (for example, operation information, other information) from the surgical instrument 300 from the surgical instrument 300 or other devices (for example, various sensors), or the acquisition unit 110. May be obtained via.

The surgical instrument information processing unit 165 acquires at least a part of information (surgical instrument information) regarding the surgical instrument 300. The surgical instrument information may include information about the endoscope 20 and information about the forceps 30. Information about the endoscope 20 may be acquired from an endoscope device including the endoscope 20 via, for example, the acquisition unit 110. Information about the forceps 30 may be obtained from, for example, the memory 150 or an external device. The endoscope 20 is inserted from the actual camera port to take an image of the inside of the subject. The forceps 30 are inserted from the actual forceps port and used for various treatments in the subject.

The surgical instrument information may include the orientation of the surgical instrument 300 (that is, the extending direction of the surgical instrument 300, the insertion direction of the surgical instrument 300) and the insertion distance of the surgical instrument 300. The orientation of the surgical instrument 300 here may be the main orientation of the entire surgical instrument 300. The orientation of the surgical instrument 300 includes the direction of inclination of the surgical instrument 300 with respect to the direction of gravity, and may include a clockwise or counterclockwise inclination about the extending direction of the surgical instrument 300, that is, a twisting direction. The insertion distance corresponds to, for example, the distance between the actual port into which the surgical instrument 300 is inserted and the tip position of the surgical instrument 300. For example, the surgical instrument 300 may be provided with a scale indicating the insertion distance of the surgical instrument 300. The surgical instrument information processing unit 165 may electronically read this scale to obtain the insertion distance of the surgical instrument 300. In this case, for example, a real encoder (reading device) may be attached to the trocar, and the surgical instrument 300 may be provided with an encoding marker. Further, the surgical instrument information processing unit 165 may acquire the insertion distance of the surgical instrument 300 by reading this scale and inputting the insertion distance via the UI 120.

Further, the surgical instrument information processing unit 165 may acquire information on the angle (that is, the direction) of the surgical instrument 300 from the angle sensor. The angle sensor may be attached to, for example, a surgical instrument 300 or a trocar. The angle of the trocar corresponds to the angle of the surgical instrument 300. The value detected by the angle sensor will be described later. The surgical instrument information processing unit 165 derives the orientation of the surgical instrument 300 based on the detection value of the angle sensor.

Further, the surgical instrument information processing unit 165 may calculate the orientation of the surgical instrument 300. For example, the orientation of the surgical instrument 300 may be calculated as a predetermined orientation. The predetermined orientation may be, for example, the orientation of approaching the affected area from a standardly conceivable port depending on the surgical procedure. Further, the predetermined orientation may be a orientation set by the user. Further, the orientation of the surgical instrument 300 may be calculated as a direction from the virtual port acquired from the port information processing unit 164 toward the target acquired from the model setting unit 163. Further, the orientation of the surgical instrument 300 may be calculated by following the change in the position of the target acquired from the model setting unit 163. Further, the orientation of the surgical instrument 300 may be calculated as a change of a predetermined angle with respect to the direction toward the target acquired from the model setting unit 163.

The surgical instrument information may include the orientation (extending direction) of the endoscope 20 and the insertion distance of the endoscope 20. The surgical instrument information may include the orientation of the forceps 30 and the insertion distance of the forceps 30.

The surgical instrument information processing unit 165 derives the tip position (position of the tip) and the tip direction (direction of the tip) of the surgical instrument 300. For example, the surgical instrument information processing unit 165 may calculate the tip position of the endoscope 20 based on at least the position of the virtual camera port and the insertion distance of the endoscope 20. Further, for example, when the endoscope 20 has a shape that extends linearly, the tip position of the endoscope 20 is determined based on the position of the virtual camera port, the insertion distance of the endoscope 20, and the orientation of the endoscope 20. It may be calculated. Further, for example, when the endoscope 20 has a non-linear shape (for example, a bent shape or a curved shape), it is based on the position of the virtual camera port, the insertion distance of the endoscope 20, and the shape of the endoscope 20. , The tip position of the endoscope 20 may be calculated. Information on the shape of the endoscope 20 may be stored in the memory 150, for example, or may be acquired from an external device via the acquisition unit 110. For example, when the endoscope 20 has a shape that extends linearly (for example, a rigid endoscope), the direction of the entire endoscope 20 may be calculated as the direction of the tip of the endoscope 20. For example, when the endoscope 20 has a non-linear shape, the tip orientation of the endoscope 20 may be calculated based on the overall orientation of the endoscope 20 and the shape of the endoscope 20.

Further, the surgical instrument information processing unit 165 may calculate the tip position of the forceps 30 based on at least the position of the virtual forceps port and the insertion distance of the forceps 30. Further, for example, when the forceps 30 has a shape extending linearly, the tip position of the forceps 30 may be calculated based on the position of the virtual forceps port, the insertion distance of the forceps 30, and the direction of the forceps 30. Further, for example, when the forceps 30 has a non-linear shape (for example, a bent shape or a curved shape), the tip position of the forceps 30 is based on the position of the virtual forceps port, the insertion distance of the forceps 30, and the shape of the forceps 30. May be calculated. The information on the shape of the forceps 30 may be stored in the memory 150, for example, or may be acquired from an external device via the acquisition unit 110. For example, when the forceps 30 has a shape that extends linearly, the direction of the entire forceps 30 may be calculated as the direction of the tip of the forceps 30. For example, when the forceps 30 has a non-linear shape, the tip orientation of the forceps 30 may be calculated based on the overall orientation of the forceps 30 and the shape of the forceps 30. For example, when the forceps 30 has a non-linear shape, the tip orientation of the forceps 30 may be calculated based on the overall orientation of the forceps 30 and the shape of the forceps 30.

The image generation unit 166 generates various images. The image generation unit 166 generates a three-dimensional image or a two-dimensional image based on at least a part of the acquired volume data (for example, a region extracted in the volume data). The image generation unit 166 may generate a three-dimensional image or a two-dimensional image based on the volume data deformed by the deformation processing unit 162 (for example, the volume data in the virtual pneumoperitoneum state). For example, a volume-rendered image or a virtual endoscope image representing a state in which the direction of the endoscope 20 is viewed from the position of the endoscope 20 may be generated.

The display control unit 167 displays various data, information, and images on the display 130. The display control unit 167 displays an image (for example, a rendered image) generated by the image generation unit 166. Further, the display control unit 167 superimposes and displays various information together with this image. The superimposed information may include information such as the distance from the tip of the surgical instrument 300 to the target, the insertion distance of the surgical instrument 300, and the like. Further, the display control unit 167 may adjust the brightness of the rendered image. The brightness adjustment may include, for example, adjustment of at least one of a window width (WW: Window Width) and a window level (WL: Window Level).

FIG. 3 is a diagram showing an example of the positional relationship between the organ including the target 40, the surgical instrument 300 used for treating the target 40, and the port PT into which the surgical instrument 300 is inserted. In FIG. 3, the surgical instrument 300 includes an endoscope 20 and forceps 30. A virtual port and a real port are assumed as port PTs. The real port of port PT is a port that is perforated with respect to the virtual port of port PT. Includes port PT, camera port PT1 and forceps port PT2. In FIG. 3, the subject is pneumoperitoneum. In FIG. 3, the organ is the liver 10. The liver 10 includes a target 40 to be operated on. In FIG. 3, bones 15 around the liver 10 (eg, spine, ribs) are also shown.

A virtual port of camera port PT1 is set on the body surface 70 of the subject in the virtual pneumoperitoneum state. A real port corresponding to the virtual port of the camera port PT1 is installed on the pneumoperitoneum body surface 70. A trocar 60 is arranged in the actual port of the camera port PT1 to keep the inside of the subject airtight. The endoscope 20 is inserted into the trocar 60 toward the inside of the subject. The endoscope 20 can be operated with the position of the actual port of the camera port PT1 as a base point. An image sensor is arranged at the tip of the endoscope 20, and the image sensor can take an image. The distance from the actual port of the camera port PT1 to the tip of the endoscope 20 is the insertion distance ID1 of the endoscope 20.

Further, a virtual port of the forceps port PT2 is set on the body surface 70 of the subject in the virtual pneumoperitoneum state. On the pneumoperitoneum body surface 70, a real port corresponding to the virtual port of the forceps port PT2 is installed. A trocar 60 is arranged at the actual port of the forceps port PT2, and the inside of the subject is kept airtight. The forceps 30 are inserted into the trocar 60 toward the inside of the subject. The forceps 30 can be operated with the position of the actual port of the forceps port PT2 as a base point. The distance from the actual port of the forceps port PT2 to the tip of the forceps 30 is the insertion distance ID 2 of the forceps 30.

The forceps 30 travel inside the subject toward, for example, the target 40 and are used for various treatments (eg, grasping, excision, peeling, suturing) on the target 40. The endoscope 20 is operated so that, for example, the target 40 and the forceps 30 are included in the imaging range 23. Therefore, the medical image processing device 100 makes it possible to visually recognize the treatment of the target 40 by the forceps 30 by visualizing the image corresponding to the endoscopic image by the endoscope 20, for example, the surgeon can use the forceps 30 and the target 40. It is possible to support the endoscopic surgery while confirming the positional relationship of the images.

Further, FIG. 3 also shows the x-direction, the y-direction, and the z-direction of the subject coordinate system (patient coordinate system) with respect to the subject. The subject coordinate system is a Cartesian coordinate system. The x direction may be along the left-right direction with respect to the subject. The y direction may be the anteroposterior direction (thickness direction of the subject) with respect to the subject. The z direction may be the vertical direction (the body axis direction of the subject) with respect to the subject. The x-direction, y-direction, and z-direction may be three directions defined by DICOM (Digital Imaging and COmmunications in Medicine). For example, the values of the subject coordinate system are used for the 3D data of the subject, the area inside the subject, the position of the virtual port, and the like.

FIG. 4 is a diagram showing an example of a port coordinate system with reference to the port PT. The port coordinate system is represented by a spherical coordinate system (polar coordinate system), and the diameter r, the first angle θ, and the second angle φ are shown. The origin of the port coordinate system is the position of the port PT. The radius r is parallel to the extending direction of the endoscope 20. The first angle θ is an angle formed by a predetermined axis (for example, the y-axis of the subject coordinate system) and the radius r. The second angle φ is an angle formed by another axis (for example, the z-axis of the subject coordinate system) in a plane perpendicular to a predetermined axis and the projection of the diameter r on this plane. The extending direction of the endoscope 20 is the depth direction of the endoscope image (real endoscope image) and the virtual endoscope image.

For example, the value of the port coordinate system is used as the insertion distance and orientation of the surgical instrument 300 with respect to the port position. The insertion distance of the surgical instrument 300 may be indicated as the coordinates of the radius r. The orientation of the surgical instrument 300 may be detected, for example, as the values of the first angle θ and the second angle φ. The values of the first angle θ and the second angle φ may be the values detected by the angle sensor. Further, the surgical instrument information processing unit 165 may calculate and obtain the first angle θ and the second angle φ based on the values detected by the angle sensor. In this case, for example, the angle sensor is an acceleration sensor and may detect an inclination angle in two directions orthogonal to the direction of gravity. Further, the angle sensor may be a 3-axis angle sensor.

When the insertion distance and orientation of the surgical instrument 300 using the port coordinate system are used for processing the 3D data of the subject, it is necessary to convert the value of the port coordinate system into the value of the subject coordinate system. Here, the conversion from the port coordinate system (spherical coordinate system) to the subject coordinate system (Cartesian coordinate system) is compared with the conversion from the subject coordinate system to the operating room coordinate system (conversion in a different Cartesian coordinate system). , The amount of calculation is small, and it can be carried out relatively easily. The orientation of the surgical instrument 300 does not change much when compared with the detected value of the position even if there is some movement of the subject (for example, the movement of the patient at the time of examination and the beating of the heart), and when compared with the alignment. Easy to adjust the angle.

FIG. 5 is a diagram showing an example of the positional relationship between the tip position 21, the tip direction 22, the imaging range 23, and the forceps 30 of the endoscope 20. The tip position 21, tip orientation 22, and imaging range 23 of the endoscope 20 are included in the information about the endoscope 20. The position of the forceps 30 is included in the information about the forceps 30. The tip direction 22 of the endoscope 20 may coincide with the direction of the moving diameter r and may coincide with the direction of the optical axis of the endoscope 20 (camera). The imaging range 23 of the endoscope 20 may be determined based on, for example, the tip position 21 of the endoscope 20, the tip orientation 22 of the endoscope 20, and the angle of view of the endoscope 20. The imaging range 23 of the endoscope 20 coincides with the image range of the virtual endoscope image.

The tip position 21 of the endoscope 20 can be said to be the viewpoint of the operator for observing the subject. The tip direction 22 of the endoscope 20 can be said to be the direction of the operator's line of sight starting from the viewpoint. The angle of view of the endoscope 20 can be said to be the viewing angle of the operator based on the viewpoint. The imaging range 23 of the endoscope 20 can be said to be the field of view of the operator.

FIG. 6 is a diagram showing a display example in which the volume rendering image G1 and various types of information are superimposed and displayed. The display target displayed in FIG. 6 is virtual information. For example, a virtual port is displayed as the port PT, a virtual forceps is displayed as the forceps 30, and a virtual endoscope is displayed as the endoscope 20.

In the volume rendered image G1, a part of the subject is visualized. In FIG. 6, various information is displayed together with the volume rendered image G1. The various information may include information about the endoscope 20, information about the forceps 30, information about the virtual port of the camera port PT1, information about the virtual port of the forceps port PT2, information about the target 40, various distance information, and the like. The display of some of the various information may be omitted.

The information about the endoscope 20 may include information such as the position, orientation, imaging range, size, shape, and the like of the endoscope 20. The information about the forceps 30 may include information such as the position, orientation, size, shape, and the like of the forceps 30. In FIG. 6, the endoscope 20 and the forceps 30 are shown in a straight line for simplification, but the present invention is not limited to this, and the endoscope 20 and the forceps 30 are shown based on the orientation, size, shape, and the like. You may.

The information on the virtual port of the camera port PT1 may include information such as the position, size, and shape of the virtual port of the camera port PT1. The information on the virtual port of the forceps port PT2 may include information such as the position, size, and shape of the virtual port of the camera port PT1. The information of the target 40 may include information such as the position, size, shape, and the like of the target 40. In FIG. 6, the target is shown by ◯ for simplification, but the target is not limited to this, and may be shown based on the orientation, size, shape, and the like of the target 40.

The various distance information may include information on the insertion distance ID1 of the endoscope 20, information on the insertion distance ID2 of the forceps 30, information on the distance d1 from the tip of the forceps 30 to the target 40, and the like. The distance d1 can be calculated by, for example, the difference between the tip position of the forceps 30 and the position of the target 40 based on the orientation of the forceps 30.

FIG. 7 is a diagram showing a display example in which the virtual endoscopic image G2 and various types of information are superimposed and displayed. In the description of FIG. 7, the same items as those described in FIG. 6 will be omitted or simplified.

In the virtual endoscopic image G2 of FIG. 7, a part of the subject is visualized. In FIG. 7, various information is displayed together with the virtual endoscopic image G2. The various information may include information on one or more forceps 30 (for example, forceps 31 and forceps 32), information on the target 40, various distance information, and the like. The display of some of the various information may be omitted.

Various distance information includes the insertion distance ID1 of the endoscope 20, the insertion distance ID21 of the forceps 31, the insertion distance ID22 of the forceps 32, and the distance d11 from the tip (tip position) of the forceps 31 to the target 40. , Information on the distance d1 from the tip of the forceps 32 to the target 40, and the like may be included. As the distance information, instead of the information on the distance from the tip of the forceps 30 to the target 40, the information on the distance from the virtual port of the port PT to the target 40 and the distance from the virtual port of the port PT to the target 40 are used. Information on the distance obtained by subtracting the insertion distance ID 2 of the forceps 30 may be included and displayed.

According to the display examples of FIGS. 6 and 7, the operator can easily confirm the distance including the depth direction of the image, which is difficult to recognize by confirming the two-dimensional image. For example, the surgeon can confirm information such as how many meters the distance to the target 40 is.

FIG. 8 is a flowchart showing an operation example of the medical image processing device 100. In addition, S11 to S15 are carried out preoperatively, for example, and S16 to S20 are carried out, for example, intraoperatively.

First, the volume data of the subject (for example, the patient) is acquired (S11). A pneumoperitoneum simulation is executed (S12). Segmentation is performed to extract regions of organs, bones, and blood vessels (S13). A model (for example, a liver organ model) is generated based on the volume data (S14). Port position simulation, port position score calculation, port position adjustment, and the like are performed to acquire information (for example, position information) of the forceps port PT2 as a virtual port, the camera port PT1 as a virtual port, and the target 40 (for example, position information).

The operation information of the forceps 30 and the operation information of the endoscope 20 are acquired via the UI 120 (S16). The orientation of the endoscope 20 and the insertion distance ID1 are acquired (S17). The direction of the forceps 30 and the insertion distance ID2 are acquired (S17). The tip position and tip orientation of the endoscope 20 are acquired (S18). The tip position and tip orientation of the forceps 30 are acquired (S18). The volume data in the virtual pneumoperitoneum state obtained by the process of S12 is rendered to generate a volume rendered image G1 (S19). Various information is displayed together with the volume rendering image G1 (S19). The various information includes various information shown in FIG. 6, information indicating the endoscope 20, information indicating the forceps 30, tip position and tip orientation of the endoscope 20, tip position and tip orientation of the forceps 30, and the like. It's fine. Further, the virtual endoscopic image G2 is generated based on the volume data of the virtual pneumoperitoneum state by the processing of S12 (S20). Various information is displayed together with the virtual endoscopic image G2 (S20). The various information may include various information shown in FIG. 7, information indicating the forceps 30, the tip position and tip orientation of the forceps 30, and the like.

After the display in S20, the processing unit 160 may proceed to S16 and repeatedly execute S16 to S20. As a result, the operator confirms the images (for example, volume rendering image G1 and virtual endoscope image G2) according to the movement while appropriately operating and moving the endoscope 20 and the forceps 30 during the operation according to the procedure. it can.

The accuracy of organ segmentation, bone segmentation, and blood vessel segmentation in S13 does not have to be high. For example, even if there is some error in the contours of organs, bones, and blood vessels, it is sufficient if there is little hindrance to the intraoperative simulation.

As described above, the medical image processing device 100 does not need to use the three-dimensional position sensor, but the tip position of the surgical instrument 300 is based on information such as the set position of the virtual port and the insertion distance ID of the surgical instrument 300. The distance between and the target 40 can be derived. Therefore, although it is difficult for the operator to grasp the sense of depth from the two-dimensional endoscopic image, it becomes easy for the surgeon to grasp the positional relationship between the tip position of the surgical instrument 300 and the target 40. Therefore, for example, it is possible to grasp how close the target 40 is. In addition, the surgeon can be confident that the surgeon's treatment is correct by the surgical support by such an image.

Therefore, in the surgical navigation, the medical image processing device 100 does not need to align the operating room coordinate system by the external camera or the sensor coordinate system by the three-dimensional position sensor with the subject coordinate system based on the subject, and performs calculation. The amount can be reduced.

In addition, even if the measurement of the 3D position of the actual port is omitted, for example, in the abdomen and lungs where the organs are greatly deformed, the 3D position moves even if the 3D position is acquired, so that the exact 3D coordinates are less important. , The effect on surgical navigation is small. In addition, the operator can intuitively determine the positional relationship between the surgical instrument 300 and the target 40, etc. in the two-dimensional position in the image that visualizes the 3D data, rather than the information based on the three-dimensional coordinates by the three-dimensional position sensor. Is easy to grasp. This is because the virtual endoscope image G2 is an image based on the position of the endoscope 20, and is an image close to the endoscope image referred to by the operator.

As described above, the medical image processing device 100 can perform surgical navigation with a simple mechanism, can improve the safety of surgery, and can shorten the surgical time.

Next, the details of the example of determining the position of the virtual port will be described.

The port information processing unit 164 executes a port position simulation. The port position simulation is a simulation for determining whether or not a desired operation on a subject is possible by operating the UI 120 by the user. In the port position simulation, the user operates the surgical instrument 300 inserted from the position of each virtual port (virtual port position) in the virtual space while assuming surgery, and the forceps 30 can access the target 40 to be operated on. It may be determined whether or not. That is, in the port position simulation, it may be determined whether or not the surgical instrument 300 can access the target 40 without any problem while manually operating the surgical instrument 300. The port information processing unit 164 may obtain planning information of the virtual port position by port position simulation.

In the port position simulation, the volume data of the subject, the combination of the acquired multiple virtual port positions, the movable information (for example, the movable position and range) regarding the movement of the surgical instrument 300, the surgical procedure, and the virtual pneumoperitoneum state. Whether or not the above access is possible may be determined based on the volume data, etc. The movable information of the surgical instrument 300 may include shape information regarding the shape of the surgical instrument 300 and motion information regarding the movement. This shape information may include at least a part of information such as the length, weight, and shape of the surgical instrument 300. This motion information may include at least a part of information such as the insertion distance of the surgical instrument 300 into the inside of the subject, the insertion angle with respect to the subject, and the like. In the case of robotic surgery, the movable information of the surgical instrument 300 may correspond to the kinematics of the end effector. The port information processing unit 164 may acquire the movable information of the surgical instrument 300 from the memory 150, or may acquire it from an external device via the acquisition unit 110. The port information processing unit 164 may acquire the information of the surgical procedure from the memory 150 or may be acquired from an external device via the acquisition unit 110.

The port information processing unit 164 may determine whether or not the target 40 can be accessed at each virtual port position while changing a plurality of virtual port positions on the body surface of the subject, and may sequentially perform port position simulation. The port information processing unit 164 may finally specify information on a preferred (for example, optimal) combination of port positions via the UI 120 according to user input. As a result, the port information processing unit 164 may plan a plurality of virtual port positions.

The port information processing unit 164 may calculate a port position score indicating the appropriateness in the case of surgery using a plurality of virtual port positions provided on the body surface of the subject. That is, a port position score based on a combination of multiple virtual port positions indicates the value of the combination of multiple virtual port positions for performing surgery. The port position score may be calculated based on a combination of a plurality of virtual port positions, movable information of the surgical instrument 300, a surgical procedure, volume data of a virtual pneumoperitoneum state, and the like. The port position score is calculated for each virtual port position.

The port information processing unit 164 may adjust the port position based on the port position score. In this case, the port information processing unit 164 may adjust the port position based on the amount of change in the port position score accompanying the movement of the port position.

In this way, the port information processing unit 164 may calculate a plurality of virtual port positions according to the port position simulation. Further, the port information processing unit 164 may calculate a plurality of virtual port positions based on the port position score.

FIG. 9 is a flowchart showing an example of the procedure of port position simulation.

First, the port information processing unit 164 acquires volume data including the subject via, for example, the acquisition unit 110 (S111). The port information processing unit 164 acquires the movable information of the surgical instrument 300 via, for example, the acquisition unit 110 (S112). The deformation processing unit 162 executes the pneumoperitoneum simulation (S113) and generates volume data of the virtual pneumoperitoneum state of the subject.

The port information processing unit 164 acquires information on the surgical procedure (S114). The port information processing unit 164 acquires and sets a plurality of virtual port positions (initial positions) according to the acquired surgical technique (S114). In this case, the port information processing unit 164 may set a plurality of virtual port positions in three-dimensional coordinates.

The port information processing unit 164 sets the position of the target 40 (target area) (S115).

The port information processing unit 164 determines whether or not each forceps 30 inserted from each virtual port can access the target 40 based on the positions of the plurality of virtual ports acquired in S114 and the positions of the target area ( S116). Whether or not each forceps 30 has access to the target 40 may correspond to whether or not each forceps 30 can reach all positions in the target region. That is, it indicates whether or not the operation according to the acquired surgical technique is possible by the forceps 30 (a plurality of forceps 30 if necessary), and if it is accessible, the operation is possible. Is shown.

When at least one of each forceps 30 is inaccessible to at least a part of the target area, the port information processing unit 164 sets the position of at least one virtual port included in the plurality of virtual ports on the body surface of the subject. (S117). In this case, the port information processing unit 164 may move the virtual port position based on the user input via the UI 120. The virtual port to be moved includes at least a virtual forceps port into which the forceps 30 that were inaccessible to at least a part of the target area are inserted. After moving the virtual port, the process proceeds to S116.

When each forceps 30 has access to the target region, the processing unit 160 ends the processing of the port position simulation of FIG.

In this way, the medical image processing apparatus 100 performs the port position simulation, and the plurality of acquired virtual ports are determined depending on whether or not the target area can be accessed using the acquired plurality of virtual port positions. It is possible to determine whether or not surgery using the position is possible. If the target area is inaccessible using multiple virtual port locations, at least part of the virtual port position can be modified via the UI 120 to access the target area using the modified multiple virtual port locations. It may be determined again whether or not there is. The port information processing unit 164 may set a combination of a plurality of virtual port positions that can access the target area at the positions of the plurality of virtual ports. In this way, the medical image processing apparatus 100 can manually adjust the virtual port position and plan the virtual port position.

The port position simulation may be performed as a part of the process of S15 in FIG. In this case, processing that overlaps with the processing of FIG. 8 (for example, S111 and S113) can be omitted.

Next, an example of calculating the port position score will be described.

It may be assumed that the plurality of virtual port positions are determined according to, for example, a surgical procedure, and are arranged at arbitrary positions on the body surface of the subject. Therefore, as for the combination of a plurality of virtual port positions, various combinations of virtual port positions may be assumed. One surgical instrument 300 can be inserted into the subject from one virtual port. Therefore, a plurality of surgical instruments 300 can be inserted into the subject from the plurality of virtual ports.

The range that one forceps 30 can reach in the subject via the virtual port is a working area (individual working area WA1 (see FIG. 11)) where work (treatment) can be performed by one forceps 30. Therefore, the area where the individual working areas WA1 by the plurality of forceps 30 overlap is the working area where the plurality of forceps 30 can reach simultaneously in the subject via the plurality of virtual ports (overall working area WA2 (see FIG. 11)). It becomes. Since it is necessary for a predetermined number (for example, three) of forceps 30 to operate simultaneously in the procedure according to the surgical procedure, the entire working area WA2 in which the predetermined number of forceps 30 can be reached at the same time is considered.

Further, since the position of the forceps 30 in the subject is different depending on the movable information of the forceps 30, this movable information is added to the derivation of the virtual port position where the forceps 30 is inserted into the subject. Further, since the position of the entire working area WA2 to be secured in the subject differs depending on the surgical procedure, this position is added to the derivation of the virtual port position corresponding to the position of the overall working area WA2.

The port information processing unit 164 may calculate the port position score for each combination of the plurality of virtual port positions. The port information processing unit 164 may plan a combination of virtual port positions having a port position score (for example, the maximum port score) satisfying a predetermined condition among a combination of a plurality of virtual port positions. That is, a plurality of virtual port positions included in the planned combination of virtual port positions may be planned at a plurality of port positions to be drilled.

In the case of robotic surgery, the relationship between the position of the virtual port and the operation of the movable part of the surgery support robot may satisfy the relationship described in, for example, Reference Non-Patent Documents 2 and 3.
(Reference Non-Patent Document 2): Mitsuhiro Hayashibe, Naoki Suzuki, Makoto Hashizume, Kozo Konishi, Asaki Hattori, “Robotic surgery setup simulation with the integration of inverse-kinematics computation and medical imaging”, computer methods and programs in biomedicine, 2006, P63-P72
(Reference Non-Patent Document 3) Pal Johan From, “On the Kinematics of Robotic-assisted Minimally Invasive Surgery”, Modeling Identication and Control, Vol.34, No.2, 2013, P69-P82

FIG. 10 is a flowchart showing an operation example when calculating the port position score by the medical image processing device 100.

Before the processing of FIG. 10, the volume data of the subject, the movement information of the surgical instrument 300, the execution of the pneumoperitoneum simulation, and the surgical procedure are obtained in the same manner as in S111 to S114 of the port position simulation shown in FIG. Information is acquired in advance. The initial value of the port position score is 0. The port position score is an evaluation function (evaluation value) indicating the value of a combination of port positions. The variable i is an example of work identification information, and the variable j is an example of port identification information.

The port information processing unit 164 generates work list works, which is a list of work works_i using each surgical instrument 300, according to the surgical procedure (S121). The work work_i contains information for working with each forceps 30 in a surgical procedure according to the surgical procedure. Working work_i may include, for example, gripping, excision, suturing and the like. The work may include a single work with a single forceps 30 and a cooperative work with a plurality of forceps 30.

The port information processing unit 164 plans the minimum area last_region_i, which is the minimum area required to perform the work work_i included in the work list works, based on the volume data of the surgical procedure and the virtual pneumoperitoneum state (S122). The minimum region may be defined as a three-dimensional region in the subject. The port information processing unit 164 generates the minimum area list Last_regions, which is a list of the minimum area last_region_i (S122).

The port information processing unit 164 is a recommended area effect_region_i, which is a recommended area for performing the work work_i included in the work list works, based on the surgical procedure, the movable information of the surgical instrument 300, and the volume data of the virtual pneumoperitoneum state. (S123). The port information processing unit 164 generates a recommended area list effects_regions, which is a list of recommended areas effective_region_i (S123). The recommended area may include, for example, a space recommended for the forceps 30 to operate, as well as a minimum space (minimum area) for performing the work.

The port information processing unit 164 acquires information on the port position list ports, which is a list of a plurality of virtual port position ports_j (S124). The virtual port position is determined by three-dimensional coordinates (x, y, z) with respect to the subject. The port information processing unit 164 may receive user input via the UI 120, for example, and acquire port position list ports including one or more virtual port positions specified by the user. The port information processing unit 164 may acquire the port position list ports held as a template in the memory 150.

The port information processing unit 164 uses each forceps 30 for each work work_i via each virtual port position port_j based on the surgical procedure, the movable information of the forceps 30, the volume data of the virtual pneumoperitoneum state, and the plurality of virtual port positions. Plan the port work area region_i, which is a workable area (S125). The port work area may be defined as a three-dimensional area. The port information processing unit 164 generates port work area list regions, which is a list of port work area regions_i (S125).

The port information processing unit 164 subtracts the port work area region_i from the minimum area rest_region_i for each work work_i to calculate a subtraction area (subtraction value) (S126). The port information processing unit 164 determines whether or not the subtraction area is an empty area (the subtraction value is a negative value) (S126) Whether or not the subtraction area is not an empty area is determined by at least a part of the minimum area forceps_region_i. , Indicates whether or not there is an area not covered by the port work area subtraction_i (the area where the forceps 30 do not reach through the virtual port).

When the subtraction area is an empty area, the port information processing unit 164 calculates the volume value Volume_i, which is the product of the recommended area effect_region_i and the port work area region_i (S127). Then, the port information processing unit 164 sums the volume values Volume_i calculated for each work work_i, and calculates the total value Volume_sum. The port information processing unit 164 sets the total value Volume_sum as the port position score (S127).

That is, when the subtraction area is an empty area, there is no area not covered by the port work area in the minimum area, and it is preferable that this port position list ports (combination of virtual port position ports_j) is selected. , A value for each work work_i is added to the port position score so that this port position list can be easily selected. Further, since the port position score is planned based on the volume value Volume_i, the larger the minimum area or the port work area, the larger the port position score, and the port position list ports can be easily selected. Therefore, the port information processing unit 164 can easily select a combination of virtual port positions that has a large minimum area and port work area and facilitates each procedure in surgery.

On the other hand, when the subtraction area is not an empty area, the port information processing unit 164 sets the port position score for the port position list ports to a value of 0 (S128). That is, since there is an area not covered by the port work area in at least a part of the minimum area and the work of the target work work_i may not be completed, it is preferable that this port position list Posts is selected. Absent. Therefore, the port information processing unit 164 sets the port position score to a value of 0 and excludes it from the selection candidates so that the port position list Posts is difficult to be selected. In this case, the port information processing unit 164 sets the total port position score to a value of 0 even if the area becomes empty when another work work_i is performed using the same port position list ports.

The port information processing unit 164 may repeat each step of FIG. 10 for all work works_i and calculate a port position score including all work works_i.

In this way, when the medical image processing apparatus 100 derives the port position score and performs an operation using a plurality of virtual port positions provided on the body surface of the subject, which combination of virtual port positions is used. You can understand whether it is appropriate. The individual working area WA1 and the entire working area WA2 depend on the arrangement positions of the plurality of virtual ports. Even in this case, the medical image processing apparatus 100 adds the score (port position score) for each combination of the plurality of virtual port positions, so that, for example, the port position score becomes the threshold value th1 or more (for example, the maximum) of the plurality of virtual ports. The combination of positions can be derived, and the virtual port position where surgery can be easily performed can be set.

In addition, by appropriately securing the working area based on the port position score, the user can secure a wide field of view in the subject that cannot be directly seen in the operation, and can secure a wide port working area, which is an unexpected situation. It will be easier to deal with.

Further, in the arthroscopic surgery, the position of the actual port to be perforated corresponding to the virtual port does not change, but the forceps 30 inserted into the actual port can move within a predetermined range. Therefore, in arthroscopic surgery, planning the virtual port position is important because the difficulty of each procedure in the surgery changes depending on the planned virtual port position.

FIG. 11 is a diagram showing an example of a working area determined based on the virtual port position. The individual working area WA1 is an individual working area corresponding to each virtual port position port_j. The individual working area WA1 may be an area within the subject PS that is reachable by individual end effectors. The area where each individual working area WA1 overlaps is the entire working area WA2. The entire working area WA2 may correspond to the port working area region_i. The medical image processing apparatus 100 can optimize each virtual port position by using the port position score, and can derive suitable individual working area WA1 and overall working area WA2.

Next, the details of the port position adjustment will be described.

The port information processing unit 164 acquires information on a plurality of virtual port positions (candidate positions) based on, for example, a template held in the memory 150 or a user instruction via the UI 120. The port information processing unit 164 calculates a port position score when the plurality of virtual port positions are used based on the acquired combination of the plurality of virtual port positions.

The port information processing unit 164 may adjust the position of the virtual port based on the port position score. In this case, the port information processing unit 164 sets the port position score in the case of the acquired plurality of virtual port positions and the port position score in the case of changing at least one of the plurality of virtual port positions. , The virtual port position may be adjusted based on. In this case, the port information processing unit 164 may take into account the minute movement and differentiation of the port position along each direction (x direction, y direction, z direction) in the three-dimensional space.

For example, the port information processing unit 164 may calculate the port position score F (ports) for a plurality of virtual port positions and calculate the differential value F'of F according to (Equation 1).

F (port_j (x + Δx, y, z)) --F (port_j (x, y, z))
F (port_j (x, y + Δy, z)) --F (port_j (x, y, z)) ・ ・ ・ (Equation 1)
F (port_j (x, y, z + Δz)) --F ((port_j (x, y, z)))

That is, the port information processing unit 164 calculates the port position score F in the case of the virtual port position F (port_j (x + Δx, y, z)), and the virtual port position F (port_j (x, y, z)). The port position score F in the case of is calculated, and the difference thereof is calculated. This difference value indicates the change in the port position score F with respect to a minute change in the x direction at the virtual port position F (port_j (x, y, z)), that is, indicates the differential value F'of F in the x direction.

Further, the port information processing unit 164 calculates the port position score F in the case of the virtual port position F (port_j (x, y + Δy, z)), and the virtual port position F (port_j (x, y, z)). The port position score F in the case of is calculated, and the difference thereof is calculated. This difference value indicates the change in the port position score F with respect to a minute change in the y direction at the virtual port position F (port_j (x, y, z)), that is, indicates the differential value F'of F in the y direction.

Further, the port information processing unit 164 calculates the port position score F in the case of the virtual port position F (port_j (x, y, z + Δz)), and the virtual port position F (port_j (x, y, z)). The port position score F in the case of is calculated, and the difference thereof is calculated. This difference value indicates the change in the port position score F with respect to a minute change in the z direction at the virtual port position F (port_j (x, y, z)), that is, indicates the differential value F'of F in the z direction.

The port information processing unit 164 calculates the maximum value of the port position score based on the differential value F'in each direction. In this case, the port information processing unit 164 may calculate the virtual port position where the port position score is maximized according to the steepest descent method based on the differential value F'. The port information processing unit 164 may adjust the virtual port position and optimize the virtual port position so that the calculated virtual port position is the port position to be drilled. It should be noted that the virtual port position that maximizes the port position score may not be used, for example, a position where the port position score is equal to or higher than the threshold value th2, and the port position score may be improved (higher).

The port information processing unit 164 applies such adjustment of the virtual port position to the adjustment of other virtual port positions included in the combination of the plurality of virtual port positions, or virtual in another combination of the plurality of virtual port positions. It may be applied to adjust the port position. As a result, the port information processing unit 164 can plan a plurality of virtual ports whose virtual port positions have been adjusted (for example, optimized) at the port positions to be drilled. In this way, the medical image processing device 100 can automatically adjust the virtual port position and set the virtual port position.

In addition, at a plurality of port positions, an error of about a predetermined length (for example, 25 mm) may occur between the position of the virtual port (planned drilling position) and the position of the actual port (actual drilling position), and the planning accuracy of the virtual port position. It is considered that 3 mm at most is sufficient. Therefore, the port information processing unit 164 sets candidates for virtual port positions for each predetermined length on the body surface of the subject, determines a combination of a plurality of virtual port positions by brute force, and determines a combination of the plurality of virtual port positions. Port position scores for each combination of virtual port positions may be calculated. That is, candidates for virtual port positions may be arranged in a grid pattern having a predetermined length (for example, 3 mm) on the body surface of the subject. Further, when the number of ports assumed on the body surface (for example, the number of intersections in a grid pattern) is n and the number of ports included in the combination of virtual port positions is m, the port information processing unit 164 From the n virtual port position candidates, m virtual port positions may be sequentially selected and combined, and the port position score for each combination may be calculated. In this way, when the grid is not excessively fine like a grid at intervals of 3 mm, it is possible to prevent the calculation load of the port information processing unit 164 from becoming excessive, and it is possible to calculate the port position scores of all combinations. ..

The port information processing unit 164 may adjust a plurality of virtual port positions according to a known method. The port information processing unit 164 may plan the port position to be drilled to a plurality of virtual port positions included in the adjusted combination of virtual port positions. Known methods of port position adjustment may include the techniques described in Reference Non-Patent Documents 4 and 5 and Reference Patent Document 1 below.

(Reference Non-Patent Document 4) Shaun Selha, Pierre Dupont, Robert Howe, David Torchiana, “Dexterity optimization by port placement in robot-assisted minimally invasive surgery”, SPIE International Symposium on Intelligent Systems and Advanced Manufacturing, Newton, MA, 28- 31, 2001
(Reference Non-Patent Document 5) Zhi Li, Dejan Milutinovic, Jacob Rosen, “Design of a Multi-Arm Surgical Robotic System for Dexterous Manipulation”, Journal of Mechanisms and Robotics, 2016
(Reference Patent Document 1) U.S. Patent Application Publication No. 2007/0249911

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modifications or modifications within the scope of the claims, which naturally belong to the technical scope of the present disclosure. Understood.

For example, the arthroscopic surgery in which preoperative simulation and intraoperative navigation are performed may be laparoscopic surgery, arthroscopic surgery, bronchoscopic surgery, colonoscopic surgery, or other arthroscopic surgery. Further, the arthroscopic surgery may be a surgery in which the surgeon directly operates the forceps or a robotic surgery using a surgical robot.

Also, the pneumoperitoneum simulation may be omitted. For example, in arthroscopic surgery and bronchoscopic surgery, pneumoperitoneum simulation may be omitted.

In addition, the surgical instrument information processing unit 165 may acquire an endoscope image from an endoscope device including the endoscope 20 via the acquisition unit 110. Then, the position of the surgical instrument 300 reflected in the endoscopic image, that is, the relative position of the surgical instrument 300 with respect to the endoscopic image may be recognized by image recognition or the like. The surgical instrument information processing unit 165 may correct (adjust) the acquired position of the surgical instrument 300 based on the position of the surgical instrument 300 reflected in the endoscopic image. Further, the surgical instrument information processing unit 165 may correct (adjust) the tip position of the endoscope 20 based on the position of the surgical instrument 300 reflected in the endoscope image. Therefore, using the endoscope image, it is possible to adjust the image range of the virtual endoscope image G2 with the tip position of the endoscope 20 on which the image sensor is arranged as a viewpoint. As a result, even if the derivation accuracy of the position and orientation (angle) of the surgical instrument 300 is insufficient, the accuracy of the position of the surgical instrument 300 in the volume rendered image G1 can be obtained by using the endoscope image of the endoscope 20. The generation accuracy of the virtual endoscopic image G2 is improved.

The surgical instrument information processing unit 165, for example, according to the methods of Reference Patent Document 2 and Reference Non-Patent Documents 6 and 7, based on the position of the surgical instrument 300 in the endoscope image of the endoscope 20, a volume rendered image The position of the surgical instrument 300 in G1 and the tip position of the endoscope 20 which is the viewpoint of the virtual endoscope image G2 may be corrected.
(Reference Patent Document 2: Japanese Patent No. 4869189)
(Reference Non-Patent Document 6: Toshiya Nakaguchi, 5 outsiders, "Implementation of automatic magnifying tracking system in laparoscopic surgery", [online], Japanese Society for Medical and Biological Engineering, 2005 Vol. 43, No. 4, P685-P693, [Search on August 9, 1991], Internet <URL: https://www.jstage.jst.go.jp/article/jsmbe/43/4/43_4_685/_pdf/-char/en>)
(Reference Non-Patent Document 7: Kyojiro Minami, 3 outsiders, "Minimally Invasive Advanced Surgery Support System by Endoscopy", [online], Toshiba Review, Vol.56, No.9, 2001, [Reiwa Search on August 9, 1], Internet <URL: https://www.toshiba.co.jp/tech/review/2001/09/56_09pdf/a09.pdf#search=%27%E3%80%8C% E5% 86% 85% E8% A6% 96% E9% 8F% A1% E3% 81% AA% E3% 81% A9% E3% 81% AB% E3% 82% 88% E3% 82% 8B% E4% BD% 8E% E4% BE% B5% E8% A5% B2% E9% AB% 98% E5% BA% A6% E6% 89% 8B% E8% A1% 93% E6% 94% AF% E6% 8F% B4% E3% 82% B7% E3% 82% B9% E3% 83% 86% E3% 83% A0% E3% 80% 8D% E5% 8D% 97% E9% 83% A8 +% E6% 9D% B1% E8% 8A% 9D% E3% 83% AC% E3% 83% 93% E3% 83% A5% E3% 83% BC2001% 27>)

Further, various sensors may measure (actual measurement) the position of the port, the position of the surgical instrument 300, and the angle of the surgical instrument 300 during the actual operation. For example, a three-dimensional position sensor or an angle sensor may be attached to the surgical instrument 300 or the trocar 60 to acquire actual measured values of the three-dimensional position sensor or the angle sensor. The surgical instrument information processing unit 165 may acquire actual measured values acquired by various sensors. The display control unit 167 may display the information of the actual measured value. The surgeon may check the displayed actual measurement value information and manually adjust the position and orientation of the virtual surgical instrument 300 and the virtual port position via the UI 120. Alternatively, the processing unit 160 may, based on the above-mentioned actual measurement value, for example, match the position, orientation, or virtual port position of the surgical instrument 300 actually measured so as to match the position, orientation, or virtual position of the virtual surgical instrument 300. The port position may be adjusted automatically. In this case, the volume rendering image G1 and the virtual endoscopic image G2 may be generated and displayed based on the adjusted virtual port position. As a result, the medical image processing device 100 can adjust the position, orientation, and port of the surgical instrument 300 based on the virtual calculation result to the actual state during the operation, and can further improve the accuracy of the intraoperative navigation. Further, the actual port position may be actually measured by the three-dimensional position sensor only when highly accurate information is required (for example, when the surgical instrument 300 is near an important target). As a result, the measurement frequency of the actual measurement can be reduced to reduce the processing load, and the timing of the actual measurement can be delayed. Further, the accuracy of the three-dimensional position sensor and the angle sensor may be low. The surgical instrument information processing unit 165 may adjust the position and orientation of the virtual surgical instrument 300 and the virtual port position according to the accuracy of the measured actual measured value.

Further, the position of the actual port position may be partially measured. The information obtained by partially measuring includes, for example, only the X coordinate of the three-dimensional position, the information acquired from the camera in the operating room, and the distance from the specific land port measured by the tape measure. In addition, the accuracy of the partially measured actual measurement information may be low. The surgical instrument information processing unit 165 may adjust the position and orientation of the virtual surgical instrument 300 and the virtual port position according to the accuracy and degree of freedom of the measured actual measured value.

Further, similarly to the target 40, the region or position of a dangerous part (for example, a blood vessel that should not be cut, another organ that does not include the target 40) that the operator should pay attention to during the operation may be set. The processing unit 160 may perform the same processing on the target 40 as on the dangerous portion. For example, the processing unit 160 sets a dangerous part in the volume data, and superimposes information based on the distance between the surgical instrument 300 and the dangerous part on the image in which the 3D data of the subject is visualized and displays it. Good. In this case, for example, it is difficult for the surgeon to grasp the sense of depth in the two-dimensional endoscopic image, but it becomes easy for the surgeon to grasp the position of the surgical instrument 300 and the positional relationship between the tip position of the surgical instrument 300 and the dangerous part. Therefore, for example, it is possible to grasp how close the dangerous part is.

Further, although it was shown that the surgical instrument 300 and the port are not provided with the three-dimensional position sensor and the marker for specifying the position, the three-dimensional position sensor and the marker may be provided as an auxiliary. For example, a three-dimensional position detection sensor or marker may be attached to the trocar 60, and the surgical instrument 300 information processing unit 165 may acquire the port position or the position or direction of the surgical instrument 300 by using the three-dimensional position detection sensor or marker. .. The method of obtaining the port position and the direction of the surgical instrument 300 using the marker may be the same as the method shown in Non-Patent Document 1. As a result, the medical image processing apparatus 100 can use the three-dimensional position detection sensor or marker to position and orient the surgical instrument 300 or the tip of the surgical instrument 300 or the tip of the surgical instrument 300 when the extraction accuracy is low. Information on the positions and directions of 300 can be acquired as an auxiliary. Further, by arranging the three-dimensional position sensor and the marker on the trocar 60, for example, the root portion of the forceps 30 can be easily seen, and the operator can easily perform the operation.

Further, the conversion of the value of the port coordinate system based on the value detected by the angle sensor into the value of the subject coordinate system has been illustrated, but the present invention is not limited to this. For example, the surgical instrument information processing unit 165 may convert the value of the port coordinate system based on the value detected by the angle sensor into the value of the coordinate system (Cartesian coordinate system) based on the bed on which the subject is placed. Good. Further, the processing unit 160 may generate an image or derive distance information using the tilt angle of the surgical instrument 300 with respect to the direction of gravity, or the surgical instrument in the subject coordinate system based on the tilt angle of the surgical instrument 300. The orientation of the 300 may be calculated, and image generation and distance information may be derived based on the orientation of the surgical instrument 300 in the subject coordinate system.

Further, although it has been illustrated that the angle information is derived for both the endoscope 20 and the forceps 30, the derivation of the angle information of the forceps 30 may be omitted. This is because the surgeon can grasp in which direction the forceps 30 are oriented by the position and orientation of the forceps 30 in the endoscopic image by the endoscope 20 or the virtual endoscopic image G2. That is, the angle of the forceps 30 can be imitated by the relative position or orientation of the forceps 30 with respect to the endoscopic image by the endoscope 20 or the virtual endoscopic image G2. Therefore, even if the information on the orientation of the forceps 30 is absent, the operator can grasp the positional relationship between the tip position of the forceps 30 and the position of the target 40.

Further, the medical image processing device 100 may include at least a processor 140 and a memory 150. The acquisition unit 110, the UI 120, and the display 130 may be external to the medical image processing device 100.

Further, it is exemplified that the volume data as the captured CT image is transmitted from the CT device 200 to the medical image processing device 100. Instead, the volume data may be transmitted to a server on the network (for example, an image data server (PACS) (not shown)) and stored so that the volume data is temporarily stored. In this case, the acquisition unit 110 of the medical image processing device 100 may acquire the volume data from the server or the like via a wired line or a wireless line when necessary, or acquire the volume data via an arbitrary storage medium (not shown). You may.

Further, it is exemplified that the volume data as the captured CT image is transmitted from the CT device 200 to the medical image processing device 100 via the acquisition unit 110. This includes the case where the CT apparatus 200 and the medical image processing apparatus 100 are substantially combined into one product. It also includes the case where the medical image processing device 100 is treated as the console of the CT device 200.

Further, although it has been illustrated that the CT apparatus 200 captures an image and generates volume data including information inside the subject, another apparatus may capture an image and generate volume data. Other devices include MRI (Magnetic Resonance Imaging) devices, PET (Positron Emission Tomography) devices, angiography devices (Angiography devices), or other modality devices. Moreover, the PET apparatus may be used in combination with other modality apparatus.

Further, it can be expressed as a medical image processing method in which the operation in the medical image processing device 100 is defined. Further, it can be expressed as a program for causing a computer to execute each step of the medical image processing method.

(Outline of the above embodiment)
One aspect of the above embodiment is a medical image processing device 100 (an example of a arthroscopic surgery support device) that supports arthroscopic surgery, and includes a processing unit 160. The processing unit 160 acquires 3D data of the subject (for example, volume data in the unhungry state, volume data in the virtual hungry state, and a three-dimensional model). The processing unit 160 sets a virtual port on the body surface 70 of the subject in the 3D data. The processing unit 160 acquires the insertion distance ID of the surgical instrument 300 inserted inside the subject from the real port corresponding to the virtual port. The processing unit 160 superimposes information indicating the surgical instrument 300 on the first image that visualizes the 3D data based on at least the position of the virtual port and the insertion distance ID of the surgical instrument 300 (display unit 160). Display in (example). The real port corresponding to the virtual port may be, for example, a real port installed as planned with respect to the virtual port, or may be a real port installed deviating from the planned virtual port position.

As a result, the medical image processing device 100 does not need to install a highly accurate three-dimensional position sensor for deriving the position of the surgical instrument 300 in the operating room, and can simplify the system configuration in the surgical navigation. This three-dimensional position sensor corresponds to, for example, an external camera installed on the ceiling of an operating room in Non-Patent Document 1. Further, since the three-dimensional position sensor is not required, the operating room coordinate system in which the three-dimensional position sensor is installed is absent. This eliminates the need for time-consuming measurement of the positional relationship between the subject coordinate system and the operating room coordinate system before surgery, and eliminates the step of registering the subject coordinate system and the operating room coordinate system. In addition, the step of re-registration according to the movement of the patient during the operation becomes unnecessary.

Further, the 3D data may include 3D data of a virtual pneumoperitoneum state in which a pneumoperitoneum simulation has been performed. As a result, the medical image processing apparatus 100 does not require a step of registering the subject coordinate system and the operating room coordinate system coordinates before the operation even for the subject who needs pneumoperitoneum in the actual operation. In addition, the step of re-registration according to the movement of the patient during the operation becomes unnecessary.

Further, the processing unit 160 may acquire the insertion direction of the surgical instrument 300 inside the subject and display information indicating the surgical instrument 300 based on the insertion direction of the surgical instrument 300. As a result, the medical image processing device 100 can display the direction (orientation) of the surgical instrument 300 in the first image. Therefore, the surgeon can intuitively grasp, for example, the orientation of the surgical instrument 300 with respect to the field of view of the endoscope 20. In addition, the accuracy of surgical navigation can be improved.

In addition, the processing unit 160 may calculate the insertion direction of the surgical instrument 300 inside the subject and display information indicating the surgical instrument 300 based on the insertion direction of the surgical instrument 300. As a result, the medical image processing device 100 can display the calculated and estimated direction (orientation) of the surgical instrument 300 in the first image. Therefore, the surgeon can intuitively grasp, for example, the orientation of the surgical instrument 300 with respect to the field of view of the endoscope 20. In addition, the accuracy of surgical navigation can be improved. In addition, the angle sensor becomes unnecessary, and the system configuration in surgical navigation can be simplified. Further, since the angle sensor is not required, the step of registering the angle between the subject coordinate system and the operating room coordinate system before the operation becomes unnecessary. In addition, the step of re-registration according to the movement of the patient during the operation becomes unnecessary.

In addition, there are two or more surgical instruments 300, and at least one of them may include an endoscope 20. There are two or more virtual ports, at least one of which may include a virtual camera port into which the endoscope 20 is inserted on the body surface 70 of the subject. The processing unit 160 may acquire the insertion distance ID 1 of the endoscope 20 inserted inside the subject from the actual camera port corresponding to the virtual camera port. The processing unit 160 may calculate the tip position 21 of the endoscope 20 based on at least the position of the virtual camera port and the insertion distance ID 1 of the endoscope 20. The first image may be a virtual endoscope image G2 with the tip position 21 of the endoscope 20 as a viewpoint. As a result, the medical image processing apparatus 100 can derive the tip position of the endoscope 20 with a simple system configuration and a small amount of calculation, and can visualize the virtual endoscope image G2 based on this position. This makes it possible to improve the accuracy of surgical navigation.

In addition, the processing unit 160 may acquire a real endoscope image which is an image captured by the endoscope 20. The processing unit 160 adjusts the calculated tip position of the endoscope 20 or the position of the surgical instrument 300 in the virtual endoscopic image G2 based on the image of the surgical instrument 300 included in the real endoscope image. It's okay. The image of the surgical instrument 300 may show relative information (eg, position in the real endoscopic image) of the surgical instrument 300 in the real endoscopic image. As a result, the medical image processing device 100 changes the field of view of the virtual endoscope so that the contents of the real endoscope image and the virtual endoscope image G2 come close to each other (for example, match). The tip position 21 of the endoscope 20 can be adjusted, and the position of the surgical instrument 300 shown in the virtual endoscope image G2 can be adjusted. Therefore, the surgeon can grasp the position of the surgical instrument 300 closer to the real space.

Further, the processing unit 160 may set the target 40 in the 3D data and display the first image by superimposing the information based on the distance between the surgical instrument 300 and the target 40 (for example, the distance d1). As a result, the surgeon can perform various treatments on the target 40 while grasping the distance from the surgical instrument 300 that the surgeon can operate during the operation to the target 40.

Further, the processing unit 160 may set a dangerous part in the 3D data and display the first image by superimposing information based on the distance between the surgical instrument 300 and the dangerous part. As a result, the surgeon can grasp the distance from the surgical instrument 300 that can be operated by the surgeon during the operation to the dangerous part. Therefore, the operator can perform various treatments on the target 40 while avoiding, for example, a dangerous part.

Further, the processing unit 160 acquires the measurement information (for example, the measurement position) measured by the real port, and operates on the first image based on the measurement information of the real port and the position of the virtual port corresponding to the real port. Information indicating the instrument 300 may be superimposed and displayed. As a result, even if there is a discrepancy between the measurement position of the real port and the set position of the virtual port, the medical image processing apparatus 100 adjusts the position of the virtual port so that this discrepancy becomes small, for example, to actually adjust the position of the virtual port. A first image can be provided relative to the position of the real port.

One aspect of the above embodiment is a step of acquiring 3D data of the subject, a step of setting a virtual port on the body surface of the subject in the 3D data, and inserting into the inside of the subject from a real port corresponding to the virtual port. Information indicating the surgical instrument 300 is superimposed on the first image that visualizes the 3D data based on the step of acquiring the insertion distance of the surgical instrument 300 and at least the position of the virtual port and the insertion distance of the surgical instrument 300. This is a method of supporting surgical surgery under a microscope, which has a step of displaying the data on a display unit.

One aspect of this embodiment is a program for causing a computer to execute the above-mentioned arthroscopic surgery support method.

The present disclosure is useful for a arthroscopic surgery support device, a arthroscopic surgery support method, a program, and the like that can simplify the system configuration and easily perform surgical navigation.

10 Liver 15 Bone 21 Endoscope tip position 22 Endoscope tip orientation 23 Imaging range 30, 31, 32 Forceps 40 Target 60 Trocar 70 Body surface 100 Medical image processing device 110 Acquisition unit 120 UI
130 Display 140 Processor 150 Memory 160 Processing unit 161 Area processing unit 162 Deformation processing unit 163 Model setting unit 164 Port information processing unit 165 Surgical instrument information processing unit 166 Image generation unit 167 Display control unit 200 CT device 300 Surgical instrument PT port PT1 Camera Port PT2 Forceps port ID, ID1, ID2, ID21, ID22 Insertion distance

Claims (11)

It is a arthroscopic surgery support device that supports arthroscopic surgery.
Equipped with a processing unit
The processing unit
Acquire 3D data of the subject and
A virtual port is set on the body surface of the subject in the 3D data, and the virtual port is set.
The insertion distance of the surgical instrument inserted inside the subject is obtained from the real port corresponding to the virtual port.
It has a function of superimposing information indicating the surgical instrument on the first image that visualizes the 3D data based on at least the position of the virtual port and the insertion distance of the surgical instrument and displaying it on the display unit.
Mirror surgery support device. The 3D data includes 3D data of a virtual pneumoperitoneum state in which a pneumoperitoneum simulation is performed.
The arthroscopic surgery support device according to claim 1. The processing unit
The insertion direction of the surgical instrument inside the subject is acquired, and the insertion direction is obtained.
Information indicating the surgical instrument is displayed based on the insertion direction of the surgical instrument.
The arthroscopic surgery support device according to claim 1 or 2. The processing unit
The insertion direction of the surgical instrument inside the subject is calculated.
Information indicating the surgical instrument is displayed based on the insertion direction of the surgical instrument.
The arthroscopic surgery support device according to claim 1 or 2. There are two or more of the surgical instruments, at least one of which includes an endoscope.
There are two or more virtual ports, at least one of which includes a virtual camera port into which the endoscope is inserted on the body surface of the subject.
The processing unit
The insertion distance of the endoscope inserted inside the subject is acquired from the real camera port corresponding to the virtual camera port.
The tip position of the endoscope is calculated based on at least the position of the virtual camera port and the insertion distance of the endoscope.
The first image is a virtual endoscope image with the tip position of the endoscope as a viewpoint.
The arthroscopic surgery support device according to any one of claims 1 to 4. The processing unit
An actual endoscopic image, which is an image captured by the endoscope, is acquired.
The position of the tip of the endoscope calculated or the position of the surgical instrument in the virtual endoscopic image is adjusted based on the image of the surgical instrument included in the real endoscopic image.
The arthroscopic surgery support device according to claim 5. The processing unit
Set the target in the 3D data and
Information based on the distance between the surgical instrument and the target is superimposed and displayed on the first image.
The arthroscopic surgery support device according to any one of claims 1 to 6. The processing unit
Set the dangerous part in the 3D data,
Information indicating the distance between the surgical instrument and the dangerous part is superimposed and displayed on the first image.
The arthroscopic surgery support device according to any one of claims 1 to 7. The processing unit
Acquire the measurement information measured by the actual port,
Based on the measurement information of the real port and the position of the virtual port corresponding to the real port, the information indicating the surgical instrument is superimposed and displayed on the first image.
The arthroscopic surgery support device according to any one of claims 1 to 8. Steps to acquire 3D data of the subject and
A step of setting a virtual port on the body surface of the subject in the 3D data,
The step of acquiring the insertion distance of the surgical instrument inserted inside the subject from the real port corresponding to the virtual port, and
A step of superimposing information indicating the surgical instrument on the first image that visualizes the 3D data based on at least the position of the virtual port and the insertion distance of the surgical instrument and displaying it on the display unit.
Mirror surgery support method having. A program for causing a computer to execute the arthroscopic surgery support method according to claim 10.

Images