JP2010200894A - Surgery support system and surgical robot system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for calculating a current positional relation between an object part, such as a tumor of a surgical target, and a surgical instrument, or a predicted positional relation in the future. <P>SOLUTION: A surgery support system includes: a processor 1 composed of a computer; a display device 2 (display) connected to the processor; a position detector 3 for detecting positions of a variety of instruments in an operating room; and an embedded marker detection device 4 for detecting an embedded marker MM embedded in a human body. The processor includes: a storage part 11 for storing target data to be operated; an area coloring model generation part 12; an augmented reality image generation part 13; a position-posture data generation part 14 of an endoscope 5; a position-posture data generation part 15 of the surgical instruments (processing tools); a position-posture data generation part 16 of embedded markers; an surgery navigation data generation part 17; and a display control unit 18. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a surgery support system and a surgery robot system.

  As a system that supports surgery, for example, there is a system for “Augmented reality image guidance” described in Non-Patent Document 1.

  Augmented Reality Image Guidance refers to “collecting 2D image data (volume data) acquired during or before surgery” taken as a digital information into a computer. A 3D image that is useful for surgical planning centered on the target organ is reconstructed on a computer, and its 3D model (computer graphics) is displayed on an endoscopic screen (usually 2 Special computer graphics that superimpose and overlay the 3D image to directly provide the surgeon with 3D information beyond the field of view (actually hidden and invisible) Technology.

  In Non-Patent Document 1, as a three-dimensional model useful for surgical planning, a three-dimensional three-dimensional image of a kidney acquired by CT (Computed Tomography) preoperatively has a tumor area of “red” and a tumor margin of 0 to 5 mm. A four-region, four-color model (four-region, four-color identification surgery planning model) is shown in which the region is “yellow”, the tumor margin is 5-10 mm in “green”, and the tumor margin is more than 10 mm in “blue”. Proposed.

  When the 4 area 4 color discriminating surgery planning model of Non-Patent Document 1 is superimposed on the actual endoscopic video screen and presented to the surgeon, the surgeon is in real-time beyond the operative field (actually Can recognize the presence of tumors.

Osamu Ukimura et al., “Computer-aided Image Guidance for Improving Accuracy, Safety and Speed of Minimally Invasive Surgery”, Urological Surgery, Vol. 21 No11, pp1481-P1487, Medical Books Publishing Co., Ltd., 2008

An object of the present invention is to provide a technique for calculating a current positional relationship between a target site such as a tumor to be a surgical target and a surgical instrument or a predicted future positional relationship.
Another object of the present invention is to provide a technique for presenting the positional relationship to an operator in an easily understandable manner.
Yet another object of the present invention is to accommodate movement / deformation of the target site during surgery.
Still another object of the present invention is to apply the positional relationship to control of robotic surgery.

  According to the present invention, a storage unit for storing target data including image data of a three-dimensional model indicating a target site for surgery, a specific reference position in the surgical instrument, and / or a specific reference position around the surgical instrument, A reference position detector for detecting spatial coordinates located in the space; a target part detector for detecting spatial coordinates where the target part during surgery is located in the surgical space; and the three-dimensional model indicated by the target data Calculating a spatial coordinate to be positioned in the surgical space based on the spatial coordinates of the target part detected by the target part detection part, the three-dimensional model indicated by the target data and the surroundings thereof Of a plurality of regions divided according to the distance from the three-dimensional model, indicating a region where the reference position is located It is surgery supporting system according to claim which includes a data generation unit for generating Activation data.

  Display means for displaying the navigation data on the screen, and the display means includes the reference data among the three-dimensional model indicated by the target data and a plurality of regions obtained by dividing the surrounding area according to the distance from the three-dimensional model. It is preferable to display a screen for indicating a region where the position is located by a color set in each of the plurality of regions.

  The reference position that is the target of detection of the spatial coordinates by the reference position detection unit includes a predicted position of the surgical instrument that is predicted based on the posture and / or the moving direction of the surgical instrument. preferable.

  It is preferable that the reference position includes a plurality of points included on a virtual straight line extending from the surgical instrument.

  The virtual straight line is preferably a straight line that coincides with the longitudinal direction of the elongated surgical instrument.

  The reference position preferably includes a position on the surface of a virtual sphere set around the surgical instrument.

  The reference position includes a position including the position of the surface of a virtual hemisphere set around the surgical instrument, and the virtual hemisphere includes the zenith of the hemisphere from the tip of the surgical instrument. It is preferable that the extending virtual straight line passes.

  The movement and / or deformation of the target part during operation is detected by a plurality of markers embedded in or around the target part, and the three-dimensional model is determined according to the detected movement and / or deformation of the target part. It is preferable to include means for performing a process of moving the coordinates of and / or a process of deforming the three-dimensional model.

  Another aspect of the present invention is a surgical robot system including a control unit that drives and controls movement of the surgical instrument and the surgery support system, and the control unit is generated by the surgery support system. The navigation robot system is characterized in that the navigation data is acquired and the movement of the surgical instrument is driven and controlled based on the acquired navigation data.

  According to the present invention, a current positional relationship between a target site such as a tumor to be a surgical target and a surgical instrument or a predicted future positional relationship is calculated.

It is a figure which shows the whole structure of a surgery assistance system. It is a functional block diagram of a processing apparatus. It is an image figure of the three-dimensional model (organ and tumor) which the target data T shows. It is an image figure of an area | region color classification model. It is a functional block diagram of a surgery navigation data generation part. It is a functional block diagram of a surgical signal data & surgical radar data generation unit. It is a figure which shows a reference position. It is a figure which shows the some area | region inside and outside an object site | part. It is a figure which shows the relationship between a reference | standard position and several area | regions. It is a figure which shows surgical signal data. It is a figure which shows the display screen of surgical signal data. It is a figure which shows the point on the hemispherical surface used as a reference position. It is a figure which shows the projection point q on yz plane. It is a figure which shows the relationship between a hemisphere and several area | regions. It is a figure which shows the display screen of surgical radar data. It is a block diagram of a surgical robot system.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
In the following description, endoscopic surgery will be described as an example, but the surgical method supported by this system is not limited to endoscopic surgery.

[1. Overall configuration of surgery support system]
As shown in FIG. 1, the surgery support system according to the present embodiment includes a processing device 1 composed of a computer, a display device (display) 2 connected to the processing device 1, and positions for detecting the positions of various instruments in the operating room. A detector 3 and an embedded marker detection device 4 for detecting an embedded marker MM embedded in a human body are provided.

The processing device 1 is connected to the position detecting device 3, the embedded marker detecting device 4, and the endoscope 5, and the position detecting camera 3, the embedded marker detecting device 4, and the endoscope Based on the data acquired from 5, the process which produces | generates the navigation data for surgery assistance is performed. Details of the function of the processing apparatus 1 will be described later.
The display device (display device) 2 displays navigation data and the like generated by the processing device on a screen, thereby presenting navigation data and the like to an operator (doctor).

  The position detector 3 is configured by a camera (infrared camera) that detects optical markers OM1, OM2, and OM3 attached to an object whose position is to be detected. In the present embodiment, the optical markers include OM1 attached to the implantable marker detection device 4, OM2 attached to the endoscope 5, and OM3 attached to the surgical instrument 6. The surgical instrument 6 here is a treatment instrument that performs a procedure for surgery such as excision, and is, for example, a scalpel or forceps.

  Each optical marker OM1, OM2, OM3 is configured by attaching a plurality of (three or more) light emitters such as LEDs for one position detection object so that the position and orientation of the position detection object can be detected. Has been. The optical marker may be a passive one that reflects light in addition to an active one that generates light.

The light (or reflected light) of a plurality of light emitters that each of the optical markers OM1, OM2, and OM3 has is acquired by the position detector 3 that includes a camera.
The captured image acquired by the position detector 3 is subjected to image processing by the processing device 1, and the processing device 1 generates data (position / posture data) indicating the position and orientation of each of the position detection objects 4, 5, and 6. Is done. The position of the position detection object is expressed by space coordinates (three-dimensional coordinates) in the space (operating space) of the operating room where the operation support system is arranged.
The detection of the position / posture of the position detection objects 4, 5, 6 is not limited to the optical method, and may be another method such as a magnetic method.

  The implantable marker detection device 4 is for detecting the positions of a plurality of implantable markers MM, MM embedded in the body (particularly, an organ to be operated). As such an embedded marker detection device 4, a Calypso 4D localization system (trademark) of Calypso Medical Technologies can be used. Such an embedded marker detection device 4 includes a magnetic field generation device 4a that generates a magnetic field and a sensor unit 4b that detects the magnetic field.

  As the embedded marker MM, a wireless marker having a size of about several millimeters called a beacon electromagnetic transponder used in the calypso 4D localization system can be used. This wireless marker (embedded marker MM) has a coil and necessary circuits, and as a response signal for specifying the position and orientation of the wireless marker (embedded marker MM) by the magnetic field of the magnetic field generator 4a. A magnetic signal is excited.

  The sensor unit 4 b detects the response signal of each of the plurality of embedded markers MM, and the detected response signal is given to the processing device 1. The processing device 1 generates data (position / posture data) indicating the position and posture of each embedded marker MM based on the detected response signal.

The position / posture of the embedded marker MM detected by the embedded marker detection device 4 is relative to the position of the embedded marker detection device 4.
However, since the optical marker OM1 is provided in the implantable marker detection device 4, the position / posture of the implantable marker MM detected by the implantable marker detection device 4 is determined by the operation support system in the processing device 1. Is converted into a position / posture in the space (operating space) of the operating room in which is placed.

  The implantable marker detection device 4 is for detecting the position and deformation of the organ including the target site for surgery. The means for detecting the position and deformation of the organ including the target site for surgery is provided by this means. However, the present invention is not limited to such an embedded marker detection device 4. For example, it may be one that irradiates an organ with laser light and detects the position / deformation of the organ from the reflected light.

[2. Configuration of Processing Device 1]
FIG. 2 shows functions of the processing apparatus 1. The processing device 1 is configured by a computer having a CPU and a storage device. A computer program for realizing the functions of the processing device 1 shown in FIG. 2 is installed in the computer constituting the processing device 1. All the functional units of the processing device 1 described below are exhibited when the computer program is executed by a computer.

  The processing device 1 includes a storage unit 11 for storing target data, an area color classification model generation unit 12, an augmented reality image generation unit 13, a position / posture data generation unit 14 of an endoscope 5, a surgical instrument (treatment tool). 6, a position / posture data generation unit 15, a position / posture data generation unit 16 for an implantable marker, a surgical navigation data generation unit 17, and a display control unit 18.

  The target data T is configured as three-dimensional image data (three-dimensional volume data) of a three-dimensional model indicating the overall shape and the like of a specific organ (such as a kidney) including an operation target site (such as a tumor). The target data T shown in FIG. 3 is a three-dimensional model image data T1 indicating the entire shape of the organ, and a tumor region (target region) where a tumor (target region) to be excised is seen from the whole organ. 3D model image data T2 for specifying the position and shape.

  This target data T is generated based on CT image data taken before surgery. The target data T can be generated by constructing and using a system for automatically generating CT image data taken before surgery, or by computer assistance.

  In order to generate the target data T, first, before the operation, a plurality of the embedded markers MM are embedded in or around a specific organ including the target site for the operation. When detecting only the position of an organ, the number of markers MM to be embedded may be about 3 to 4, but it is preferable that the number of markers MM is larger (for example, about 30) in order to detect the deformation of the organ.

  Next, CT image data of the patient in which the implantation marker MM is implanted in the body is acquired. Then, from the CT image data, an organ including an operation target site is specified, and three-dimensional image data T1 of a three-dimensional model indicating the shape of the organ is generated. In addition, the position and shape of the tumor in the organ are specified from the CT image data, and three-dimensional image data T2 of a three-dimensional model for specifying the position and shape of the tumor is generated.

Further, the position and orientation of each embedded marker MM in the three-dimensional image data T1 (and three-dimensional image data T2) is specified from the image of the embedded marker MM present in the CT image, and the three-dimensional image data T1 (and The position / posture data T3 of the embedded marker MM in the three-dimensional image data T2) is included in the target data.
Note that when the image of the embedded marker MM is included in the three-dimensional image data T1 indicating the shape of the organ, the image of the embedded marker MM indicates the position / posture of the embedded marker MM. Data T3 indicating the position / posture of the marker MM may not be generated separately from the three-dimensional image data T1.

The target data T generated as described above is stored in the target data storage unit 11 of the processing apparatus 1 before the operation.
The image that is the source of the target data T is not limited to a CT image, but by MRI (Magnetic Resonance Imaging), FMRI (Functional MRI), PET (Positron Emission Tomography), ultrasonic tomography, scintigram, etc. The obtained image may be used.

  The area color classification model generation unit 12 generates an area color classification model (4 area 4 color identification surgery planning model) shown in FIG. 4 from the target data T in the target data storage unit 11. The region color coding model shown in FIG. 4 indicates that the tumor region indicated by the three-dimensional image data T2 of the target data T is “red”, the tumor margin 0-5 mm region is “yellow”, and the tumor margin 5-10 mm region is “ It is a three-dimensional model in which “green” and a tumor margin region of more than 10 mm are indicated by “blue”.

  In order to generate this region color coding model, in the three-dimensional image data T1 indicating the whole organ shape of the target data T, the portion of the three-dimensional image data T2 indicating the tumor is set to “red”, and the three-dimensional image data T2 is set. The range from 5 mm to 5 mm from the tumor surface indicated by is set to “yellow”, and the range from 5 to 10 mm from the tumor surface indicated by the 3D image data T2 is set to “green”, and the 3D image data T2 is set. The range of more than 10 mm from the tumor surface indicated by may be set to “blue”.

  In the present embodiment, the processing device 1 is configured to generate a region color coding model from the target data T stored in the target data storage unit 11, but for displaying an augmented reality image. The region color classification model generated before the operation may be stored in advance in the storage unit of the processing device.

The augmented reality image generation unit 13 superimposes (superimposes) the region color coding model shown in FIG. 4 and the video imaged by the endoscope 5 and displays the augmented reality on the display device 2. Generate a feeling image.
If the surgeon advances the excision within the "green" range, it should be attached to the tumor side with sufficient cancer-negative excision area (yellow area) and preserve the maximum renal function (blue area). Can do.
The augmented reality image is displayed on the display device 2 together with later-described surgical navigation data (surgical signal data SD and surgical radar data RD).

  The position / posture data generation unit 14 of the endoscope 5 and the position / posture data generation unit 15 of the surgical instrument 6 are detected by the position detection objects 5 and 6 based on the images acquired by the position detector 3 (camera). Data (position / posture data) indicating the position and posture of a certain endoscope 5 and surgical instrument 6 is generated. This position / posture data is expressed as the position / posture of the coordinate system in the space (surgical space) of the operating room in which the surgery support system is arranged.

In addition, the position / posture data generation unit 16 of the embedded marker MM includes the position / posture data of each of the plurality of embedded markers MM detected by the embedded marker detection device 4 and the embedded marker detected by the position detector 3. Based on the position / posture data of the embedded marker detection device 4, position / posture data of each of the plurality of embedded markers MM in the coordinate system in the operating room space (surgical space) is generated.
As a result, the position / posture of each of the plurality of implantable markers MM are represented in the same coordinate system as the coordinate system of the position / posture data of the endoscope 5 and the surgical instrument 6.

Here, since the position / posture data of the embedded marker MM indicates the position / shape of the embedded marker MM or its vicinity, the position / posture data of the embedded marker MM is the target part ( Tumor) indicates the position and shape in the coordinate system in the operating room space (surgical space).
That is, the position / posture data generation unit 16 of the embedded marker MM functions as a coping site detection unit that detects spatial coordinates where the target site during surgery is positioned in the surgical space.

The position / posture data generated by the data generators 14, 15, 16 is given to the surgical navigation data generator 17.
Based on the target data T stored in the target data storage unit 11 and each position / posture data, the surgical navigation data generation unit 17 determines the surgical instrument 6 and the surgical target site (tumor) indicated by the target data. Navigation data indicating the positional relationship is generated. In the present embodiment, surgical signal data SD and surgical radar data RD are generated as the navigation data.

[3. Generation of surgical navigation data]
As shown in FIG. 5, the surgical navigation data generation unit 17 includes a movement / deformation processing unit 21 for target data T, and a surgical signal data & surgical radar data generation unit 22.

  The target data movement / deformation processing unit 21 performs intraoperative based on the target data T and the position / posture data of the plurality of embedded markers MM from the target data storage unit 11 and the position / posture data generation unit 16 of the embedded marker. Target data (hereinafter referred to as corrected target data) corresponding to the position / shape of an organ (a living organ to be operated) is generated.

  In the corrected target data, the positions / postures of the plurality of embedded markers MM in the target data T are the positions indicated by the position / posture data of the plurality of embedded markers MM generated by the position / posture data generation unit 16 of the embedded marker. The three-dimensional image data T1 of the organ indicated by the target data T is obtained by calculating the spatial coordinates and shape to be located in the spatial coordinates in the operating room so as to match the posture.

More specifically, the movement / deformation processing unit 21 includes the positions / postures of the plurality of embedded markers MM in the target data T and the plurality of embedded markers generated by the position / posture data generation unit 16 of the embedded marker. The coordinates at which the organ indicated by the three-dimensional image data T1 of the target data T should be positioned in the spatial coordinate system so that the error between the position and orientation indicated by the position and orientation data of the MM is minimized for each embedded marker MM. And the shape of the organ indicated by the three-dimensional image data T1 is deformed as necessary.
As an algorithm for minimizing the error, a known algorithm such as a least square method can be minimized.

When the target organ does not move / deform, the movement / deformation processing of the organ in the movement / deformation processing unit 21 can be omitted. Further, the movement / deformation processing unit 21 calculates only the position in the spatial coordinate system of the three-dimensional model (tumor) indicated by the three-dimensional image data T2 in the target data T, and may omit the deformation processing of the organ or the like. Good.
Thus, it is sufficient for the movement / deformation processing unit 21 to function as a calculation unit that calculates the spatial coordinates at which the three-dimensional model (tumor) indicated by the target data T should be located in the surgical space.

  Here, since the position and shape of the 3D image data T1 indicating the organ and the 3D image data T2 indicating the tumor are associated with each other, the processing unit 21 determines the position / organ of the organ indicated by the 3D image data T1. When the shape is calculated, the position / shape of the tumor indicated by the corresponding data T2 is also calculated.

As described above, the corrected target data indicating the position and shape of the organ and tumor (the target site for surgery) indicated by the target data T in the surgical space in real time during surgery is obtained.
The corrected target data is given to the surgical signal data & surgical radar data generation unit 22.

  The surgical signal data & surgical radar data generation unit 22 is based on the corrected target data and the position / generation data of the surgical instrument 6 generated by the position / posture data generation unit 15 of the surgical instrument 6. Radar data RD is generated.

[3.1 Calculation of main reference position and unit vector]
As shown in FIG. 6, the surgical signal data & surgical radar data generation unit (hereinafter simply referred to as “data generation unit”) 22 is a three-dimensional unit for calculating the distal point p ₀ of the surgical instrument 6 as the main reference position. A coordinate calculation unit (reference position detection unit) 24 is provided.
The three-dimensional coordinate calculation unit 24 is for calculating a specific main reference position from the position / posture data of the surgical instrument 6, and the main reference position is the tip of the surgical instrument 6 as shown in FIG. In addition to the point p ₀ , another position on the surgical instrument 6 or a position around the surgical instrument 6 may be used, but the surgical instrument 6 is at or near the position where the surgical instrument 6 acts on the target site for surgery for incision or the like. Is preferred.

The data generating unit 22, a position where the optical markers OM3 is provided in the surgical instrument 6, and information indicating a positional relationship between the main reference position p ₀ is set in advance, the calculation unit 24, a surgical instrument 6 and position and orientation data (in optical marker OM3), on the basis of preset positional relationship information, calculates the main reference position (tip point) p _0, 3-dimensional coordinates in the operative space (spatial coordinates).

In addition, the data generation unit 22 includes a calculation unit 25 that calculates a unit vector u indicating the longitudinal axis direction 1 of the surgical instrument 6 at the main reference position (tip point) p ₀ . The calculation unit 25 can calculate by calculating the longitudinal axis direction 1 (posture of the surgical instrument 6) of the elongated surgical instrument 6 from the position / posture data of the surgical instrument 6.

  In this embodiment, the unit vector u indicates the longitudinal axis direction l of the surgical instrument 6 at the main reference position (surgical instrument tip point), but the direction indicated by the unit vector is limited to the longitudinal axis direction l. It is not necessarily provided and may indicate other directions. Further, the direction indicated by the unit vector may indicate not only the direction uniquely determined from the posture of the surgical instrument 6 as in the longitudinal axis direction but also the moving direction of the surgical instrument (the tip thereof). Moreover, although it is easy to predict the movement locus | trajectory estimated as a straight line, if possible, the movement locus | trajectory shown with a curve may be estimated.

Further, the data generation unit 22 includes a calculation unit (reference position detection unit) 26 for calculating three-dimensional coordinates (spatial coordinates) of other points (sub-reference positions) p _{1 to} p _M. The other points p _{1 to} p _M calculated by the calculation unit 26 are distances k _i (i: 1) from the main reference position (tip point) p _{0 in the} direction indicated by the unit vector u (longitudinal axis direction 1). It is set as a point (spatial coordinates in the surgical space) that is about (an integer of .about.M).
The distance k _i can be adjusted by the point-to-point distance k _i adjustment unit 27 in the data generation unit 22, but may be a fixed value set in advance. It will be described in detail later point distance k _i adjustment.

In the present embodiment, the main reference position p ₀ has the significance of indicating the “current” position of the distal end of the surgical instrument 6, whereas the other reference positions are sub-reference positions p ₁ to p ₁ . p _M indicates a predicted position where the main reference position p ₀ is predicted to be positioned in the “future”. Accordingly, the main reference position p ₀ is preferably on the surgical instrument 6 or very close to the periphery of the surgical instrument 6, and the sub-reference position is p ₁ in view of the longitudinal axis direction and the moving direction of the surgical instrument 6. ˜p _M is preferably set as a position where the main reference position p ₀ is predicted to be located in the near future.

The reference position p _i (i: 0 to M) calculated as described above is a point located on a virtual straight line (predicted movement trajectory of the surgical instrument) in the direction indicated by the unit vector u. The position and direction of the virtual straight line extending from the surgical instrument 6 are not particularly limited, but as described above, the longitudinal direction from the distal end (main reference position) of the surgical instrument 6 Those extending in the range are preferred. Note that the reference position may be set on the predicted movement locus indicated by the curve.
As shown in FIG. 7, in the present embodiment, M = 2, and three reference positions, p ₀ , p ₁ , and p ₂ , are set. The reference position number i can be adjusted by the user of the system, but may be a fixed value set in advance.

The reference position p _i (i: 0 to M) is a point that is a target for determining the distance from the target site (tumor) of the operation.

[3.2 Multiple areas according to the distance from the target part]
Here, the data generation unit 22 includes a determination unit 28 for determining the three-dimensional region S _j (j: 0 to 3) to which the respective reference positions (points) p _{0 to} p _M belong. The three-dimensional regions S _{0 to} S ₃ are divided according to the distance from the surface of the tumor region (target site) indicated by the corrected target data. When there is no movement / deformation of the organ, target data T before correction stored in the storage unit 11 may be used instead of the correction target data.

Of these areas S ₀ to S _3, the area S ₀ is a three-dimensional region corresponding to the three-dimensional image data T2 showing a tumor in the modified target data, area S ₁ to S _3, the tumor (target site) The outer three-dimensional region is divided into a plurality (here, three). Note that the number j of areas including the area S ₀ is not particularly limited and may be plural. The number of areas j can be adjusted by the system user, but may be a fixed value set in advance.

As shown in FIG. 8, the size / range of the region S ₀ of the target part corresponds to the size / range of the target part indicated by the three-dimensional image data T2 in the target data T. The sizes of the regions S _{1 to} S ₃ outside the target region are represented by margins w ₁ and w ₂ . In the present embodiment, the region S ₁ refers to three-dimensional area a minimum distance D include point is greater w ₁ less than 0 up area S ₀ representing the target site (tumor), the area S _2, the subject A three-dimensional region including a point where the minimum distance D to the region S ₀ indicating the region (tumor) is greater than w _{1 and} equal to or less than w ₂ is included, and the region S ₃ is up to the region S ₀ indicating the target region (tumor). Is a three-dimensional region in which the minimum distance is greater than w ₂ .
When w ₁ = 5 mm and w ₂ = 10 mm are set, the method of dividing the regions S _{0 to} S ₃ matches the region color classification model shown in FIG.

The margins w ₁ and w ₂ can be adjusted by the margin adjustment unit 34, but may be fixed values set in advance. The margin adjustment will be described later.

[3.3 Surgical signal data]
As shown in FIG. 9, the judging unit 28, for each reference position (point) p _i, the minimum distance D (p _i, S ₀₎ from the region S ₀ seek, each reference position (point) p _i , It is determined which of the areas S _j (j: 0 to 3) belongs to.
In the case of FIG. 9, it is determined that the main reference position p ₀ is in the region S ₂ , the first sub reference position p ₁ is in the region S ₂ , and the second sub reference position p ₂ is in the region S ₁ .

FIG. 10 shows the surgical signal data SD generated by the above determination. This surgical signal data includes, for each reference position p _i , information (Area) indicating the area to which each reference position belongs, number information (j) of the area to which each reference position belongs, and color information (Cj for the area to which each reference position belongs. ).
As the color of the area, the area s ₀ "red", the area s ₁ "yellow", the area s ₂ "green", the area s ₃ is set with "blue".

FIG. 11 shows a screen (surgical signal screen) D1 when the surgical signal data SD of FIG. 10 is displayed on the display device 2. The screen D1 is generated by the display control unit 18 of the processing device 1 based on the surgical signal data SD.
The screen D1 includes a plurality of circular display units arranged in a matrix arrangement in which the reference positions p _{0 to} p ₂ are arranged in the vertical axis direction and the regions S _{0 to} S ₃ are arranged in the horizontal axis direction. Each display unit in the screen D1, for each reference position p ₀ ~p _2, which reference position p ₀ ~p ₂ corresponds to the area S ₀ to S _3, which belongs, configured to light up in a set color Has been.

For example, in the state of FIGS. 9 and 10, since the reference position p ₀ belongs to the region S ₂ , the display unit C 1 at the position (p ₀ , S ₂ ) is set to “green” set in the region S _2. "Lights up. The display portions for the areas S ₀ , S ₁ , S _{3 to} which the reference position p ₀ does not belong are non-lighting displays (displays in black or white). Further, since the reference position p ₁ also belongs to the region S ₂ , the display unit C _{2 at} the position (p ₁ , S ₂ ) lights up with “green” set in the region S ₂ . The display sections for the areas S ₀ , S ₁ , S _{3 to} which the reference position p ₁ does not belong are non-lighting displays (displays in black or white). In addition, since the reference position p ₂ belongs to the area S ₁ , the display unit C3 located at the position (p ₂ , S ₁ ) lights up in “yellow” set in the area S ₁ . The display portions for the areas S ₀ , S ₂ , S _{3 to} which the reference position p ₂ does not belong are non-lighting displays (displays in black or white).

The surgeon who viewed the screen D1 in FIG. 11 shows to which region S _{0 to} S ₃ the current position p ₀ of the distal end of the surgical instrument 6 belongs, that is, to what position as viewed from the tumor S _0. Can be easily grasped visually.
Here, when the margins w ₁ and w ₂ for determining the sizes of the areas S _{1 to} S ₃ are set in the same manner as the area color coding model, the operator should excise the area S ₂ set to green. This is the area to which the ablation line belongs. The surgeon advances the excision in the region S ₂ set in green so that a sufficient cancer-negative excision marginal region (yellow region s ₁ ) is attached to the tumor side and the blue region S ₃ is maximized. Can be kept warm.
In the screen D1 of FIG. 11, the current position p ₀ of the distal end of the surgical instrument 6 indicates that the current position p ₀ is located in an appropriate region S ₂ for resection. The position can be easily grasped as appropriate.

Since the other reference positions p ₁ and p ₂ are predicted positions at which the current position p ₀ is predicted to be in the future, when the operator views the screen D1 in FIG. It is possible to easily grasp whether the moving direction is appropriate in the future.
That is, as shown in FIG. 11, when p ₂ (or p ₁ ), which is a predicted position in the future, indicates that the tumor side region s ₁ (or region S ₀ , S ₃ ) belongs, The operator can grasp in advance in the posture and movement direction of the surgical instrument 6 that there is a possibility of deviation from an appropriate state in the future.
On the other hand, if the screen D1 displays that all of p ₀ , p ₁ , and p ₂ belong to the region S ₂ , the surgeon can grasp that the current posture and moving direction of the surgical instrument 6 are appropriate. In this state, the surgery can be appropriately performed.

[3.4 Surgical radar data]
As shown in FIG. 6, in addition to the function of generating the surgical signal data SD, the data generation unit 22 generates the surgical radar data RD, and therefore the surface coordinate calculation unit 29 of the hemisphere H centered on the reference position p _0. And a determination unit 30 for the regions S _{0 to} S ₃ to which the surface of the hemisphere H belongs.

The surgical radar data RD does not indicate the areas S _{0 to} S ₃ to which the points (reference positions) belong, as in the case of the surgical signal data SD, but each position included in the surface as a set of points (reference positions) belongs. Regions S _{0 to} S ₃ are shown.

In the present embodiment, a spherical surface is adopted as the surface to be determined for the regions S _{0 to} S ₃ to which it belongs. Furthermore, in the present embodiment, a hemispherical surface is adopted as a partial surface of the spherical surface, not the entire spherical surface, as the surface to be determined for the regions S _{0 to} S ₃ to which it belongs. The surface that is the determination target of the regions S _{0 to} S ₃ to which the region belongs is not limited to a spherical surface or a hemispherical surface, and can be a surface of any shape (a flat surface or a curved surface).

In the present embodiment, a hemispherical surface to be determined in the region S ₀ to S ₃ belonging the direction indicated by the unit vector u (longitudinal l) is the surface of the hemisphere passing through the apex (zenith).
That is, as shown in FIG. 12, the reference position p ₀ is the origin, the plane perpendicular to the direction of the unit vector u (longitudinal axis direction 1) is defined as the yz coordinates (z and y are orthogonal), and the unit vector u Assuming an xyz three-dimensional Cartesian coordinate system with the x coordinate as the direction (longitudinal axis direction l), the hemisphere according to the present embodiment is such that the x axis of the xyz three-dimensional Cartesian coordinate system is the apex (zenith) p of the hemisphere. _It passes through _i, and the bottom of the hemisphere (the great circle of the sphere) exists on the yz plane where x = 0.

The h any point on the hemisphere surface is expressed by _{h = k i r (θ,} φ). Here, k _i is the radius of the hemisphere, and r (θ, φ) is a unit vector at p ₀ . θ represents the azimuth angle formed by the unit vector r with respect to the unit vector u (unit vector from p ₀ to p _i ), and φ represents the elevation angle formed by the unit vector r with respect to the unit vector. However, the direction of the unit vector u (x-axis direction) is φ = 0. Here, in the hemisphere, 0 ≦ θ <2π, − (π / 2) ≦ φ <(π / 2).
Accordingly, r (θ, φ) = (cosφ, cosθsinφ, sinθsinφ) is, the xyz3 dimensional orthogonal coordinate system of the coordinate of an arbitrary point h on the semi-spherical _{surface, (k i cosφ, k i} cosθsinφ, k i sinθsinφ ).

The calculation unit 29 for calculating the coordinates of the surface of the hemisphere is based on the first reference position p ₀ and the unit vector u in the range of 0 ≦ θ <2π, − (π / 2) ≦ φ <(π / 2). r (θ, φ) is calculated, and the three-dimensional coordinates of each point (reference position) h on the hemisphere are calculated.

Note that the radius k _i of the hemisphere can be adjusted by the adjusting unit 30. The radius adjustment unit 30 determines any one of the distances k _i set by the point-to-point distance k _i adjustment unit 27 as the radius k _i of the hemisphere by the system or the user of the system. Incidentally, the radius k _i of the hemisphere may be determined independently of the distance k _i which is set by a point distance k _i adjusting unit 27.

Here, the hemispherical surface, which is a set of the reference positions h, may be located in the future at the current position p ₀ of the surgical instrument 6 (the tip thereof), similarly to the sub-reference positions p ₁ and p ₂ in the surgical signal data SD. It has the significance of indicating a sex position.
That is, the sub-reference positions p ₁ and p ₂ in the surgical signal data SD are points on a straight line in the direction indicated by the detected unit vector u (longitudinal direction or moving direction of the surgical instrument). The hemispherical surface is a case where the longitudinal direction or moving direction of the surgical instrument changes from the current direction u in a range of {0 ≦ θ <2π, − (π / 2) ≦ φ <(π / 2)}. A set of predicted positions where the current position p ₀ may be located is shown.
Since the hemisphere is arranged in front of the surgical instrument in the longitudinal axis direction or movement direction, it is appropriate as a set of predicted positions where the current position p ₀ may be located.
In the present embodiment, the entire hemispheric surface is targeted as a set of predicted positions, but may be a part of the hemispheric surface.

The region determination unit 31 determines, for each point (reference position) h on the hemispherical surface, the regions S _{0 to} S ₃ to which each point h belongs. This determination is performed in the same manner as the determination for the surgical signal data SD.

By the region determination in the region determination unit 31, three-dimensional surgical radar data RD indicating which region S _{0 to} S ₃ each position h on the surface of the hemisphere belongs to is generated. The surgical radar data RD includes the three-dimensional coordinates of each position h on the surface of the hemisphere and information indicating the areas S _{0 to} S ₃ to which each position belongs, as in the case of the surgical signal data SD.

  Although the three-dimensional surgical radar data RD may be displayed three-dimensionally, in consideration of the convenience when the two-dimensional display device 2 is adopted, the data generation unit 22 of the present embodiment is configured with the three-dimensional surgical radar data RD. Is projected onto a two-dimensional plane (bottom of hemisphere; great circle of sphere; yz plane with x = 0).

The two-dimensional projection processing unit 32 projects all the positions h in the three-dimensional surgical radar data RD onto the bottom surface of the hemisphere (the great circle of the sphere; the yz plane with x = 0), thereby producing the two-dimensional surgical radar data RD. Is generated.
As shown in FIG. 13, any projection point q in the yz plane (x = 0) is a projection plane, the distance from the origin 0 (the position of the p ₀₎ is k _i sin [phi, the angle formed with the z-axis θ.
Therefore, the projection point q (θ, φ) when each position h (θ, φ) on the surface of the hemisphere is projected onto the bottom surface (yz plane) of the hemisphere is q = (k _i cos θsinφ, k _i sinθsinφ). ). Note that {0 ≦ θ <2π, − (π / 2) ≦ φ <(π / 2)}.

The two-dimensional projection processing unit 32 indicates, as the two-dimensional surgical radar data RD, the coordinates of each projection point q and the areas S _{0 to} S ₃ to which each projection point q (position h corresponding to each projection point q) belongs. And information. The surgical radar data RD has the same format as the surgical radar data SD shown in FIG. 10, and three reference positions p ₀ , p ₁ and p ₂ of the surgical radar data RD are many in the bottom surface (yz plane) of the hemisphere. Is the projection point q.

  The two-dimensional surgical radar data (ambient data) RD and the display control unit 18 of the processing device 1 generate a screen D2 as shown in FIG.

As shown in FIG. 14, when the reference position p ₂ in the surgical data SD is adopted as the apex p _i of the hemisphere (hemisphere radius = k ₂ ), a screen D1 showing the two-dimensional surgical data RD is shown in FIG. It becomes like this.
FIG. 15 shows the color corresponding to the region to which each position h on the surface of the hemisphere that is a part of the sphere SQ centered on the tip p ₀ of the surgical instrument 6 in FIG. 14 on the yz coordinates with the tip p ₀ as the origin. It is represented by.

Operator who viewed screen D2 in FIG. 15, moving the surgical instrument 6 in the current position to the moving direction, the future predicted position p ₂ of the current position p ₀ is too close slightly in the region S ₁ becomes the tumor area S ₀ It can be seen that if the posture or movement direction of the surgical instrument 6 is changed downward in FIG. 15, it further approaches the tumor region S ₀ .

On the other hand, in FIG. 15, above the estimated position p2 as viewed from the current posture to the moving direction of the surgical instrument 6, since the area S ₂ which is suitable for ablation has spread, the attitude or direction of movement of the surgical instrument 6, It is preferable to change upward in FIG. By presenting the surgical radar display as shown in FIG. 15 to the surgeon, the surgeon can easily recognize the appropriate posture or moving direction of the surgical instrument 6.

  Since the surgical radar display screen D2 as shown in FIG. 15 is presented to the surgeon simultaneously with the surgical signal display screen D1 of FIG. 11, the surgeon looks at both display screens D1 and D2, and sees the surgical instrument. 6 and the current positional relationship between the tumor (target site) and the future positional relationship can be more easily grasped. Further, since both the display screens D1 and D2 are displayed at the same time as the display of the augmented reality image data, the surgeon can recognize the state of the operative field in combination by viewing these displays.

Further, as shown in FIG. 6, the data generation unit 22 calculates the distance from the surgical instrument 6 (the tip p ₀ thereof) to the center (target area) S ₀ indicating the tumor as other surgical radar data. A calculation unit 33 is provided. The calculation unit 33 calculates the positional relationship (distance data) between the surgical instrument 6 and the center of the tumor region S ₀ . This distance data is displayed on the screen together with other data and presented to the operator.

[3.5 Automatic scale adjustment]
The distance data calculated by the calculation unit 33 includes the adjustment of the inter-point distance k _i by the inter-point distance adjustment unit 27, the adjustment of the radius k _i by the hemispherical radius adjustment unit 30, and / or the margin w ₁ by the merge adjustment unit 34. , W ₂ is used for adjustment.
When the distance between the surgical instrument 6 and the tumor region S ₀ center is large, each adjustment unit 27, 30, 34 relatively increases each value to be adjusted, and the distance between the surgical instrument 6 and the tumor region S ₀ center is When the value is small, each value to be adjusted is relatively small.
Thereby, when the surgical instrument 6 is away from the tumor region S ₀ , the surgical signal data SD or the surgical radar data RD can be displayed on a relatively large scale, and when the surgical instrument 6 approaches the tumor region S _0. Each value can be automatically scaled down to provide detailed information.
The adjustment of each value described above may be performed not only by automatic adjustment by the system but also by user selection.

[4. Surgical robot system]
FIG. 16 shows a surgical robot system having the surgical operation support system 50. This surgical robot system includes a surgical instrument 51, a drive unit 52 that drives the surgical instrument 51, and a control unit 53 that controls driving so that the surgical instrument 51 performs an operation for surgery.

The control unit 53 is connected to the surgery support system 50 (the processing device 1 thereof), and acquires navigation data (surgical signal data and / or surgical radar data) generated by the processing device 1.
Since the control unit 53 that has acquired the navigation data can grasp the current or future positional relationship between the surgical treatment site (tumor) and the surgical instrument 51 based on the navigation data, the drive control of the surgical instrument 51 is appropriately performed. It can be carried out.
For example, when the control unit 53 detects that the current or future position with the surgical instrument 51 becomes the tumor region S ₀ or the tumor vicinity region S ₁ , the control unit 53 performs safety control to forcibly stop the operation of the surgical instrument 53. be able to.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation is possible.

11 Storage Unit 16 Implanted Marker Position / Posture Data Generation Unit (Target Part Detection Unit)
21 Movement / deformation processing unit (calculation unit)
22 Surgical Signal Data & Surgical Radar Data Generator (Data Generator)
24 3D coordinate calculation unit (reference position detection unit) of main reference position
26 Sub-reference position three-dimensional coordinate calculation unit (reference position detection unit)

Claims (9)

A storage unit for storing target data including image data of a three-dimensional model indicating a target site for surgery;
A reference position detector for detecting a spatial coordinate where a specific reference position in the surgical instrument and / or a specific reference position around the surgical instrument is located in the surgical space;
A target part detection unit for detecting a spatial coordinate where the target part during surgery is located in the surgical space;
A calculation unit that calculates, based on the spatial coordinates of the target part detected by the target part detection unit, the spatial coordinates that the three-dimensional model indicated by the target data should be located in the surgical space;
A data generation unit that generates navigation data indicating a region where the reference position is located among a plurality of regions obtained by dividing the three-dimensional model indicated by the target data and the periphery thereof according to a distance from the three-dimensional model;
An operation support system characterized by comprising: Comprising display means for displaying the navigation data on a screen;
The display means includes a plurality of regions in which the reference position is located among a plurality of regions obtained by dividing the three-dimensional model indicated by the target data and the periphery thereof according to a distance from the three-dimensional model. The operation support system according to claim 1, wherein a screen for displaying in a color set in is displayed. The reference position that is the detection target of the spatial coordinates by the reference position detection unit includes a predicted position of the surgical instrument that is predicted based on the posture and / or the moving direction of the surgical instrument. The surgery support system according to claim 1 or 2.   The surgical support system according to claim 1, wherein the reference position includes a plurality of points included on a virtual straight line extending from the surgical instrument.   The surgical support system according to claim 4, wherein the virtual straight line is a straight line that coincides with a longitudinal direction of the elongated surgical instrument.   The surgery support system according to any one of claims 1 to 3, wherein the reference position includes a position on a surface of a virtual sphere set around the surgical instrument. The reference position includes a position that includes the position of the surface of a virtual hemisphere set around the surgical instrument,
The surgical support system according to claim 1, wherein the virtual hemisphere is configured such that a virtual straight line extending from a distal end of the surgical instrument passes through the zenith of the hemisphere.   The movement and / or deformation of the target part during operation is detected by a plurality of markers embedded in or around the target part, and the three-dimensional model is determined according to the detected movement and / or deformation of the target part. The surgery support system according to any one of claims 1 to 7, further comprising means for performing a process of moving the coordinates of and / or a process of deforming the three-dimensional model. A surgical robot system comprising: a control unit that drives and controls movement of the surgical instrument; and the surgical operation support system according to any one of claims 1 to 8,
The said control part acquires the navigation data produced | generated by the said surgery assistance system, Based on the acquired navigation data, drive control of the movement of the said surgical instrument is characterized.

Images