JP2002017751A - Surgery navigation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a surgery navigation device capable easily providing detailed inner tissue information near a surgery tool from a tomogram even when the entire tomogram required for navigating the surgery tool to a lesion is displayed. SOLUTION: This surgery navigation device comprises the surgery tool used for observing and treating a specimen, a three-dimensional position attitude measuring means for measuring three-dimensional position attitudes of the specimen and the surgery tool, a storing means for storing tomogram information of the specimen, a tomogram extracting means for extracting a tomogram from the tomogram information stored in the storing means based on the three- dimensional position attitude information of the specimen and the surgery tool, a projection image adding means for adding the position attitude information of the surgery tool to the extracted tomogram, a distance measuring means for measuring a distance between the specimen and the surgery tool, a projection image control means for changing size of an image outputted from the projection image adding means based on the distance information from the distance measuring means, and an image displaying means for displaying an image outputted from the projection image control means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surgical navigation device, and more particularly to a surgical navigation device used for a surgical operation or the like.

[0002]

2. Description of the Related Art A surgical navigation system not only displays the position and posture of a surgical tool with respect to a surgical subject such as a patient, but also displays internal tissue information near a target site to be observed, treated, and watched during a surgical procedure. There is a need to.

[0003] By ensuring that the operator grasps the internal tissue information, it becomes possible to distinguish an affected part to be treated, a blood vessel, a normal tissue, etc. to be treated, and to perform an accurate treatment. Can be.

[0004] For example, "Endoscopic surgery navigation system by real-time perspective view display" (Transactions of the Virtual Reality Society of Japan 2nd Conference, pp. 107-110, 1)
997) introduces a navigation method for displaying the current position of a surgical instrument on a tomographic image with a cross cursor as a method adopted in many existing surgical navigation systems.

According to this method, first, tomographic images in three directions (axial, coronal, and sagittal directions) are selected or generated from the tomographic image information of the patient and the distal end position of the surgical tool.

[0006] Then, a cross cursor indicating the distal end of the surgical tool is added and displayed on the three-dimensional tomographic image.

The operator recognizes the three-dimensional position of the surgical tool from the positional relationship of the cross cursor added to each tomographic image, and obtains necessary internal tissue information from the tomographic image near the cross cursor.

Further, a "three-dimensional image display device" (Japanese Patent Laid-Open No.
In Japanese Patent Application Laid-Open No. 1-110588, a viewpoint is set inside three-dimensional tomographic image information of a subject, an image obtainable from the viewpoint, and a guide image enabling recognition of the viewpoint position on the tomographic image. Is displayed.

According to this method, the user can obtain internal tissue information near the viewpoint position from the endoscopic image, and the guide image allows the viewpoint of the endoscopic image to be located at any position inside the observation target. You can know if there is.

[0010]

SUMMARY OF THE INVENTION The above-mentioned document, "Endoscopic surgery navigation system by real-time perspective view display"
With the method of adding and displaying a cross cursor indicating the tip of a surgical instrument on a three-dimensional tomographic image introduced as a conventional technique, the position of the tip of the endoscope with respect to the patient can be grasped.

However, in order to obtain detailed internal tissue information near the distal end of the endoscope, which is particularly important for treatment and the like, it is necessary to read internal tissue information near the cross cursor from the entire displayed tomographic image. is there.

Normally, this important area is not large compared to the entire tomographic image, so the display area on the monitor is small.

Obtaining necessary internal tissue information from such a small display area is a great burden for the operator.

Further, the above-mentioned document "3D image display device"
In this example, the position of the viewpoint in the observation target is displayed on the tomographic image as a guide screen, but the information near the viewpoint is an endoscopic image.

[0015] For this reason, in an endoscopic image which is important in an actual operation, information on a part which is obstructed and cannot be observed cannot be obtained.

To obtain this information, it is necessary to pay attention to the vicinity of the viewpoint of the tomographic image displayed on the guide screen.

Normally, the area required to obtain the information near the viewpoint is not large compared to the entire tomographic image, and the display area on the monitor is small.

Obtaining necessary internal tissue information from such a small display area places a great burden on the operator.

That is, if the entire tomographic image necessary for navigating the surgical tool to the affected part is displayed, it is difficult to obtain detailed internal tissue information near the surgical tool from the tomographic image. .

The present invention has been made in view of the above circumstances, and even if the entire tomographic image necessary for navigating the surgical tool to the affected part is displayed, it is possible to display details of the vicinity of the surgical tool from the tomographic image. It is an object of the present invention to provide a surgical navigation device capable of easily obtaining various internal tissue information.

[0021]

According to the present invention, there is provided, in order to solve the above-mentioned problems, (1) a surgical navigation apparatus for displaying the position and orientation of a surgical tool with respect to a subject, and for observing or treating the subject. A surgical instrument to be performed, three-dimensional position and orientation measuring means for measuring the three-dimensional position and orientation of the subject and the surgical tool, storage means for storing tomographic image information of the subject, Tomographic image extracting means for extracting a tomographic image from the tomographic image information stored in the storage means based on three-dimensional position and orientation information, and a projection image for adding position and orientation information of the surgical tool to the extracted tomographic image Adding means, a distance measuring means for measuring a distance between the subject and the surgical instrument, and a projection image for changing a size of an image output from the projection image adding means based on distance information from the distance measuring means. System There is provided a surgical navigation apparatus having control means and image display means for displaying an image output from the projection image control means.

According to the present invention, in order to solve the above-mentioned problems, (2) when the distance between the subject and the surgical instrument is less than a preset distance, the projection image control means sets the tomographic image The surgical navigation device according to (1) is provided, wherein the surgical navigation device is output by enlarging.

According to the present invention, in order to solve the above-mentioned problems, there is provided (3) a surgical navigation device for displaying the position and orientation of a surgical tool with respect to a subject, wherein the surgical tool has a function of observing or treating the subject. Three-dimensional position and orientation measurement means for measuring the three-dimensional position and orientation of the subject and the surgical tool; storage means for storing tomographic image information of the subject; three-dimensional positions of the subject and the surgical tool Based on the posture information, from the tomographic image information stored in the storage means, a tomographic image extracting means for extracting a tomographic image, a distance measuring means for measuring a distance between the subject and the surgical instrument, and Tomographic image control means for changing the size of the tomographic image output from the tomographic image extracting means based on the distance information, and a projection image addition for adding the position and orientation information of the surgical tool to the extracted tomographic image There is provided a surgical navigation apparatus having means and image display means for displaying an image output from the projection image control means.

[0024]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram showing a configuration of a surgical navigation device according to a first embodiment of the present invention.

The surgical navigation device according to the first embodiment of the present invention is configured as follows.

First, as shown in FIG. 1, it is assumed that the subject 1 is lying on his / her back on the operating table.

A sensing plate 2 having infrared LEDs arranged in a triangular shape is fixed to the head of the subject 1 with a tape.

A sensing plate 4 having infrared LEDs arranged in a triangular shape is fixed to the rigid endoscope 3.

In this case, on the sensing plate 2 and the sensing plate 4, the positional relationship between the infrared LEDs arranged in the same area is not changed.

The positions where the infrared LEDs are arranged are measured in advance with respect to the coordinate system p defined on the sensing plate 2 and the coordinate system e defined on the sensing plate 4. It is stored in the sensor information storage unit 5 as definition data.

The sensor information storage unit 5 is connected to the sensor control unit 6.

Further, an image sensing type sensor assembly 7 is arranged so that the sensing plate 2 and the sensing plate 4 are located within the measurement range.

The sensing plate 2 and the sensing plate 4 and the sensor assembly 7 are connected to the sensor control section 6 to constitute a three-dimensional position and orientation measuring means.

The three-dimensional position and orientation information obtained by such three-dimensional position and orientation measuring means is passed from the sensor control unit 6 to the navigation control unit 8.

Here, the tomographic image information of the subject 1 is, for example, 256 (pixels) × 256 in the axial direction.
By stacking the (pixel) tomographic images in the vertical direction, 256 (pixel) × 256 (pixel) × 2
56 (pixel) 3D volume data
The dimension information is reconstructed and stored in the navigation information storage unit 9 as tomographic image data 10.

In this embodiment, 1 (pixel)
The size of l) is 1 (mm).

Characteristic points on the body of the subject 1, such as the ears and eyes,
The coordinate values of markers and the like attached to the subject 1 are stored in the navigation information storage unit 9 as model data 11 on the object coordinate system m defined for the tomographic image data 10.

FIG. 2 shows the data of the subject 1 and the model data 11 on the object coordinate system m such as characteristic points on the body such as the ears and eyes and markers attached to the subject 1. And the coordinate values of the corresponding feature points on the coordinate system p defined by the sensing plate 2 are measured, and the coordinate conversion matrix mHp14 is calculated, indicating that the correlation is established.

The coordinate transformation matrix mHp14 is stored in the navigation information storage unit 9.

Here, the coordinate transformation matrix is, as shown in FIG. 3, a three-row, three-column component R representing a rotation operation in a three-dimensional space.
And a 3-row, 1-column component T representing a translation operation in a three-dimensional space
And a 4-by-4 matrix composed of constant components.

As shown in FIG. 4, a coordinate conversion matrix eHc15 from a coordinate system c used in a camera model representing the optical system of the rigid endoscope 3 to a coordinate system e defined by the sensing plate 4 is obtained. ing.

The origin of the coordinate system c used in the camera model representing the optical system of the rigid endoscope 3 coincides with the tip 21 of the rigid endoscope 3, and the optical axis of the rigid endoscope 3 is set in the camera coordinate system c.
Z axis.

The coordinate transformation matrix eHc15 is stored in the navigation information storage unit 9.

A part to be observed, treated, and cautioned during the operation is set as a target part of the subject 1.

In this embodiment, the affected area of the subject 1, the carotid artery and the optic nerve are set as target sites.

The coordinate values on the object coordinate system m indicating the position of the target part are stored in the navigation information storage unit 9 as target part position information.

Note that the number of target sites is not limited to one, and a plurality of target sites, such as diseased parts, blood vessels, nerves, and normal tissues, may be set.

Next, the operation of the surgical navigation device according to the first embodiment of the present invention will be described.

During the operation of the present operation navigation apparatus, the sensor control unit 6 which is a component of the three-dimensional position and orientation measurement means is used.
Causes the infrared LEDs on the sensing plate 2 and the sensing plate 4 to sequentially emit light, and the sensor assembly 7 captures the state in which these infrared LEDs emit light as an image.

The sensor control unit 6 calculates the three-dimensional position of each infrared LED using the timing at which the infrared LED was made to emit light and the image obtained from the sensor assembly 7.

In the sensor control section 6, each infrared ray LE
D stored in the sensor information storage unit 5 and the three-dimensional position of D
The three-dimensional position and orientation of the sensing plate 2 and the sensing plate 4 are calculated using the D definition data.

Further, the sensor control unit 6 converts the three-dimensional position and orientation of the sensing plate 2 and the sensing plate 4 into a relative position and orientation of the sensing plate 2 with respect to the sensing plate 4, and obtains a coordinate conversion matrix pH.
e17 is obtained.

FIG. 5 is a flowchart showing a processing procedure inside the navigation control unit 8.

That is, inside the navigation control unit 8, the processing is executed in steps as shown in FIG.

First, in step S101, a coordinate conversion matrix eHc15 and a coordinate conversion matrix pHe1 as shown in FIG.
7. Camera coordinate system c using coordinate transformation matrix mHp14
Is converted into a coordinate value on the object coordinate system m.

In step S102, the tip 21 of the rigid endoscope 3 converted into the coordinate values on the object coordinate system m
Then, tomographic images in three directions to be displayed on the monitor 13 are extracted by using the positions of.

The procedure will be described with reference to FIG.

The navigation information storage unit 9 stores three-dimensionally reconstructed tomographic image data 10 of the subject 1.

An object coordinate system m is defined in the tomographic image data 10, and the navigation control unit 8 controls the object coordinate system including the position of the tip end 21 of the rigid endoscope 3 converted into coordinate values on the object coordinate system m. m, the XY plane is an axial section 30, the YZ plane is a sagittal section 31,
The ZX plane is extracted as the coronal section 32.

For example, the position of the tip 21 of the rigid endoscope 3 converted into the coordinate value on the object coordinate system m is (124, 2).
In the case of (11,88), the XY plane of Z = 88 is the axial section 30, the YZ plane of X = 124 is the sagittal section 31, and the ZX plane of Y = 211 is the coronal section 3.
2.

In step S103, the tip 2 of the rigid endoscope 3
A straight line representing 1 is added to the three-dimensional tomographic image axial section 30, sagittal section 31, and coronal section 32 generated in step S102.

This procedure will be described with reference to FIGS. 7 and 8.

The position of the distal end 21 of the rigid endoscope 3 is determined in step S
Three-dimensional tomographic image axial section 30, generated in 102,
It is expressed by adding two straight lines parallel to the axis of the object coordinate system m having the position of the tip 21 of the rigid endoscope 3 as an intersection on the sagittal section 31 and the coronal section 32.

The two straight lines parallel to the axis of the object coordinate system m at the intersection of the position of the tip 21 of the rigid endoscope 3 are the three-dimensional tomographic image axial section 30, sagittal section 31, coronal section generated in step S102. The intersection of the cross section 32 is formed.

For example, the position of the distal end 21 of the rigid endoscope 3 converted into the coordinate value on the object coordinate system m is (124, 2).
11, 88), the XY section on the axial section 30
The position where the tip 21 of the rigid endoscope 3 is projected to the coordinates (124,
211) the X axis of the object coordinate system m at the intersection, Y
Two straight lines 33 and 34 parallel to the axis are added, and on the sagittal section 31, the object coordinate system m having the intersection (211, 88) where the tip 21 of the rigid endoscope 3 is projected on the YZ coordinate system. Two straight lines 34 and 35 parallel to the Y-axis and the Z-axis are added, and on the coronal cross-section 32, an object having the intersection (88, 124) at the position where the tip 21 of the rigid endoscope 3 is projected on the ZX coordinate. Two straight lines 35 and 33 parallel to the Z axis and the x axis of the coordinate system m are added.

In step S104, the tip 2 of the rigid endoscope 3
The distance between the object 1 and the target site of the subject 1 is calculated.

The target part position information indicating the position of the target part of the subject 1 stored in the navigation information storage unit 9 is:
Since the coordinate value is a three-dimensional position on the object coordinate system m expressed as a point, the distance between the tip 21 of the rigid endoscope 3 and the target part of the subject 1 is calculated by the following elimination formula. Is calculated as

D = sqrt ((xo−xc) ² + (yo
−yc) ² + (zo−zc) ² ) where, d: distance between each target site and the tip 21 of the rigid endoscope 3, xo, yo, zo: position coordinate values of each target site, xc, yc, zc : Coordinate value of the tip 21 of the rigid endoscope 3 on the object coordinate system m.

Next, in step S105, step S105
It is determined whether to enlarge the tomographic image according to the distance between the distal end 21 of the rigid endoscope 3 calculated in 104 and each target site of the subject 1.

In the present embodiment, when the distance between the distal end 21 of the rigid endoscope 3 and each target site of the subject 1 becomes less than 15 (mm), the tomographic image is enlarged twice in length and width. (Step S106).

When the distance between the distal end 21 of the rigid endoscope 3 and each target site of the subject 1 is all 15 (mm) or more, the tomographic image is not enlarged.

In step S107, the monitor 13 is actually
The tomographic image is cut out according to the size displayed by.

In the present embodiment, the size of the display area on the monitor 13 set to one tomographic image is 256 (pi)
xel) × 256 (pixel).

If the tomographic image is not enlarged, the size of the tomographic image is equal to the size of the display area on the monitor 13, so that the clipping process is not executed and the navigation control unit 8
Are straight lines 33, 34, 3 representing the position of the tip 21 of the rigid endoscope 3.
The tomographic image to which 5 is added is output to the monitor 13 as it is.

On the other hand, when the tomographic image is enlarged in step S106, the size of the tomographic image is 512 (pixel) ×
512 (pixel), which exceeds the display area on the monitor 13.

In step S107, as shown in FIG.
The display range is 25 around the position of the distal end 21 of the rigid endoscope 3.
An area of 6 (pixel) × 256 (pixel) is cut out, and the cut out area is output to the monitor 13.

Although FIG. 9 shows the processing on the axial section 30, the same processing is performed on the sagittal section 31 and the coronal section 32.

The monitor 13 displays information output from the navigation control unit 8 as shown in FIGS.

FIG. 10 shows the distal end 21 of the rigid endoscope 3 and the subject 1
Shows the case where the distance to each target part is equal to or longer than the set distance, and the distal end 21 of the rigid endoscope 3 is set so that the entire tomographic image can be accommodated.
The three-dimensional tomographic image axial section 30, sagittal section 31, and coronal section 32 to which the straight lines 33, 34, and 35 representing the position are added are displayed.

FIG. 11 shows a case where the distance between the distal end 21 of the rigid endoscope 3 and each target site of the subject 1 is smaller than the set distance. Tomographic image axial section 30, sagittal section 3
1. A part of the coronal section 32 is displayed.

The surgeon makes a three-dimensional tomographic axial section 3 including the tip 21 of the rigid endoscope 3 displayed on the monitor 13.
0, straight lines 33, 34, 3 representing the position of the tip 21 of the rigid endoscope 3 added on the sagittal section 31, coronal section 32
By manipulating the rigid endoscope 3 while viewing the view 5, when the distal end 21 of the endoscope 3 approaches the target site, the three-dimensional tomographic image axial section 30 and the sagittal section 3 are enlarged.
1. Surgery is performed while viewing the coronal section 32.

Next, the effect of the surgical navigation device according to the first embodiment of the present invention will be described.

When the distal end 21 of the rigid endoscope 3 and the target site set on the subject 1 are separated from each other by a distance equal to or greater than the set distance, the present surgical navigation apparatus uses a straight line 33 representing the position of the distal end 21 of the rigid endoscope 3. Since the entire tomographic image to which 34 and 35 are added is displayed on the monitor 13, the surgeon can easily grasp where the rigid endoscope 3 is located inside the subject, and move the rigid endoscope 3 near the affected part. Can be moved up.

On the other hand, when the distal end 21 of the rigid endoscope 3 and the target portion set in the subject 1 are closer than the set distance, a tomographic image magnified twice in length and width around the distal end 21 of the rigid endoscope 3. Is displayed on the monitor 13, so that the operator can easily grasp the internal tissue information of the subject 1 near the distal end 21 of the rigid endoscope 3.

As a result, the surgeon can obtain navigation information according to the situation, and can safely and reliably perform an operation.

Incidentally, each configuration of the first embodiment of the present invention can of course be variously modified and changed.

For example, the three-dimensional position measuring means is not limited to the optical type of this embodiment, but may be of any type such as a magnetic type or a mechanical type.

Further, it is also possible to cope with a case where the rigid endoscope 3 is replaced with a treatment tool such as a flexible endoscope, an arthroscopic scope, an operating microscope, or a suction tube to perform an operation.

Further, there may be a plurality of surgical tools to be navigated, that is, a case where a plurality of treatment tools are used simultaneously with the rigid endoscope 3.

The monitor 13 can be replaced with a video presentation device such as an HMD.

Further, instead of the tomographic image extracted so as to include the tip of the rigid endoscope 3, a tomographic image extracted so as to include the focal point of the rigid endoscope 3 or a distance set in advance from the focal point or the end of the rigid endoscope 3. Lines 33, 34, and 35 representing the position of the tip 21 of the rigid endoscope 3 with respect to the tomographic image extracted to include the point just ahead and the tomographic image extracted at a position fixedly set in advance with respect to the tomographic image information. May be added.

The tomographic image is not limited to three directions, but may be any direction as long as it is one direction or more.

The direction in which the tomographic image is extracted is not limited to a plane perpendicular to the axis set in the tomographic image information as in the present embodiment, but can be set freely.

For example, two tomographic images including the optical axis and a tomographic image orthogonal to these and passing through a focal point may be extracted. The extracted tomographic images need not be orthogonal.

Further, the user can arbitrarily set a set value serving as a criterion for determining whether or not to perform the enlargement process in step S105.

It is also possible to set different values for each target part of the subject 1.

When a plurality of surgical tools are used, different values can be set for the combination of each surgical tool and the target region of the subject 1.

The magnification of the tomographic image may be changed stepwise or continuously in accordance with the distance between the distal end 21 of the rigid endoscope 3 and the target portion of the subject 1.

For example, the relationship between the distance and the enlargement ratio is set to 10 times for 0 to 2 (mm), 5 times for 2 to 5 (mm),
10 (mm) may be doubled.

The distance between the arbitrary portion of the rigid endoscope 3 and the target portion of the subject 1 may be used instead of the distance between the distal end 21 of the rigid endoscope 3 and the target portion of the subject 1.

In this case, even if the distance between the distal end 21 of the rigid endoscope 3 and the target part is equal to or greater than the set value, for example, the distance between a certain part on the insertion axis and the target part of the subject 1 is set. If the value is smaller than the set value, the tomographic image is enlarged.

The calculation of the distance between the distal end 21 of the rigid endoscope 3 and the target part of the subject 1 is not limited to the distance between two points. For example, the distance from the surface of the target part to the three-dimensional distance is calculated. Information storage unit 9 as a simple distance reference table
And may be obtained by referring to this.

(Second Embodiment) The second embodiment is different from the first embodiment in that the rigid endoscope 3 is replaced with a surgical microscope 50 and a microscope control unit is added. Since the configuration is the same as that of the first embodiment except for, the illustration is omitted.

Therefore, only portions different from the first embodiment will be described below.

The sensing plate 4 in which the infrared LEDs are arranged in a triangular shape is fixed to a surgical microscope 50.

As shown in FIG. 12, a coordinate transformation matrix eHc15 from the coordinate system c used in the camera model representing the optical system of the operating microscope 50 to the coordinate system e defined by the sensing plate 4 is obtained. I have.

The origin of the coordinate system c used in the camera model representing the optical system of the operating microscope 50 is above the optical axis 51 of the operating microscope 50, for example, from the lower end 53 of the lens barrel 52 to 1200. (Mm) It is located above, and the optical axis 51 of the surgical microscope 50 is the Z axis of the camera coordinate system c.

The coordinate transformation matrix eHc15 is stored in the navigation information storage unit 9.

A microscope control unit (not shown) for transmitting the distance and the angle of view from the origin of the camera coordinate system c of the surgical microscope 50 to the focal point 55 to the navigation control unit 8 is provided between the surgical microscope 50 and the navigation control unit 8. Are connected to each other.

The tomographic image data 10 of the subject 1 is stored in the navigation information storage section 9 as 512 (pixel) × 512.
It is stored as (pixel) × 512 (pixel) three-dimensional volume data.

In the present embodiment, 1 (pixel)
Is 0.5 (mm).

Next, the operation of the surgical navigation apparatus according to the second embodiment of the present invention will be described.

In the surgical navigation apparatus, a coordinate conversion matrix pHe17 representing the relative position and orientation of the sensing plate 2 with respect to the sensing plate 4 is obtained from the three-dimensional position and orientation measuring means in the same procedure as in the first embodiment.

The microscope control unit transmits the distance from the origin of the camera coordinate system c of the surgical microscope 50 to the focal point 55 to the navigation control unit 8.

The distance from the origin of the camera coordinate system c to the focal point 55 is expressed as a coordinate value of a point on the Z axis of the camera coordinate system c.

FIG. 13 is a flowchart showing a processing procedure inside the navigation control unit 8 in the surgical navigation apparatus according to the second embodiment of the present invention.

The navigation control unit 8 uses the coordinate conversion matrix eHc15, the coordinate conversion matrix pHe17, and the coordinate conversion matrix mHp14 in step S201 shown in FIG.
The position of the origin of the camera coordinate system c and the position of the focal point 55 in the camera coordinate system c transmitted from the microscope control unit are converted into coordinate values on the object coordinate system m.

Further, the navigation control unit 8 uses the position of the focal point 55 of the operation microscope 50 converted into the coordinate value on the object coordinate system m in step S202 shown in FIG. An axial section 30, a sagittal section 31, and a coronal section 32 are extracted.

This procedure is a procedure for extracting a three-dimensional tomographic axial section 30, a sagittal section 31, and a coronal section 32 including the tip 21 of the rigid endoscope 3 in the first embodiment. 21 is replaced with the focal point 55 of the operating microscope 50.

In step S203 shown in FIG. 13, the navigation control unit 8 calculates the distance between the focal point 55 of the surgical microscope 50 and the target site of the subject 1.

In this procedure, the coordinate value of the distal end 21 of the rigid endoscope 3 on the object coordinate system m in step S104 in the first embodiment is converted into the coordinate of the focal point 55 of the surgical microscope 50 on the object coordinate system m. It is replaced with a value.

In step S204 shown in FIG. 13, the navigation control unit 8 extracts in step S202 according to the distance between the focal point 55 of the surgical microscope 50 calculated in step S203 and each target site of the subject 1. It is determined whether or not the reduced tomographic image is reduced.

In this embodiment, the distance between the focal point 55 of the operating microscope 50 and each target site of the subject 1 is 10
(Mm) or more, the tomographic image is reduced to half (step S205).

Further, at least one of the distances between the focal point 55 of the operating microscope 50 and each target site of the subject 1 is 1
If it is less than 0 (mm), the tomographic image is not reduced.

Next, in step S206 shown in FIG.
1 is added to the unreduced tomographic image.

The optical axis 51 of the operating microscope 50 is represented by straight lines 60, 61, and 62 connecting the focal point 55 of the operating microscope 50 and the origin of the camera coordinate system c.

For example, as shown in FIG. 14, the position of the focal point 55 of the surgical microscope 50 converted into the coordinate value on the object coordinate system m is (248, 422, 176), and the origin of the camera coordinate system c is Is (488, 30, 722)
At the time, on the axial section 30, the position (248, 4) where the focal point 55 of the surgical microscope 50 is projected on the XY coordinates
22) and a thickness (pixel) connecting the position (488, 30) where the origin of the camera coordinate system c of the rigid endoscope 3 is projected.
Is added to the projection image on the sagittal section 31, the position (422, 176) where the focal point 55 of the surgical microscope 50 is projected on the YZ coordinates and the origin of the camera coordinate system c of the rigid endoscope 3 are A straight line 61 having a thickness of 1 (pixel) connecting the projected position (30, 722) is added, and the focal point 55 of the surgical microscope 50 is projected on the ZX coordinate in the projected image on the coronal section 32. The position (176,248) and the position (72) where the origin of the camera coordinate system c of the rigid endoscope 3 is projected
2,488) and a straight line 6 of thickness 1 (pixel)
2 is added.

Next, the navigation controller 8 adds the projection image of the optical axis 51 of the operating microscope 50 to the tomographic image reduced in step S205 in step S207.

This procedure is the same as that in step S206 except that the axial section 30, the sagittal section 31 and the coronal section 32 are reduced to half in step S205, so that the focal point 55 of the operating microscope 50 is projected. The coordinate value of the position and the coordinate value of the position where the origin of the camera coordinate system c is projected are also halved.

Next, the navigation control unit 8 executes the processing shown in FIG.
In step S208, a tomographic image is cut out according to the size actually displayed on the monitor 13.

In the present embodiment, the size of the display area on the monitor 13 set to one tomographic image is 256 (pi)
xel) × 256 (pixel).

When the tomographic image is reduced to one half in step S205, the size of the tomographic image is equal to the size of the display area on the monitor 13, so that the cut-out processing is not executed. The tomographic image to which the straight lines 60, 61 and 62 representing the optical axis 51 are added is output to the monitor 13 as it is.

On the other hand, if the tomographic image was not reduced,
The size of the tomographic image is 512 (pixel) × 512 (pi
xel), it exceeds the display area on the monitor 13.

In step S208, as shown in FIG. 15, the position of the focal point 55 of the operating microscope 50 is set as the center,
The display range is 256 (pixel) x 256 (pixel)
An area as indicated by 1) is cut out, and this cut out area is output to the monitor 13.

Although FIG. 15 shows the processing on the axial section 30, the same processing is performed on the sagittal section 31 and the coronal section 32.

The monitor 13 displays information output from the navigation control unit 8 as shown in FIGS.

FIG. 16 shows a focal point 55 of the operating microscope 50.
FIG. 3 shows a case where the distance between the object and each target site of the subject 1 is greater than or equal to a set distance, and straight lines 60, 61, 6 representing the optical axis 51 of the surgical microscope 50 so that the entire tomographic image can be accommodated.
An axial section 30, a sagittal section 31, and a coronal section 32 are displayed as tomographic images in three directions to which 2 is added.

On the other hand, FIG. 17 shows a case where the distance between the focal point 55 of the operating microscope 50 and each target site of the subject 1 is less than the set distance.
Section 3 as a tomographic image in three directions centered on
0, a part of the sagittal section 31, and a part of the coronal section 32 are displayed.

The surgeon can view the tomographic image in three directions including the position of the focal point 55 of the operating microscope 50 displayed on the monitor 13, the axial section 30, the sagittal section 31, and the coronal section 3.
2 and the operating microscope 50 while observing the straight lines 60, 61 and 62 representing the optical axis 51 of the operating microscope 50 added thereto.
Is operated, the focal point 55 of the operating microscope 50 is
When the user approaches the target site of the subject 1, the surgeon performs an operation while viewing the three-dimensional tomographic image axial section 30, sagittal section 31, and coronal section 32 in the vicinity of the focal point 55 of the surgical microscope 50, which is largely displayed.

Next, the effect of the second embodiment of the present invention will be described.

When the distance between the focal point 55 of the surgical microscope 50 and each target site of the subject 1 is larger than the set distance, the present surgical navigation apparatus uses the surgical microscope 5
Since the entire tomographic image to which the straight lines 60, 61, and 62 representing the optical axis 51 of 0 are added is displayed on the monitor 13, the operator can easily determine which position inside the subject 1 the current observation position is. And the focal point 5 of the surgical microscope 50
5 can be moved to the vicinity of the affected part.

On the other hand, when the distance between the focal point 55 of the surgical microscope 50 and each target site of the subject 1 is less than the set distance, a part of the tomographic image centered on the focal point 55 of the surgical microscope 50 Is displayed on the monitor 13 so that the operator can easily grasp the internal tissue information of the subject 1 near the observation position.

As a result, the surgeon can obtain navigation information according to the situation, and can safely and reliably perform an operation.

It is to be noted that each configuration of the second embodiment of the present invention can of course be variously modified and changed.

For example, the three-dimensional position measuring means is not limited to the optical type of this embodiment, but may be of any type such as a magnetic type or a mechanical type.

It is also possible to cope with performing surgery by replacing the operating microscope 50 with a rigid endoscope, a flexible endoscope, or an articulated endoscope.

The surgical microscope 50 may be replaced with a treatment tool having no observation function.

Further, a plurality of surgical instruments to be navigated may be provided, that is, a case where an endoscope and a plurality of treatment tools are used together with the surgical microscope 50 is also included.

The monitor 13 can be replaced with an image presentation device such as an HMD.

In addition, instead of a tomographic image extracted so as to include the focal point 55 of the operating microscope 50, a point ahead of the focal point or the lower end 53 of the lens barrel 52 of the operating microscope 50 by a predetermined distance is used. The optical axis 51 of the operating microscope 50 may be added to the tomographic image extracted to include the tomographic image or tomographic image extracted at a position fixedly set in advance with respect to the tomographic image information.

In this case, the projection image information to be added to the tomographic image is not limited to the optical axis 51 of the surgical microscope 50. For example, a plurality of lens barrels, angles of view, etc. of the surgical microscope 50 may be added.

The tomographic image is not limited to three directions, but may be any direction as long as it is one direction or more.

Further, the direction of extracting the tomographic image is not limited to a plane perpendicular to the axis set in the tomographic image information as in the present embodiment, but can be set freely.

For example, two tomographic images including the optical axis and a tomographic image orthogonal to these and passing through a focal point may be extracted, and the extracted tomographic images need not be orthogonal.

In step S204, the user can arbitrarily set a set value serving as a criterion for determining whether to perform the reduction process.

It is also possible to set different values for each target site of the subject 1.

When a plurality of surgical tools are used, different values can be set for the combination of each surgical tool and the target region of the subject 1.

Further, the enlargement / reduction ratio of the tomographic image may be changed stepwise or continuously according to the distance between the focal point 55 of the operating microscope 50 and the target site of the subject 1.

The distance between the focal point 55 of the operating microscope 50 and the target site of the subject 1 is not the distance between the operating microscope 50 and the subject.
The enlargement / reduction ratio of the tomographic image may be changed stepwise or continuously according to the distance between the arbitrary part of the subject 1 and the target part of the subject 1.

In this case, even if the distance between the focal point 55 of the operating microscope 50 and the target site is equal to or greater than the set value,
For example, if the distance between a portion of the optical microscope 51 of the operating microscope 50 and the target site is larger than a set value, the tomographic image is reduced.

The calculation of the distance between the focal point 55 of the operating microscope 50 and the target part of the subject 1 is not limited to the distance between two points. The information may be stored in the navigation information storage unit 9 as a dimensional distance reference table, and may be obtained by referring to the table.

In the present specification described in the above embodiments, in addition to claims 1 to 3 described in the claims, the inventions described as appendices 1 to 5 below will be described. include.

(Supplementary Note 1) When the distance between the subject and the surgical tool is equal to or longer than a predetermined distance, the projection image control means outputs the tomographic image without changing the size. The surgical navigation device according to claim 1, which performs the operation.

(Supplementary note 2) The tomographic image control means reduces the tomographic image and outputs the reduced tomographic image when the distance between the subject and the surgical tool is equal to or longer than a preset distance. A surgical navigation device according to claim 1.

(Supplementary Note 3) If the distance between the subject and the surgical tool is less than a preset distance, the tomographic image control means outputs the tomographic image without changing the size of the tomographic image. The surgical navigation device according to claim 1, which performs the operation.

(Supplementary Note 4) A surgical instrument for performing a treatment including photographing of an image of a subject, and three items of the subject and the surgical tool
Three-dimensional position and orientation measurement means for measuring three-dimensional position and orientation, storage means for storing tomographic image information of the subject, and three-dimensional positions of the subject and the surgical instrument measured by the three-dimensional position and orientation measurement means A tomographic image extracting means for extracting a tomographic image based on the posture information and the tomographic image information of the subject stored in the storage means, and a projection image information of the surgical tool extracted by the tomographic image extracting means. In a surgical navigation apparatus having a projection image adding means for adding on an image and an image display means for displaying an image output from the projection image adding means, the object measured by the three-dimensional position and orientation measuring means and A distance calculating unit that calculates a distance between the subject and the surgical tool based on the three-dimensional position and orientation information of the surgical tool; and a projection image based on the calculation result of the distance calculating unit. Surgical navigation apparatus; and a tomographic image control means for expanding and / or reducing the image output from the unit.

(Corresponding Embodiment of the Invention) The embodiment relating to the present invention corresponds to the first embodiment.

In the first embodiment, the surgical tool in the above components corresponds to the rigid endoscope 3, but the same effect can be obtained with a flexible endoscope, a surgical microscope, a treatment tool, a suction tube, and the like.

In the first embodiment, the three-dimensional position / orientation measuring means in the above-mentioned components comprises a sensing plate 2, a sensing plate 4, a sensor information storage 5,
The sensor control unit 6 includes the sensor assembly 7 of an image capturing method, but includes not only an optical type and a magnetic type but also a mechanical type and other three-dimensional position and orientation measurement methods.

The storage means in the above components corresponds to the navigation information storage section 9 in the first embodiment.

In the first embodiment, the navigation control unit 8 corresponds to the tomographic image extracting means in the above components.

In the first embodiment, the navigation control section 8 corresponds to the projection image adding means in the above components.

The image display means in the above components corresponds to the monitor 13 in the first embodiment.

In the first embodiment, the component distance calculating means in the above components is the navigation control unit 8.
Is applicable.

In the first embodiment, the tomographic image control means in the above components corresponds to the navigation control unit 8.

(Operation) The three-dimensional position / posture measuring means measures the three-dimensional position / posture of the surgical tool and the subject, and transmits the result to the tomographic image extracting means.

[0178] The tomographic image extracting means determines the desired image based on the tomographic image information of the subject stored in the storage section and the three-dimensional position and orientation information of the surgical tool and the subject transmitted from the three-dimensional position and orientation measuring means. Is extracted and transmitted to the projection image adding means.

The projection image adding means adds projection image information of the surgical instrument to the tomographic image transmitted from the tomographic image extracting means based on the three-dimensional position and orientation information.

The distance calculating means calculates the distance between the surgical instrument and the target part of the subject based on the surgical tool and the three-dimensional position and orientation information of the subject transmitted from the three-dimensional position and orientation measuring means.

The tomographic image control means enlarges and / or reduces the image output by the projection image adding means on the basis of the distance between the surgical tool calculated by the distance calculating means and the target part of the subject, and displays the image. Is transmitted to the image display means.

The image display means displays the tomographic image to which the projection image of the surgical instrument transmitted from the tomographic image control means is added and whose display size is controlled.

(Effect) The present surgical navigation device displays on the image display means a tomographic image of the subject to which the projection image information of the surgical tool is added. Can be easily grasped.

In addition, the present surgical navigation apparatus appropriately controls the size of the tomographic image in accordance with the distance between the surgical instrument calculated by the distance calculating means and the target part of the subject, and displays the size on the image display means. The operator can easily grasp the internal tissue information necessary for making an accurate situation determination from the tomographic image.

(Supplementary Note 5) A surgical tool for performing a procedure including photographing of an image of a subject, and three of the subject and the surgical tool
Three-dimensional position and orientation measurement means for measuring three-dimensional position and orientation, storage means for storing tomographic image information of the subject, and three-dimensional positions of the subject and the surgical instrument measured by the three-dimensional position and orientation measurement means A tomographic image extracting means for extracting a tomographic image based on the posture information and the tomographic image information of the subject stored in the storage means, and a projection image information of the surgical tool extracted by the tomographic image extracting means. In a surgical navigation apparatus having a projection image adding means for adding on an image and an image display means for displaying an image output from the projection image adding means, the object measured by the three-dimensional position and orientation measuring means and Distance calculating means for calculating a distance between the subject and the surgical tool based on the three-dimensional position and orientation information of the surgical tool; and extracting the tomographic image based on a calculation result of the distance calculating means. Surgical navigation apparatus; and a tomographic image control means for expanding and / or reducing the tomographic image extracted from the stage.

(Corresponding Embodiment of the Invention) The embodiment relating to the present invention corresponds to the second embodiment.

In the second embodiment, the surgical tool in the above-described components corresponds to the surgical microscope 50. However, the same effect can be obtained by using a rigid scope, a flexible scope, a treatment tool, a suction tube, and the like.

In the second embodiment, the three-dimensional position / posture measuring means in the above-mentioned components comprises a sensing plate 2, a sensing plate 4, a sensor information storage unit 5, a sensor control unit 6, a sensor assembly of an image taking system. 7, but is not limited to the optical type and the magnetic type, but also includes other three-dimensional position and orientation measurement methods such as a mechanical type.

In the second embodiment, the storage means in the above components corresponds to the navigation information storage unit 9.

In the second embodiment, the navigation control unit 8 corresponds to the tomographic image extracting means in the above components.

In the second embodiment, the navigation control section 8 corresponds to the projection image adding means in the above components.

In the second embodiment, the monitor 13 corresponds to the image display means in the above components.

In the second embodiment, the navigation control unit 8 corresponds to the distance calculation means in the above components.

In the second embodiment, the tomographic image control means in the above components corresponds to the navigation control unit 8.

(Operation) The three-dimensional position / posture measuring means measures the three-dimensional position / posture of the surgical tool and the subject, and transmits the result to the tomographic image extracting means.

[0196] The tomographic image extracting means is configured to obtain a desired tomographic image based on the tomographic image information of the subject stored in the storage means and the three-dimensional position and orientation information of the surgical tool and the subject transmitted from the three-dimensional position and orientation measuring means. Extract the image.

The distance calculating means calculates the distance between the surgical instrument and the target part of the subject based on the surgical tool and the three-dimensional position and orientation information of the subject transmitted from the three-dimensional position and orientation measuring means.

The tomographic image control means displays the tomographic image output by the tomographic image extracting means by enlarging and / or reducing it based on the distance between the surgical tool calculated by the distance calculating means and the target part of the subject. The size is controlled and transmitted to the projection image adding means.

The projection image adding means adds projection image information of the surgical instrument to the tomographic image transmitted from the tomographic image control means based on the three-dimensional position and orientation information.

The image display means adds the projection image of the surgical tool transmitted from the projection image addition means and displays a tomographic image whose size to be displayed is controlled.

(Effect) Since the present surgical navigation apparatus displays on the image display means a tomographic image of the subject to which the projection image information of the surgical tool is added, the operator can operate the surgical tool relative to the subject. Can be easily grasped.

[0202] In addition, the present surgical navigation apparatus appropriately controls the size of the tomographic image in accordance with the distance between the surgical tool calculated by the distance calculating means and the target part of the subject, and displays the tomographic image on the image display means. The operator can easily grasp the internal tissue information necessary for making an accurate situation determination from the tomographic image.

[0203]

Therefore, as described above, according to the present invention, even if the entire tomographic image required for navigating the surgical tool to the affected part is displayed, the tomographic image near the surgical tool can be displayed from the tomographic image. A surgical navigation device capable of easily obtaining detailed internal tissue information can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a surgical navigation device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing data of the subject 1 and the subject 1 shown in FIG. 1 on an object coordinate system m such as feature points on the body such as ears and eyes and markers attached to the subject 1; The coordinate value model data 11 and the coordinate values of the corresponding feature points on the coordinate system p defined by the sensing plate 2 are measured,
It is a figure which shows that it is related by calculating a coordinate conversion matrix mHp14.

FIG. 3 is a diagram illustrating a three-row / three-column component R representing a rotation operation in a three-dimensional space, a three-row / one-column component T representing a translation operation in a three-dimensional space, and a constant component. FIG. 6 is a diagram showing that the matrix is a 4 × 4 matrix composed of.

FIG. 4 is a coordinate conversion matrix eH from a coordinate system c used in a camera model expressing the optical system of the rigid endoscope 3 in FIG. 1 to a coordinate system e defined by the sensing plate 4;
It is a figure which shows that c15 is calculated | required.

FIG. 5 is a navigation control unit 8 in FIG. 1;
It is a flowchart which shows an internal processing procedure.

FIG. 6 shows the position of the tip 21 of the rigid endoscope 3 which coincides with the origin of the camera coordinate system c using the coordinate transformation matrix eHc15, coordinate transformation matrix pHe17, and coordinate transformation matrix mHp14 in step S101 in FIG. FIG. 9 is a diagram showing that is converted into coordinate values on an object coordinate system m.

FIG. 7 shows a three-dimensional tomographic image displayed on the monitor 13 using the position of the tip end 21 of the rigid endoscope 3 converted into the coordinate value on the object coordinate system m in step S102 in FIG. It is a figure explaining the extraction procedure.

8 is a procedure of adding a straight line representing the distal end 21 of the rigid endoscope 3 to the three-dimensional tomographic axial section 30, sagittal section 31, and coronal section 32 generated in step S102 in step S103 in FIG. FIG.

FIG. 9 is a view showing an example in which the display range is 256 in step S107 in FIG.
It is a figure explaining the procedure which cuts out the area | region which becomes (pixel) * 256 (pixel).

FIG. 10 is a diagram showing a state in which information output from the navigation control unit 8 is displayed on the monitor 13 in FIG.
FIG. 4 is a diagram illustrating a case where a distance between a distal end 21 of the rigid endoscope 3 and each target site of the subject 1 is equal to or longer than a set distance.

FIG. 11 is a diagram showing a state in which information output from the navigation control unit 8 is displayed on the monitor 13 in FIG.
FIG. 3 is a diagram illustrating a case where a distance between a distal end 21 of a rigid endoscope 3 and each target site of a subject 1 is less than a set distance.

FIG. 12 is a diagram illustrating a surgical navigation device according to a second embodiment of the present invention, which is defined by a sensing plate 4 from a coordinate system c used in a camera model representing an optical system of a surgical microscope 50; It is a figure showing that coordinate transformation matrix eHc15 to coordinate system e is calculated.

FIG. 13 is a flowchart showing a processing procedure inside the navigation control unit 8 in the surgical navigation device according to the second embodiment of the present invention.

FIG. 14 is a flowchart showing step S206 in FIG. 13;
Then, when adding the projection image of the optical axis 51 of the operating microscope 50 to the unreduced tomographic image, the optical axis 5 of the operating microscope 50
1 is a focal point 55 of the operating microscope 50 and a camera coordinate system c.
FIG. 6 is a diagram illustrating an example of a form expressed by straight lines 60, 61, and 62 connecting the origins of FIG.

FIG. 15 is a flowchart showing step S208 in FIG. 13;
With the position of the focal point 55 of the operating microscope 50 as the center,
The display range is 256 (pixel) x 256 (pixel)
It is a figure explaining the procedure which cuts out the field used as l).

FIG. 16 is a view showing the operation of the surgical navigation apparatus according to the second embodiment of the present invention when displaying the information output from the navigation control unit 8 on the monitor 13; FIG. 3 is a diagram showing a case where a distance between a sample 1 and each target site is greater than or equal to a set distance.

FIG. 17 is a view showing a state in which the information output from the navigation control unit is displayed on the monitor in the surgical navigation apparatus according to the second embodiment of the present invention; FIG. 3 is a diagram illustrating a case where a distance between a sample 1 and each target site is less than a set distance.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Subject, 2 ... Sensing plate, 3 ... Rigid endoscope, 4 ... Sensing plate, 5 ... Sensor information storage part, 6 ... Sensor control part, 7 ... Sensor assembly, 8 ... Navigation control part, 9 ... Navigation information storage part Reference numeral 10: tomographic image data, 11: model data, 13: monitor, 14: coordinate conversion matrix mHp, 15: coordinate conversion matrix eHc, 17: coordinate conversion matrix pHe, 21: tip of the rigid mirror 3, 30: axial section, 31: sagittal section, 32: coronal section, 33, 34, 35: straight line representing the position of the tip 21 of the rigid endoscope 3, 50: surgical microscope, 51: optical axis of the surgical microscope 50, 52: lens barrel, 53: lower end, 55: focal point, 60, 61, 62: straight line representing the optical axis 51 of the surgical microscope 50.

Claims (3)

[Claims] 1. A surgical navigation device for displaying the position and orientation of a surgical tool with respect to a subject, wherein the surgical tool performs observation or treatment of the subject, and measures a three-dimensional position and orientation of the subject and the surgical tool. 3
Three-dimensional position and orientation measurement means; storage means for storing the tomographic image information of the subject; and tomographic information from the tomographic image information stored in the storage means based on the three-dimensional position and orientation information of the subject and the surgical tool. Tomographic image extracting means for extracting an image, projection image adding means for adding position and orientation information of the surgical tool to the extracted tomographic image, distance measuring means for measuring a distance between the subject and the surgical tool, Projection image control means for changing the size of the image output from the projection image addition means based on the distance information from the distance measurement means, and image display means for displaying the image output from the projection image control means Surgical navigation device having 2. The apparatus according to claim 1, wherein when the distance between the subject and the surgical tool is less than a predetermined distance, the projection image control means enlarges and outputs a tomographic image. Surgical navigation device. 3. A surgical navigation device for displaying the position and orientation of a surgical tool with respect to a subject, wherein the surgical tool performs observation or treatment of the subject and measures a three-dimensional position and orientation of the subject and the surgical tool. 3
Three-dimensional position and orientation measurement means; storage means for storing the tomographic image information of the subject; and tomographic information from the tomographic image information stored in the storage means based on the three-dimensional position and orientation information of the subject and the surgical tool. Tomographic image extracting means for extracting an image, distance measuring means for measuring a distance between the subject and the surgical instrument, and a tomographic image output from the tomographic image extracting means based on distance information from the distance measuring means. Tomographic image control means for changing the size, projection image adding means for adding position and orientation information of the surgical tool to the extracted tomographic image, and image display means for displaying an image output from the projected image control means And a surgical navigation device having: