JP2003339735A - Operation supporting apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an operation supporting apparatus which displays an operated site on an MRI image as the operation history along an operation time course. <P>SOLUTION: The operation supporting apparatus overlaps the position of an inserted operation tool in real time on the image of the site concerned of a previously collected specimen to display, and is equipped with a time storage memory 16, a time lag calculating means 17 and a pixel value converting section 12, wherein the time storage memory 16 stores the time at which the position information of the operation tool 2 is formed on the image space from an image spatial coordinate calculator 8; the time lag calculating means 17 calculates a time lag between the time stored by the time storage memory 16 and a current time; and the pixel value converting section 12 converts the position of the operation tool on the image to the corresponding pixel value along the operation time course. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surgical instrument or a surgical instrument such as an endoscope for inserting a surgical instrument into a subject for surgical operation or biological sample collection. The present invention relates to a surgical operation assistance device which is also called a surgical operation navigation device that assists a surgeon who performs a surgical operation by displaying the position inside the surgical operation device.

[0002]

2. Description of the Related Art In a medical institution such as a hospital, a tomographic image (for example, MRI image, X-ray CT image or transmission X-ray image) of an operation target region (region of interest) such as an organ or tumor in a subject is photographed.
2. Description of the Related Art During a surgical operation performed with reference to a line image or the like), a surgical navigation device is used that superimposes and displays a position of a surgical instrument operated by an operator on a reference tomographic image. This surgical navigation device detects the position of a surgical instrument such as a suction tube, a biopsy puncture needle or an incision scalpel while being in contact with a region of interest by magnetic, optical or multi-indirect position detection. In addition to the detection by the mechanism, the position of the surgical instrument is superimposed and displayed in real time in a state of being aligned with the tomographic image of the region of interest obtained by the MRI apparatus or the X-ray CT apparatus, so that the operator during surgery can display it on the tomographic image It is configured so that the position of the surgical instrument can be visually confirmed.

With this configuration, the operator can visually confirm and grasp the current position of the surgical instrument inserted in the subject by using the reference tomographic image and the image in which the surgical instrument is superimposed and displayed. Therefore, the operator can operate the surgical instrument accurately, and thus the operation can be performed quickly. Therefore, in the surgical navigation device, a device for accurately detecting the position of the surgical instrument and a device for displaying the detected surgical instrument position are devised so that the position of the surgical instrument can be displayed accurately and easily on the tomographic image. Power is being put into doing.

Therefore, in a surgical operation using a surgical navigation device, the position of the surgical instrument itself is displayed in real time on the image during the operation, so that, for example, the position of the tumor to be excised and the approach route to the excised site are objective. However, whether or not the surgery was actually performed from the start of surgery to a certain point in time, in other words, whether the tumor was removed or not, and the operation history such as the area of removal was determined by the operator's memory and feeling during the operation. Since the operation history is objectively grasped, it is necessary to judge it by a CT image or an MRI image obtained by reexamination by a post-operative X-ray CT apparatus or MRI apparatus.

If it is found in the post-examination reexamination that there is an unextracted portion, it may be necessary to perform reoperation in some cases, which imposes a great burden on the operator and the patient. As a method of obtaining an operation history during surgery, there is also a method of obtaining CT or ultrasound images during surgery by a method such as CT-guided stereotactic brain surgery or ultrasound-guided stereotactic brain surgery. They are,
It uses a modality that has a well-established interpretation method during surgery, and it is possible to leave an objective surgery history at each point during surgery, but on the other hand, the surgery operation is temporarily interrupted to capture images. Therefore, there is a drawback in that the operation time is significantly extended, and at the present time, it is not possible to use an MRI image that best depicts the lesion.

[0006] Therefore, the applicant of the present invention, "surgery history" up to a certain point during surgery, in real time, objectively
In addition, there is proposed a surgical operation support device as a surgical navigation device that is presented to the operator without hindering the surgical operation (JP-A-8-140992). The proposed surgical navigation device detects the tip position of a surgical instrument on a tomographic image from moment to moment and changes the pixel value on the image at the tip position to another pixel value (for example, red) different from the pixel value of the tissue image. By performing replacement in real time, the “surgery history” is displayed with the replacement pixel value on the image, and by storing it in the storage means, it is saved as a “history image”. With this configuration, the pixel value of the surgical instrument tip position on the image is replaced with a pixel value different from the image, for example, a red pixel value, and the momentary position of the surgical instrument tip during the operation indicates a red dot on the image. It is displayed as accumulated, and the range covered by the surgical operation can be confirmed from the accumulated display of red dots.

[0007]

According to the proposed surgical operation navigation device, the operation history can be displayed / stored in real time even during the operation without disturbing the operation operation. The distinction between the (organization) and the part (organization) that is not so is effective because it can be objectively judged at a glance.

However, in the proposed surgical navigation apparatus, the movement locus of the tip position of the surgical instrument inserted in the subject is represented by a single pixel value such as red regardless of the elapsed time (progress) of the surgery. Since the accumulated pixel value (red) is displayed as a history and stored in the image storage means, the operation time (e.g., surgery method or purpose Depending on), the following inconvenience may occur. Generally, the position of the subject is changed during surgery. Even in this case, if the surgery is continued before the posture change, the surgical operation is performed before the posture change, that is, the surgical operation is continuously performed from the site where the surgical instrument tip portion is located and the surgical operation is performed. Need to do.

However, the movement locus of the tip position of the surgical instrument inserted into the subject is converted into a single pixel value regardless of the elapsed time of the operation, and the accumulated pixel value (red) is displayed as a history. In addition, since it is stored in the image storage means, it is not possible to know the temporal context of each point, that is, the time course of the operation, when accumulating red points. Therefore, there is a problem that it is not possible to efficiently proceed with the operation because it is not known which part of the surgical operation was performed immediately before the posture change or in what order (procedure) the surgical operation was performed. This is the same even when the posture is not changed, even when the surgery is interrupted due to a bad situation or the like caused by a long-time major surgery.

Therefore, the present invention has been made in view of the above circumstances, and an object of the present invention is to obtain a surgical operation history image showing a temporal progress of the operation up to a certain point during the operation. An object of the present invention is to provide a surgical operation support device that can be presented to the operator without hindrance.

[0011]

In order to achieve the above object, a surgical operation support apparatus according to the present invention has the following configuration. That is, the position detection means detects the position of the surgical instrument inserted into the subject, and based on the detected position information, image data of a tomographic image in which the region where the distal end of the surgical instrument is located is imaged. A surgery support apparatus which is selected from image storage means and which is displayed by superimposing a predetermined pattern indicating the position of the surgical instrument tip on the selected image, the surgery being performed on the tomographic image detected by the position detection means. It is characterized in that the operation history image showing the operation elapsed time can be displayed by converting the pixel value at the position of the instrument into a different pixel value according to the operation elapsed time and displaying it.

Specifically, image storage means for storing the image data of the tomographic image of the region of interest of the subject, and real space coordinate value detection means for sequentially detecting the position of the distal end portion of the surgical instrument in the real space, Image space coordinate value conversion means for converting the position information from the real space coordinate value detection means into coordinate values in the image space;
An image selection unit that selects image data including a tomographic image in which a region where the distal end portion of the surgical instrument is located based on position information from the image space coordinate value conversion unit, from the image storage unit, Image combining means for superimposing a predetermined pattern indicating a surgical instrument on the position converted by the image space coordinate value converting means on the image selected by the image selecting means, and the image combined by the image combining means are displayed. A surgical operation support apparatus comprising: a display unit, wherein the position information of the surgical instrument detected by the real space coordinate value detection unit or the position information of the surgical instrument converted by the image space coordinate value conversion unit. The time storage means for storing the occurrence time and the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value conversion means are recorded in the time storage means. Has been provided a pixel value replacing means for replacing the pixel value corresponding to operation time elapsed from the generation time and the present time of the positional information, it is characterized in that to be able to view the surgical history image representing an operation elapsed time.

The pixel value replacing means calculates a pixel value different from the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value converting means, and the image space coordinate value. A pixel value replacement unit that replaces the pixel value at the position on the image of the surgical instrument converted by the conversion unit with the pixel value calculated by the pixel value calculation unit, and the generation time of the position information stored in the time storage unit. And a time difference calculation unit that calculates the operation elapsed time from the time difference of the current time, and the pixel value replacement unit calculates the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value conversion unit, the time difference. The pixel value is preferably replaced with a pixel value corresponding to the elapsed surgical time calculated by the calculation unit, and the pixel value calculation unit is a pixel value at a position on the image of the surgical instrument converted by the image space coordinate value conversion means. The picture Even if it is replaced with a uniform pixel value different from the value, or a new pixel value is calculated based on the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value conversion means. Further, a history image storage unit for storing the history image after the pixel value is replaced may be further provided.

It should be noted that the pixel value replacement means may be capable of selecting whether or not to replace with another pixel value in accordance with a surgical instrument to be used and / or an instruction of an operator,
Further, even if the history image storage means is capable of storing history images up to that point at any time during the operation, the pixel value replacement means further comprises:
As a result, even if a pixel value displayed in color on the display means or a color code is calculated,
It may be replaced (converted) with a different and uniform pixel value that is completely different from the image data stored in the image storage means. With this configuration, the pixel value calculation unit is omitted, or the pixel value replacement unit also serves as the pixel value calculation unit.

Further, it is preferable that the operation elapsed time or time difference is a unit time difference, and the time storage means detects the actual space coordinate value detecting means or the image space coordinate value converting means of the surgical instrument. The time when the position information is generated and the position information may be sequentially stored.

[Operation] The operation of the present invention is as follows. That is, the current position of the surgical instrument (position and direction of the distal end of the surgical instrument), which is inserted into the subject during the surgical operation and changes momentarily along with the progress of the operation, is sequentially detected by the real space coordinate value detecting means. Then, based on the position information of the surgical instrument detected by the real space coordinate value detecting means, the image space coordinate value converting means converts into coordinate values indicating the position in the image space. The image selecting means, based on the position information from the image space coordinate value converting means, obtains image data including a tomographic image in which the distal end portion of the surgical instrument is imaged from a tomographic image of a surgical site (region of interest) previously imaged. It is selected from the image storage means that stores the image data. At the same time, the position information from the image space coordinate value conversion means is supplied to the time storage means and the pixel value calculation section.

In the time storage means, the current position of the surgical instrument (the position and direction of the tip of the surgical instrument), which changes momentarily along with the progress of surgery, is sequentially detected by the real space coordinate value detecting means, and the image space coordinate value is obtained. The occurrence (detection) time of the momentary position information (coordinate value) in the image space converted by the converting means is stored based on the time information (time data) from the timer. The time storage means may store both the generation time of the position information and the position information (coordinate value), and also stores the generation time of the position information sequentially detected by the real space coordinate value detection means. It may be one that does.

The pixel value calculation means calculates a value for new replacement based on the pixel value at the position on the image space converted by the image space coordinate value conversion means, or is different from the pixel value of the image. The uniform pixel value is calculated, and the time difference calculation unit calculates the operation elapsed time from the time difference between the generation time of the position information stored in the time storage means and the time data from the timer. The pixel value replacement unit, based on the position information of the surgical instrument, a value of the pixel value newly calculated by the pixel value calculation unit, or
The pixel value of the image is replaced with a uniform pixel value, and the pixel value newly calculated and replaced by the pixel value calculation means or the uniform pixel value is calculated by the time difference calculation unit. Replace with the pixel value according to the elapsed operation time,
Immediately return to the image storage means.

The image synthesizing means sets the current position of the surgical instrument to the position converted by the image space coordinate value converting means on the image selected by the image selecting means and having the pixel values replaced by the pixel value replacing means. Overlay the given patterns shown. The combined image is displayed on the display means. The history image storage means retrieves the history image whose pixel values have been replaced by the pixel value replacement means from the image storage means in accordance with an instruction from the operator, and also stores the history image in another area of the image storage means.

Therefore, on the display means, the position of the surgical instrument during the surgical operation is converted into a pixel value according to the elapsed time of the surgery, and the pixel value of the displayed image indicates the history of the surgical time and the previous values. Images showing the surgical procedure are displayed in real time. As a result, even if the body position was changed during the operation or the temporary operation was interrupted due to a long-time operation, the region where the operation was performed when the position was changed or the operation was interrupted was more than the pixel value of the image. Since it can be recognized, it is possible to efficiently proceed with the surgery from the site where the surgery was previously performed, and the possibility of medical mistakes such as insufficient extraction and the need for re-operation are greatly reduced. Therefore, the burden on the operator and the patient can be reduced. In addition, a history image that directly reflects the history of surgery is created, which is one of the effective and highly objective materials at the time of postoperative examination.

For example, as shown in FIG. 7A, the relationship between the elapsed time of surgery and the pixel value to be converted is shown in FIG. 7A. The pixel value of A (color A) up to the minute before, 1 at the present time
The pixel value of B (B color) between 0 minutes and 20 minutes ago, the pixel value of C (C color) between 20 minutes and 30 minutes before the current time, and the pixel value of D from 30 minutes before the current time If it is (D color), the display or history image shows the procedure in which the surgical operation was performed as shown in (b) of the figure, that is, the surgical operation is performed 10 minutes before the current time (current time) t0. The area of the position of the surgical instrument is the A color point, and the area of the position of the surgical instrument that has been operated 10 to 20 minutes before the current time is the B color point, 20 to 30 minutes before the current time. The area of the position of the surgical instrument covered by the surgical operation is displayed as a C color point, and the area of the position of the surgical instrument covered by the surgical operation within 30 minutes from the present time is displayed as a set (accumulation) of the D color point. It will be.

Further, the display color (pixel value) of each display color is replaced based on the calculation time calculated by the time difference calculation unit, and after 10 minutes, the characteristic diagram of FIG. The color A in the history image of FIG. 7B is replaced with the color B, the color B is replaced with the color C, and the color C is replaced with the pixel value of the color D. Therefore, even if the surgery is interrupted by the display pixel value (display color), and if the surgery is interrupted due to postural change during surgery, or if the surgery is interrupted during a long operation, immediately before the interruption. It can be objectively judged from the display pixel value (display color) which part the surgical operation was performed,
As a result, even after the operation is resumed, it becomes possible to efficiently and effectively perform the operation successively from the site where the operation was performed immediately before the interruption.

The replacement of each color (pixel value) of B, C, and D is performed by the pixel value replacement unit based on the calculation time calculated by the time difference calculation unit, and the replacement of color A (pixel value) is performed by the pixel. A new pixel value calculated based on the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value conversion means by the value calculation unit, or the current pixel value converted by the image space coordinate value conversion means The pixel value replacement unit performs the pixel value replacement based on a uniform pixel value different from the pixel value of the image of the position of the surgical instrument. Even if the pixel value replacement is sequentially performed in real time based on the calculation time calculated by the time difference calculation unit, it takes 5 minutes or 10 minutes.
Minutes, 15 minutes, etc. may be collectively performed in batches every predetermined unit time elapses.

Further, as for the relationship between the elapsed time of surgery and the pixel value (color tone) to be converted, the progress of surgery is not set to a predetermined unit time (for example, 10 minutes) as shown in FIG. It is also possible to continuously change the pixel value or the color tone as time passes, as shown in FIG. However, it is more advantageous to replace the pixel value every time the unit time elapses as in the characteristic shown in FIG. 7A, in order to grasp the elapsed time (history) of the operation and the operation procedure.

[0025]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of an embodiment of a surgical operation support device according to the present invention. In FIG. 1, reference numeral 1
Is an operating table on which the subject M is placed. Reference numeral 2 is a surgical instrument such as a suction tube that is inserted into the subject M by an operator during surgery. A magnetic source 3a that generates a magnetic field in each of the three-dimensional directions is provided on the operating table 1 so that the relative positional relationship with the subject M does not change. A magnetic sensor 3b that detects a three-dimensional magnetic field generated from the magnetic source 3a is attached to the rear end of the surgical instrument 2.

As shown in FIG. 2, the magnetic source 3a is composed of three coils 3ax, 3ay and 3az which generate magnetic fields in the three-dimensional directions X, Y and Z, respectively. Magnetic source
3a is connected to the drive circuit 4 and generates a three-dimensional magnetic field H by receiving an alternating current from the drive circuit 4. Similarly, the magnetic sensor 3b has three coils 3b orthogonal to each other.
It consists of x, 3by, and 3bz. The magnetic sensor 3b is
It is connected to the detection circuit 5. The detection circuit 5 amplifies the detection signals from the coils 3bx, 3by, 3bz and outputs position data converted into appropriate digital signals. The magnetic source 3a, the magnetic sensor 3b, the drive circuit 4, the detection circuit 5, and the real space coordinate value detection unit 7 described later correspond to the real space coordinate value detecting means in the present invention.

The image processing computer 6 calculates the position of the image space coordinate value of the surgical instrument 2, selects an image according to the calculated position, synthesizes a pattern indicating the position of the surgical instrument 2 on the selected image, Further, the pixel value to be replaced is calculated, and the pixel value is replaced according to the elapsed operation time. When the internal configuration is roughly classified by function, the real space coordinate value detection unit 7 and the image space coordinate value calculation unit are described. 8, an image selection unit 9, an image composition unit 10, a pixel value calculation unit 11, a pixel value replacement unit 12,
A time memory 16 as a time storage means, a time difference detection unit 17,
It is composed of a time memory 16 and a timer 18 which gives current time data to the time difference detection unit 17. Note that it is realistic for the timer 18 to use the clock function of the image processing computer 6.

The real space coordinate value detection unit 7 calculates the three-dimensional position (real space coordinates) and direction of the magnetic sensor 3b based on the position data from the detection circuit 5, and further, the surgical instrument 2
Since the tip of the surgical instrument 2 is a fixed point with respect to the magnetic sensor 3b, the size of the surgical instrument 2 is taken into consideration by giving it as the initial data of the positional relationship in advance, and finally the actual tip of the surgical instrument 2 is considered. It obtains three-dimensional position coordinate values and directions in space, and corresponds to the real space coordinate value detecting means in the present invention. The magnetic source 3a and the magnetic sensor 3b described above are used.
A method of calculating the real space coordinates and the direction of the magnetic sensor 3b by using, for example, Japanese Patent No. 2653210 (Japanese Patent Laid-Open No.
No. 267054), the description thereof is omitted here.

The image space coordinate value calculation unit 8 determines the position of the surgical instrument 2 in the image space based on the position information (coordinate value: tip position and direction of the surgical instrument 2) from the real space coordinate value detection unit 7. The coordinate value of the direction is calculated and corresponds to the image space coordinate value conversion means in the present invention. The image selection unit 9 includes a tomographic image in which the region where the surgical instrument tip is located is imaged based on the coordinate value of the tip of the surgical instrument 2 in the image space given from the image space coordinate value calculation unit 8. The required image data (tomographic image) is selected from the image memory 13, and corresponds to the image selection means in the present invention.

The image memory 13 stores image data of a tomographic image for each position in the region of interest (surgical region) of the subject M, which is preliminarily imaged by a tomographic imaging device such as an X-ray CT device or an MRI device prior to surgery. The "history image" after the pixel value replacement unit 12 has replaced the pixel value is stored, and corresponds to the image storage means in the present invention. Specifically, a hard disk or a memory on the image processing hardware is assigned to this. The pixel value calculation unit 11 includes an image space coordinate value calculation unit 8
A new pixel value of the position of the surgical instrument 2 on the image is calculated based on the position information (coordinate values of the position and the direction of the surgical instrument 2 in the image space) calculated in step 3 and the image data selected by the image selection unit 9. It is to be calculated.

The pixel value replacing section 12 replaces the pixel value at the position on the image of the surgical instrument 2 with the value newly calculated by the pixel value calculating section 11. In the embodiment, the pixel value calculating section 11, In addition, the time difference detection unit 17 described later constitutes the pixel value replacement means in the present invention. Also, the image replacement unit
Reference numeral 12 has, for example, the time-to-pixel value characteristic shown in FIG. 7A, and the pixel value is explained in the section of the previous action according to the time difference from the present time by the time difference signal from the time difference detection unit 17 described later. As described above, the area of the position of the surgical instrument that has been operated 10 minutes before the current time (current time) t0 is colored A, and the operation that is 10 to 20 minutes before the current operation is performed. The area of the position of the instrument is in B color, the area of the position of the surgical instrument that has been operated 20 to 30 minutes before the present time is in C color,
The area of the position of the surgical instrument that has been operated 30 minutes before the present time is switched to D color.

The image synthesizing unit 10 has an image space coordinate value calculating unit 8 on the image in which the pixel values at the positions selected by the image selecting unit 9 and operated by the pixel value replacing unit 12 have been replaced. A predetermined pattern indicating the position of the surgical instrument 2 (the tip position of the surgical instrument,
And / or patterns indicating directions) are overlapped and correspond to the image synthesizing means in the present invention. The image combined by the image combining unit 10 is given to the monitor TV 14 corresponding to the display means of the present invention. The history image storage unit 15
The “history image” after the pixel values at the position where the surgical operation is applied is replaced by the pixel value replacement unit 12 is stored in another area of the image memory 13 according to the instruction of the operator. It corresponds to a storage means.

The time memory 16 includes an image space coordinate value calculation unit 8
The time when the position information is generated is stored based on the clock signal from the timer 18 together with the position information from, and corresponds to the time storage means in the present invention. The time difference calculation unit 17
The operation elapsed time is calculated based on the time stored in the time memory 16 and the time data from the timer 18, and the time difference signal is given to the pixel value replacing unit 12.

The operation of the apparatus of this embodiment will be described below. In order to display the position of the surgical instrument 2 in the apparatus having the above configuration, it is necessary to collect and store tomographic images in the image memory 13 before operating the apparatus. FIG. 3 shows the image memory 13
It is explanatory drawing about collection of the tomographic image memorize | stored in FIG.

Cross-sectional images of a plurality of slice planes S _{1 to} S _n so as to include the region of interest of the subject M are photographed by an MRI apparatus or the like. Figure 3 is a schematic view of the resulting tomographic image I ₁ ~I _n (b)
Shown in. These tomographic images I ₁ ~I _n are stored are assigned appropriate unique file name (image file name) in the image memory 13. At this time, at the same time, the reference point on the image for "space correspondence" for associating the position of the surgical instrument in the real space with the position in the image space is also recorded. In this space correspondence, the image data collected and stored in the image memory 13 is M
It is represented by the unique spatial coordinates (CT coordinate system) of a tomographic apparatus such as an RI apparatus. The unique spatial coordinates of this CT coordinate system and the real space coordinates of the surgical instrument position display device (surgical instrument). Required for creating a conversion matrix for converting the coordinate values of the surgical instrument in the real space into the coordinate values in the image space, in order to associate the coordinate values with the coordinate system based on the real space coordinate value detection means). Processing.

There are several methods for spatial correspondence, and the reference points to be recorded differ depending on the method used. In the case of the method using a marker attached on the body surface of the subject, a tomographic image is collected with the marker attached, and the marker coordinate values drawn on the image are recorded at the same time as the image data. Keep it. Without using markers
To adopt the method in which an arbitrary point on the surface of the subject is used as a reference point for conversion, the coordinate values of all the points forming the contour of the subject are recorded from the collected tomographic image. The former method is disclosed in Japanese Patent No. 1848083 (Japanese Patent Publication No. 5-55141).
No.) and Japanese Patent No. 2653210 (JP-A-3-26705).
No. 4) and the latter method is disclosed in
No. 98847, the description is omitted here. At the same time as the recording of the reference points for spatial correspondence, the average value of all pixel values is calculated for each image and recorded.

The average value of all pixel values for each image is calculated by
As shown in the flowchart of FIG. 4, image data is obtained from the image memory 13 (step S1), and the average value of all pixel values in the image is calculated (step S2). To calculate the average value,
For example, it is calculated by adding all pixel values and dividing the added value by the total number of pixels. The calculated average value is stored in the image memory 13 (step S3), and the calculation of the average value in steps S1 to S3 and the storing operation in the image memory 13 are performed for all images (tomographic images) stored in the image memory 13. Image I ₁
~I _n) is performed on (step S4). The average value of the pixel values of each image obtained in this way is used during the operation to calculate a new pixel value for replacement. When the apparatus of this embodiment is applied to surgery, the magnetic source 3a is fixed to the subject M, and then the reference point position in the real space for spatial correspondence is obtained.

When the method using the marker attached to the surface of the subject as a reference point is adopted, the magnetic sensor 3b
The marker is pointed at the tip of the surgical instrument 2 to which is attached, and the coordinate value in the real space is obtained. When an arbitrary point on the surface of the subject is used as the reference point, the tip of the surgical instrument 2 is appropriately pointed to the surface of the subject, and the coordinate values in the real space are obtained.
A conversion matrix is created based on the coordinate values of the reference points in the real space thus obtained and the previously recorded coordinate values of the reference points in the image space, and the image space coordinate value calculation unit 8 Give to. When the tomographic image acquisition, the calculation / recording of the average value of all pixel values for each image, and the spatial correlation are completed as described above, the operation is started.

During surgery, the coordinate values of the surgical instrument 2 in the real space are always sent from the real space coordinate value detecting section 7 at predetermined time intervals. The image space coordinate value calculation unit 8 converts the coordinate value of the surgical instrument 2 in the real space into the coordinate value of the position and direction of the surgical instrument in the image space using the above conversion matrix. Image selector 9
Selects and extracts an image from the image memory 13 according to the coordinate value of the tip position of the surgical instrument 2 in the image space.

In parallel with the image selecting operation of the image selecting section 9, the pixel value calculating section 11 calculates a new pixel value to be replaced. FIG. 5 is a flowchart showing the operations of the pixel value calculation unit 11 and the pixel value replacement unit. First, the coordinate values in the image space of the tip position and the insertion direction of the surgical instrument 2 calculated by the image space coordinate value calculation unit 8 are obtained (step S1).
0). Next, an image corresponding to the coordinate values on the image of the tip position and direction of the surgical instrument 2 selected by the image selection unit 9 is obtained, and among the pixels forming the image, the image is inserted into the subject. A pixel value that matches the coordinate value on the image of the tip position of the surgical instrument 2 is obtained (step S11).

If the surgical instrument currently in use is a surgical instrument that performs a surgical operation or a surgical instrument that does not add a surgical operation such as an endoscope even if the tip position is on the tissue, the pixel value is The calculator 11 does not calculate the pixel value, returns to step S10, obtains the position of the instrument at the next moment, and repeats the same operation (step S12), but the instrument currently in use is the suction tube, and If it is necessary to keep the history of the tip position, processing for pixel value replacement is performed. Further, even if the suction tube is used, if the operator judges that it is not necessary to record the history and informs the intention to the apparatus with a keyboard or the like, the processing is not performed and the step S10 is performed. Then, the same operation is repeated (step S13). It should be noted that both steps S12 and S13 are judgments as to whether or not it is necessary to keep a history, so both steps can be combined.

Next, if it is judged that it is necessary to leave a history through steps S12 and S13, if the unit time (for example, 10 minutes) has not elapsed from the start of the operation, the image space coordinate value calculation unit 8 calculates The pixel value at which the tip position of the surgical instrument 2 exists and the average value of all the pixels calculated and stored previously are obtained (step S15), and the pixel value calculation unit 11 calculates a new pixel value by the following equation in step S16. To do. Inew = Iaverage− | Iold−Iaverage | However, Inew: pixel value that is newly replaced Iaverage: average value of all pixels Iold: pixel value before applying surgical operation

Usually, in an image obtained by an MRI apparatus, an X-ray CT apparatus, or the like, a tissue such as a tumor that requires a surgical operation is visualized at a high signal using a contrast agent, which is clearly higher than the average value of all pixels. They often have large pixel values. According to the above equation, the new pixel value is clearly lower than the average value of all pixels because the pixel value is folded back with respect to the average value of all pixels as shown in FIG. As a result, the tissue to which the surgical operation has been applied is depicted in good contrast to the tissue that does not. The pixel value replacement unit 12 replaces the pixel value that matches the coordinates of the tip of the surgical instrument 2 in the image space with the new pixel value calculated in this way (step S
17) The image whose pixel value has been replaced is immediately overwritten in the image memory 13, and when the image selection unit 9 selects next time, the image whose pixel value has been replaced until that time is selected.

The image synthesizing unit 10 has a predetermined pattern (surgical pattern) indicating the position of the surgical instrument 2 at the position calculated by the image space coordinate value calculating unit 8 on the image whose pixel value is replaced by the pixel value replacing unit 12. Patterns indicating the tool tip position and direction) are overlaid. Then, the combined image is displayed on the monitor 14. If the operator instructs to save the history image during or after the operation, the history image saving unit 15
Reads the images up to that point from the image memory 13,
The image is copied to an empty area on the image memory 13. Further, when a save instruction is given at another time, the image is copied to a free area of the image memory 13 different from the previous one. In this way
Each time the instruction to save the history image is issued, the history image up to that point is saved.

By sequentially executing the above-described processing in real time, the monitor TV 14 displays the current position of the surgical instrument 2 in a predetermined pattern and is operated by that time.
The images with the pixel values replaced are sequentially displayed.

On the other hand, the elapsed operation time is calculated by the time difference calculation unit 17 based on the momentary occurrence time of the position information of the surgical instrument stored in the time memory 16 and the current time data from the timer 18, and this calculation It is determined whether a unit time (for example, 10 minutes) has elapsed based on the determined time (step S14), and when the unit time has elapsed, the pixel value calculated and replaced by the pixel value calculation unit 11 is replaced by the pixel value replacement. The pixel value corresponding to a unit time (for example, 10 minutes) is replaced by the unit 12 (step S18), and the surgical instrument 2 is placed at the position calculated by the image space coordinate value calculation unit 8 on the image with the replaced pixel value. A predetermined pattern indicating the current position of (the pattern indicating the tip position and direction of the surgical instrument) is superimposed on the image combining unit 10, and the combined image is displayed on the monitor 14.

As a result, as shown in FIG. 7 (b), the area of the position of the surgical instrument that the surgical operation has reached within 10 minutes before the current time (current time) is the A color point, and 10 minutes before the current time. The position of the surgical instrument that was operated 20 minutes ago was the B color point, and the position of the surgical instrument that was operated 20 minutes to 30 minutes ago was the C color point, and it was 30 minutes from the current time. The position of the surgical instrument that has been operated before the minute is displayed as a set (accumulation) of D color points. The A color region is a new pixel value calculated based on the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value conversion unit by the pixel value calculation unit, or the image space coordinate value. The pixel value is a uniform pixel value different from the pixel value of the image of the current position of the surgical instrument converted by the conversion means, and is performed by the pixel value pixel value replacement unit. In addition, the B, C, and D color regions are processed by the pixel value replacement unit based on the calculation time calculated by the time difference calculation unit, and are batch-processed in real time according to the elapsed operation time or every unit time elapsed. The pixel values are replaced with.

Therefore, when the operation is interrupted by the display pixel value (display color) of the display image, and the operation is interrupted due to the postural change during the operation, or when the operation is interrupted during the operation for a long time. Also, it is possible to objectively judge which part the surgical operation was performed at the time of the interruption from the display pixel value (display color), and it is possible to continue the surgery from the part where the surgical operation was performed at the time of the interruption even after resuming the surgery. Become.

[Modified Embodiment] 1) In the embodiment, FIG.
In (b), the pixel value of a region indicating the position of the surgical instrument from the start of surgery to 10 minutes (a new pixel value to be replaced indicating a surgical site where history should be retained) is set to the pixel value before the surgical operation and all pixels of the image. Although it is calculated from the average value of the values, the present invention is not limited to this, without calculating for each pixel,
It may be replaced with a maximum value or a minimum value of all pixels obtained in advance, or a uniform fixed pixel value different from the pixel value indicating the lapse of operation time. Further, the image may be displayed in a color tone different from the color tone representing the image of the treated region, and the point is that the image of the treated region may be distinguished from the images of other regions. When a new pixel value to be replaced is replaced with a uniform pixel value without being calculated from the pixel value of the image, the pixel value calculation unit in FIG. 1 is unnecessary.

2) In the embodiment, the operator decides whether or not to leave a history. For example, a range centered on the tip position of the surgical instrument is set as the region of interest, and the region of interest is set within that region. If it has been present for a predetermined time (about 30 seconds) or more, it may be automatically determined that the operation has been performed. 3) In the embodiment, the image photographed by the MRI apparatus is directly stored in the image memory. However, even if the film of the tomographic image photographed by the MRI apparatus is read by a film reader or the like and stored in the image memory, Alternatively, it may be read from a recording medium such as a magneto-optical disk and stored in the image memory.

4) In the embodiment, the position of the surgical instrument is detected by using the magnetic source and the magnetic sensor. However, the present invention is not limited to this. For example, the surgical instrument may be attached to the tip of an articulated arm. The position and orientation of the surgical instrument may be detected by mounting and detecting the angle of the arm at each joint. With such a configuration, it is possible to use an instrument composed of a magnetic body or a conductor that disturbs the magnetic field formed by the magnetic source. 5) In the embodiment, the surgical instrument is used for spatial correspondence, and the tip of the surgical instrument is used to point to the body surface of the subject. However, a spatial correspondence probe equipped with a magnetic sensor is used separately from the surgical instrument. Therefore, the body surface of the subject may be indicated. In this way, the surgical instrument can be kept clean.

[0052]

As is apparent from the above description, according to the present invention, a historical image of the position of the surgical instrument during the surgical operation over time is displayed on the monitor without disturbing the surgical operation. From the displayed image, it is possible to at a glance and objectively recognize or determine the site where the surgical operation was performed when the posture was changed or the operation was interrupted, and the operation history up to that point.

As a result, even if the posture is changed during the operation, or even if the temporary operation is interrupted by a long-time operation, the operation can be efficiently performed continuously from the site where the operation was performed for the interruption. It is possible to proceed, and since the time history of the surgical operation (surgical procedure) can be objectively determined by that time, it is possible to proceed with the surgery efficiently and faithfully, and when the extraction is insufficient, etc. The possibility of medical errors and the need for re-operation are greatly reduced, which can contribute to reducing the burden on the operator and patient.

In addition, an objective history image that directly reflects the time course of the position of the surgical instrument during the surgical operation, in other words, the history of the surgery, is created, and the history image is saved for postoperative examination. It becomes one of the influential materials when doing.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of a surgical operation support device according to an embodiment.

FIG. 2 is an explanatory diagram of a magnetic source and a magnetic sensor.

FIG. 3 is a diagram for explaining acquisition of a tomographic image.

FIG. 4 is a flowchart showing an operation of calculating an average value of all pixel values for each image.

FIG. 5 is a flowchart showing operations of pixel value calculation and pixel value replacement.

FIG. 6 is an explanatory diagram of a pixel value replacement operation.

FIG. 7 is a diagram for explaining the operation of the embodiment apparatus.

[Explanation of symbols]

1: Operating table 2: Surgical instrument 3a: Magnetic source 3b: Magnetic sensor 4: Drive circuit 5: Detection circuit 6: Image processing computer 7: Real space coordinate value detection unit 8: Image space coordinate value calculation unit 9: Image selection unit 10: Image composition unit 11: Pixel value calculation unit 12: Pixel value replacement unit 13 Image memory 14: Monitor TV 15: History image storage unit 16: Time memory 17: Time difference detection unit 18: Timer

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) G01R 33/28 A61B 5/05 390 G02B 23/24 G01N 24/02 Y F term (reference) 2H040 DA03 GA02 GA10 GA11 4C093 AA22 CA24 FF15 FF19 FF20 FF28 FF35 FF37 FF41 FG01 FG13 FG16 GA01 4C096 AA18 AB50 AD14 AD19 AD23 DC14 DC22 DC37 DD07 EA04 FC20

Claims (7)

[Claims] 1. A tomographic image in which a position of a surgical instrument inserted into a subject is detected by a position detecting means, and a region where a distal end portion of the surgical instrument is located is imaged based on the detected position information. A surgical assistance apparatus for selecting image data from image storage means and superimposing and displaying a predetermined pattern indicating the position of a surgical instrument on the selected image, the slice of the surgical instrument being detected by the position detecting means. A surgery support device characterized in that a pixel value at a position on an image is replaced with a different pixel value according to the elapsed time of surgery and displayed, and a surgery history image showing the elapsed surgery time can be displayed. 2. An image storage unit that stores image data of a tomographic image of a region of interest of a subject, a real space coordinate value detection unit that sequentially detects a position of a surgical instrument distal end in a real space, and the real space coordinates. An image space coordinate value conversion means for converting the position information from the value detection means into a coordinate value in the image space, and a portion where the surgical instrument tip is located based on the position information from the image space coordinate value conversion means. An image selecting means for selecting imaged image data from the image storing means, and a predetermined pattern indicating a surgical instrument at a position converted by the image space coordinate value converting means on the image selected by the image selecting means. A surgery assisting apparatus comprising: an image synthesizing means for superimposing and a display means for displaying an image synthesized by the image synthesizing means, the surgery being detected by the real space coordinate value detecting means. Time information storage means for storing the position information of the instrument or the generation time of the position information of the surgical instrument converted by the image space coordinate value conversion means, and on the image of the surgical instrument converted by the image space coordinate value conversion means. Pixel value replacement means for replacing the pixel value at the position with a pixel value corresponding to the time of occurrence of the position information stored in the time storage means and the elapsed operation time from the current time is provided, and the operation history representing the elapsed operation time is provided. An operation support device characterized in that an image can be displayed. 3. The surgery support apparatus according to claim 1 or 2, wherein the surgery elapsed time is a unit time. 4. The surgery support apparatus according to claim 2, wherein the time storage means is position information of the surgical instrument detected by the real space coordinate value detection means, or the image space coordinate value conversion means. A surgical operation supporting device, wherein the position information is stored together with the generation time of the position information of the surgical instrument converted by. 5. The surgical operation support apparatus according to claim 2, wherein the pixel value replacement unit is a position on the image of the surgical instrument converted by the image space coordinate value conversion unit. A pixel value calculation unit that calculates a pixel value different from the pixel value,
A pixel value replacement unit that replaces the pixel value of the position on the image of the surgical instrument converted by the image space coordinate value conversion unit with the pixel value calculated by the pixel value calculation unit, and the position information stored in the time storage unit. And a time difference calculation unit that calculates an operation elapsed time from the time difference between the occurrence time and the current time, and the pixel value replacement unit is a pixel value at a position on the image of the surgical instrument converted by the image space coordinate value conversion unit. Is replaced with a pixel value according to the surgery elapsed time calculated by the time difference calculation unit. 6. The surgical operation support apparatus according to claim 5, wherein the pixel value calculation unit is a pixel value at a position on the image of the (current) surgical instrument converted by the image space coordinate value conversion unit. Is replaced with a uniform pixel value different from the pixel value of the image. 7. The surgical operation support apparatus according to claim 5, wherein the pixel value calculation unit is based on a pixel value of a position on the image of the surgical instrument converted by the image space coordinate value conversion means. Surgery assisting device, which calculates various pixel values.