JP4468672B2 - Surgery support system - Google Patents

Description

  The present invention relates to a surgery support system that supports surgery in real time with reference to images.

  Usually, an operator in the operating room performs an operation on a patient. For example, when it is necessary for an operator in the operating room to perform an operation with little experience in the past, Connect to the operator (remote support surgeon) in the conference room, for example, in the hospital away from the operating room via a line, and give instructions on the part to be removed by the remote support surgeon during the operation. A system that assists the patient in the operating room to perform an appropriate operation by allowing the operator in the operating room to perform an operation with remote support so that the patient can receive it. Such a remote support system is proposed in, for example, Japanese Patent Laid-Open No. 2002-306509.

In such a support system, the support side performs support by referring to a related image such as a CT image or MR image of a patient together with a live image of a procedure. Further, in such a support system, a procedure image in the operating room is recorded. For example, an endoscopic image is recorded on an in-hospital server or the like when an affected area is observed using an endoscopic apparatus or a treatment is performed, and the post-operative operation is performed in the in-hospital conference room support apparatus in which the support system is constructed. Endoscopic images are reproduced and subsequent guidance on the procedure is performed.
JP 2002-306509 A

  However, in the conventional support system, related images that can be referred to are displayed as thumbnails. However, since the number of images that can be displayed in the display area is limited, it is difficult to find thumbnails of desired related images. .

  The present invention has been made in view of the above circumstances, can efficiently search thumbnails of a plurality of related images that can be referred to, and can provide appropriate support by displaying original images of the searched related images. The purpose is to provide a surgical operation support system.

The surgery support system of the present invention is a surgery support system including a support device that provides support information to a surgical device performing a procedure.
The support device includes:
Image data obtaining means for obtaining a plurality of image files having a live image during the procedure and a plurality of related images related to the procedure taken before the procedure;
File thumbnail generation means for generating a plurality of file thumbnail images of the plurality of image files;
First allocating means for allocating the plurality of file thumbnail images to a first display area of a display screen;
A related image thumbnail generating unit configured to generate the plurality of related image thumbnail images of the plurality of related images included in the plurality of image files;
A second allocation means for allocating the plurality of related image thumbnail images to a second display area of a display screen;
Thumbnail selection means for selecting one of the plurality of file thumbnail images allocated to the first display area or the plurality of related image thumbnail images allocated to the second display area;
Third allocation means for selectively allocating the original image of the thumbnail image selected by the thumbnail selection means and the live image to the third display area of the display screen;
When the live image is selected by the third allocating means, attention site selection means for selecting a site of interest on the live image displayed in the third display area;
Superimposing means for superimposing a marker graphic on the position of the target region on the live image selected by the target region selecting unit;
Equipped with a,
The surgical device comprises:
In the support device, the live image is selected by the third allocating unit, the site of interest on the live image displayed in the third display area is selected by the site of interest selecting unit, and the superimposing unit When a marker graphic is superimposed on the position of the target region on the live image selected by the target region selection unit and information on which the marker graphic is superimposed is received from the support device, the information is based on the information. Display control means for displaying the live image on which the marker graphic is superimposed on the display means on the surgical apparatus side is provided.
It is characterized by that.

  According to the present invention, there is an effect that thumbnails of a plurality of related images that can be referred to can be efficiently searched, and appropriate support can be provided by displaying an original image of the searched related images.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIGS. 1 to 12 relate to the first embodiment of the present invention, FIG. 1 is a configuration diagram showing the configuration of the surgery support system, and FIG. 2 is displayed on the operating room display device and conference room endoscope image monitor of FIG. FIG. 3 is a block diagram showing a functional configuration of the support apparatus of FIG. 1, FIG. 4 is a diagram showing an integrated screen displayed on the support monitor of FIG. 1, and FIG. 5 is a diagram of FIG. FIG. 6 is a first diagram of an integrated screen for explaining the processing of FIG. 5, FIG. 7 is a second diagram of an integrated screen for explaining the processing of FIG. 5, and FIG. FIG. 9 is a fourth view of the integrated screen for explaining the processing of FIG. 5, FIG. 10 is a fifth view of the integrated screen for explaining the processing of FIG. 11 is a sixth view of an integrated screen for explaining the process of FIG. 5, and FIG. 12 is a display device in the operating room for explaining the process of FIG. It is a diagram showing an endoscopic image of a live being displayed.

  As shown in FIG. 1, the surgical operation support system employed in this embodiment is installed in a hospital having a base such as an operating room 100, a CT examination room 101, an MR examination room 102, a conference room 103, etc. The first controller 1 that obtains various medical information that is generated and the operating room 100 are installed at remote locations, and collects various medical treatment information obtained by the first controller 1 and generates a medical chart. 2 and the in-hospital server 3 as patient information storage means that is installed at a base different from the surgical department server 2 and refers to the medical record information obtained by the surgical department server 2, and supports the procedure in the operating room 100 The operating room 100 is configured to include a support device 104 provided in a conference room 103 installed at a remote location, and the first controller 1 and the CT examination room 10. The first and second communications between the CT examination apparatus, the MR examination apparatus of the MR examination room 102, the support apparatus 104 and the surgical department server 2 and between the surgical department server 2 and the in-hospital server 3, respectively. Connected by lines 4 and 5.

  Each of the communication lines 4 and 5 may be any communication line such as a public line, a dedicated line, or a LAN (local communication network). Further, the in-hospital server 3 may be disposed at the same base as the surgical department server 2.

  The second controller 14 captures a video signal obtained by the endoscope apparatus 11 and an endoscope apparatus 11 that obtains a video signal including an endoscopic image by imaging a test site such as a body cavity of a patient. Display device 12 and a surgical device 13 such as an electrosurgical device, a pneumoperitoneum device, and an ultrasonic surgical device, for example, to control the surgical device 13 or to obtain endoscopic image data from the endoscope device 11 Surgical device information such as measurement information by the surgical device 13 is obtained.

  The second controller 14 is connected to a surgical device control input means such as a touch panel 15 for inputting a control instruction or the like for the surgical device 13 to the second controller 14.

  Endoscopic image data, surgical device information, and patient information obtained from the living body monitoring device 19 provided from the second controller 14 are accumulated in the first controller 1 via a communication line, and set in another base. Is transmitted to the operating unit server 2 via the communication line 4.

  The video signal obtained from the endoscope apparatus 11 and the video signal obtained from the indoor camera 20 that captures the scenery in the operating room 100 are input to the signal transmission device 17 via the first controller 1. And transmitted to the outside via the communication line 6.

  Further, the operating room information obtained by the first controller 1 in the operating room, patient information obtained from the operating section server 2, integrated image data which is a support image obtained from the support apparatus 104 in the conference room 103, A display device 18 for displaying information received by the signal transmission device 17 from another base is provided.

  An input device 1 a is connected to the first controller 1. The input device 1a is a keyboard, a magnetic card reader, an optical card reader, an IC card reader, an RF-ID (IC tag) reader, or the like. Using this input device 1a, a chart number (ID number) and a name related to a patient Procedure information such as patient information, procedure name, and operator ID is input. In this case, patient information and procedure information are transmitted from the first controller 1 to the second controller 14.

  Note that the input device 1 a may be connected to the second controller 14. When the input device 1 a is connected to the second controller 14, patient information and procedure information are transmitted from the second controller 14 to the first controller 1.

  The surgical department server 2 is connected to the hospital server 3 and the WEB server 21 via the communication line 5. The WEB server 21 is also connected to the outside of the hospital via the Internet. When the patient is examined at another medical institution outside the hospital, the examination image data (MR, CT, X-ray image, etc.) ) Is stored in the out-of-hospital servers 201 and 202 in the medical institution, but when the WEB server 21 accesses the out-of-hospital servers 201 and 202 via the Internet, the support device 104 and the The inspection image data stored in the out-of-hospital servers 201 and 202 can be downloaded to the first controller 1.

  The in-hospital server 3 includes patient registration information from the patient registration terminal 22 of the hospital, medical images (MR, CT, X-ray images, etc.) such as preoperative images related to the patient, and related images (internal) Endoscope still images, endoscope moving images) are accumulated.

  Prior to the surgery, the surgery department server 2 reads and stores registration information such as the name, date of birth, and medical record of the patient performing the surgery and medical image information related to the patient from the hospital server 3 via the communication line 5. Can be done.

  The support device 104 provided in the conference room 103 includes an endoscope image monitor 111 that displays an endoscopic image obtained by the endoscope device 11 in the operating room 100, and a support image generated by the support device 104. And a support monitor 112 that displays a certain integrated screen (described later), and the endoscope image monitor 111 is configured to receive an endoscope through the second controller 14, the first controller 1, and the operating room communication line 4. The live endoscopic image captured by the apparatus 11 is displayed, and the live endoscopic image, the medical image related to the patient stored in the surgical operation section server 2, and downloaded via the WEB server 21. An integrated screen that is a support image using medical images stored in the out-of-hospital servers 201 and 202 is displayed on the support monitor 112.

  As shown in FIG. 2, the display device 12 in the operating room 100 and the endoscopic image monitor 111 in the conference room 103 have a display area in which a live endoscopic image (hereinafter, endoscopic live image) is a full screen. 121 is displayed.

  As shown in FIG. 3, the support device 104 in the conference room 103 includes an integrated screen generation function 151, a related / live switching function 152, a thumbnail original image display function 153, and a video playback function 154 as first to third allocation means. , An enlarged cursor superimposing function 155, a thumbnail image generating function 156 as a file thumbnail generating means and a related image thumbnail generating means, a data communication function 157 as an image data obtaining means, and the like, and the support monitor 112 has a mouse or a touch panel. A pointing device (not shown) such as the above is provided to construct a GUI (thumbnail selection means), and various functions are stored in the data storage unit 150 and read / written so that the above functions can be performed by the GUI of the support monitor 112. The processing based on is performed. Details of the processing of each function will be described later.

  The operation of this embodiment configured as described above will be described. The support apparatus 104 acquires patient information including registration information such as a patient's name, date of birth, medical record, and medical image information related to the patient from the surgical operation section server 2 by the data communication function 157.

  Next, the integrated screen generation function 151 accesses the CT examination apparatus in the CT examination room 101, the MR examination apparatus in the MR examination room 102, and the out-of-hospital servers 201 and 202. Download from the CT examination apparatus 101, the MR examination apparatus in the MR examination room 102, and the out-of-hospital servers 201 and 202.

  When downloading of the related image data is completed, a thumbnail image is generated from the related image data by the thumbnail image generation function 156.

  Specifically, the related image data is stored in a plurality of types of medical image files (endoscopic moving image file, endoscope still image file, CT still image file, MR still image file). Some of them are composed of a single image file such as a still image, and others are composed of a plurality of frame images such as a moving image file.

  Therefore, the thumbnail image generation function 156 generates related image data, that is, a “thumbnail image” indicating a medical image file and a “related thumbnail image” that is a thumbnail of each medical related image file data.

  In the present embodiment, for example, the “thumbnail image” is matched with the thumbnail of the first medical related image file data among a plurality of “related thumbnail images”.

  When thumbnails are generated in this way, the integrated screen generation function 151 generates an integrated screen 160 as shown in FIG. 4 and displays it on the support monitor 112 in step S6.

  As shown in FIG. 4, the integrated screen 160 displays an image display area 161 for displaying an endoscope live image or the like in normal display, and a thumbnail for displaying a plurality of “thumbnail images” indicating respective medical image files of related image data. An image display area 162, a related thumbnail image display area 163 for displaying a plurality of “related thumbnail images” which are thumbnails of the respective medical related image file data in the related image data, and a key area 164 including various keys SW are provided. The key area 164 is provided with at least a related / live switch SW165.

  Next, the operation on the integrated screen 160 will be described. As shown in FIG. 5, when the integrated screen 160 is displayed on the support monitor 112 in step S <b> 1, the cursor 168 is moved to any one of the thumbnail image display areas 162 in the GUI as the thumbnail selection unit of the support monitor 112 in step S <b> 2. It is determined whether or not “thumbnail image” has been selected. When “thumbnail image” is selected, the image in the image display area 161 is integrated with the original image of “thumbnail image” selected by the thumbnail original image display function 153 in step S3. A screen 160 is generated.

  For example, as shown in FIG. 4, when an image in the thumbnail image display area 162 is selected as the CT still image file 1 thumbnail image 171, the image in the image display area 161 is displayed as a CT still image by the thumbnail original image display function 153 as shown in FIG. The original image of the image file 1 thumbnail image 171 is used. In the present embodiment, as described above, the “thumbnail image” matches the thumbnail of the first medical-related image file data among a plurality of “related thumbnail images”, so that the image display area 161 in FIG. The image is an original image of the CT still image file 1 related image 1 thumbnail image 172 in the related thumbnail image display area 163. In addition, when the “related thumbnail image” on the related thumbnail image display area 163 is selected with the cursor 168, the original image of the selected “related thumbnail image” is displayed on the cursor 168.

  Here, by displaying the frame of the CT still image file 1 thumbnail image 171 and the frame of the CT still image file 1 related image 1 thumbnail image 172 in a thick frame, it is possible to recognize which thumbnail is the original image.

  Further, when selecting a “thumbnail image” in step S19 using a related image as a moving image, the moving image is composed of a plurality of continuous scene moving images. For example, as shown in FIG. When the image 175 is selected, the image in the image display area 161 is made the original image of the endoscope moving image file 1 thumbnail image 175 by the thumbnail original image display function 153 as shown in FIG. At this time, in the related thumbnail image display area 163, the first frame images of a plurality of scene moving images constituting the moving image are displayed as “related thumbnail images”. As a result, in this embodiment, the image in the image display area 161 is the endoscope moving image file 1 frame image 1.

  In the integrated screen 160 in the state of FIG. 8, a moving image playback operation key area 180 is displayed in the lower part of the image display area 161. When the playback button 181 in the movie playback operation key area 180 is selected, the playback image of the endoscope movie file 1 is displayed as a movie in the image display area 161 by the movie playback function 154 as shown in FIG.

  Returning to FIG. 5, it is determined whether or not the cursor 168 has selected the related / live switching SW 165 by the GUI of the support monitor 112 in step S <b> 4. When the related / live switching SW 165 is selected, the related / live switching function 152 is selected in step S <b> 5. Thus, the endoscope live image and the original image of the selected “thumbnail image” are switched to obtain an integrated screen 160 as an image in the image display area 161, and the process proceeds to step S6. If the related / live switch SW165 is not selected, the process directly returns to step S6.

  In step S6, it is determined whether the image displayed in the image display area 161 is an endoscope live image. If the image is not an endoscope live image, the process returns to step S4, and if it is an endoscope live image, the process proceeds to step S7.

  In step S7, the cursor 168 is moved by the GUI of the support monitor 112 on the endoscope live image displayed in the image display area 161 as shown in FIG. When the right cursor is clicked and selected), the enlarged cursor 191 is superimposed and displayed at the position on the endoscopic live image of the affected part selected in step S8 as shown in FIG. The position information on the mirror live image is transmitted to the first controller 1.

  In step S10, an instruction to end the integrated screen display (support end) is awaited. If there is an instruction, the process ends. If there is no instruction, the process returns to step S2.

  In addition, the 1st controller 1 which received the positional information on the endoscope live image of an affected part by step S9 is shown in the endoscope live image currently displayed on the display area 121 of the display apparatus 12, as shown in FIG. The enlarged cursor 191a is displayed in a superimposed manner on the upper affected part position.

  As described above, in this embodiment, a plurality of “thumbnail images” indicating files of related image data are displayed in the thumbnail image display area 162, and the “thumbnail image” in the thumbnail image display area 162 is selected, thereby A plurality of “related thumbnail images”, which are thumbnails of medical related image file data in the related image data file, can be expanded and displayed in the thumbnail image display area 163, and a desired medical related image can be displayed by performing such hierarchical display. Can be easily searched, and by making the medical related image the image display area 161, the medical related image can be referred to and appropriate support can be performed.

  In addition, when an endoscopic live image is displayed in the image display area 161, an enlargement cursor can be superimposed on the display device 12 on the operating room side at the position of interest such as an affected area, so Appropriate instructions can be given without obstructing the observation with the endoscope live image.

  The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

The block diagram which shows the structure of the surgery assistance system which concerns on Example 1 of this invention. The figure which shows the live endoscopic image displayed on the display apparatus and conference room endoscopic image monitor of the operating room of FIG. The block diagram which shows the function structure of the assistance apparatus of FIG. The figure which shows the integrated screen displayed on the support monitor of FIG. The flowchart explaining the effect | action of the assistance apparatus of FIG. 1st figure of the integrated screen explaining the process of FIG. 2nd figure of the integrated screen explaining the process of FIG. 3rd figure of the integrated screen explaining the process of FIG. 4th figure of the integrated screen explaining the process of FIG. The 5th figure of the integrated screen explaining the process of FIG. 6th figure of the integrated screen explaining the process of FIG. The figure which shows the live endoscopic image displayed on the display apparatus of an operating room explaining the process of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... 1st controller 1a ... Input device 2 ... Surgical department server 3 ... Hospital server 4,5 ... Communication circuit 11 ... Endoscope apparatus 12 ... Display apparatus 13 ... Surgical apparatus 14 ... 2nd controller 103 ... Conference room 104 ... support device 150 ... data storage unit 151 ... integrated screen generation function 152 ... related / live switching function 153 ... thumbnail original image display function 154 ... video playback function 155 ... enlarged cursor superimposition function 156 ... thumbnail image generation function 157 ... data communication function 160 ... Integrated screen Agent Patent attorney Susumu Ito

Claims (1)

In a surgery support system including a support device that provides support information to a surgical device performing a procedure,
The support device includes:
Image data obtaining means for obtaining a plurality of image files having a live image during the procedure and a plurality of related images related to the procedure taken before the procedure;
File thumbnail generation means for generating a plurality of file thumbnail images of the plurality of image files;
First allocating means for allocating the plurality of file thumbnail images to a first display area of a display screen;
A related image thumbnail generating unit configured to generate the plurality of related image thumbnail images of the plurality of related images included in the plurality of image files;
A second allocation means for allocating the plurality of related image thumbnail images to a second display area of a display screen;
Thumbnail selection means for selecting one of the plurality of file thumbnail images allocated to the first display area or the plurality of related image thumbnail images allocated to the second display area;
Third allocation means for selectively allocating the original image of the thumbnail image selected by the thumbnail selection means and the live image to the third display area of the display screen;
When the live image is selected by the third allocating means, attention site selection means for selecting a site of interest on the live image displayed in the third display area;
Superimposing means for superimposing a marker graphic on the position of the target region on the live image selected by the target region selecting unit;
With
The surgical device comprises:
In the support device, the live image is selected by the third allocating unit, the site of interest on the live image displayed in the third display area is selected by the site of interest selecting unit, and the superimposing unit When a marker graphic is superimposed on the position of the target region on the live image selected by the target region selection unit and information on which the marker graphic is superimposed is received from the support device, the information is based on the information. Display control means for displaying the live image on which the marker graphic is superimposed on the display means on the surgical apparatus side is provided.
An operation support system characterized by that .

Images