WO2006070441A1

WO2006070441A1 - Operator supporting system

Info

Publication number: WO2006070441A1
Application number: PCT/JP2004/019545
Authority: WO
Inventors: Hideyuki Majima
Original assignee: Celltec Project Management Co., Ltd.
Priority date: 2004-12-27
Filing date: 2004-12-27
Publication date: 2006-07-06

Abstract

An operating room terminal comprises a point indication position detector which has an image display provided with a predetermined display screen and an input operation receiver adapted for receiving an input operation and so disposed at a place as not to obstruct the display screen of the image display and can detect the position of a point indicating object such as a finger tip present within a surface area close and parallel to the display screen in a noncontact way and command generating means for generating an operation command corresponding to the position of the point indicating object detected by the point indication position detector and the moving mode. A three-dimensional image is rotated through the operation of moving the point indicating object such as a finger tip along the display screen, and a desired cross-section image of the patient on the three-dimensional image is displayed on the screen of the image display in the operating room.

Description

Specification

Surgeon support system

Technical field

TECHNICAL FIELD [0001] The present invention relates to a surgeon support system suitable for a doctor or dentist performing a surgical operation, and more particularly to a surgeon support system using an interactive image display device that displays an in-vivo image of a patient. .

Background art

Conventionally, as a surgeon support system, a radiograph of a patient taken before surgery is pasted on the front of an illumination panel in the operating room, and the surgeon refers to it appropriately. Was common. The X-ray image that can be observed with such a system is defined by the body angle at the time of imaging, so even if you look at another angle force inside the surgery, you can not respond to it. .

[0003] On the other hand, based on 3D image data obtained by diagnosis using a CT device, an MRI device, or a PET device, interactively observing a patient's body using an image display device is It is performed at various medical diagnosis sites (for example, see Patent Document 1).

Patent Document 1: Japanese Patent Laid-Open No. 2002-534204

Disclosure of the invention

Problems to be solved by the invention

[0004] However, the interactive medical support system using the image display device described above uses an operation unit for office workers such as a keyboard and a mouse as the operation unit. For surgeons with treatment tools, they can't be used or handled as they are.

[0005] Power! In other words, conventional image display systems are often designed solely for the convenience of the diagnostician. In particular, the guide screen design has considerable room for improvement in terms of operability. It is left.

[0006] The present invention has been made paying attention to the above-mentioned problems, and the object of the invention is

, An interactive system that uses an image display device. In particular, it is to provide a surgeon support system that is easy to use.

[0007] Still other objects and operational effects of the present invention will be easily understood by those skilled in the art based on the following description of the specification.

Means for solving the problem

[0008] A surgeon support system according to the present invention includes an operating room terminal device that functions as a man-machine interface and is installed in the vicinity of an operating table in an operating room, and a diagnostic device such as a CT device, an MRI device, and a PET device. An image storage device storing the 3D image data obtained by diagnosing a predetermined part of the patient via the computer, and the 3D image data read out from the image storage device and designated in advance. Alternatively, an image processing device that generates image display data viewed from a predetermined viewpoint by performing image processing specified each time by an operation command and displays an image corresponding to the terminal device in the operating room. Contains. The diagnostic equipment includes ultrasonic diagnostic equipment and interventional radiology (IVR) equipment.

[0009] The operating room terminal device has an image display having a predetermined display screen and an input operation accepting device for receiving an input operation. The input operation acceptor includes a point indication position detector and a point indication position detector that can detect the position of a point indication object such as a fingertip existing in a plane region close to and parallel to the display screen in a non-contact manner. Command generating means for generating a corresponding operation command based on the position of the point pointing object detected in step 1 and its movement mode.

[0010] The image processing apparatus generates a stereoscopic image with a cursor obtained by superimposing a predetermined planar cursor that is sequentially updated according to a cursor movement operation on a solid image corresponding to three-dimensional image data related to a predetermined part of the patient. In response to the first image processing means that generates corresponding image display data and displays it on the image display device of the operating room terminal device, and the first operation command that also receives the input operation acceptor power of the operating room terminal device. Second image processing means for generating image display data corresponding to a state in which the stereoscopic image with the cursor is rotated by the direction and amount specified by the operation command and displaying the data on the surgeon's terminal device. And third image processing means for generating image display data corresponding to a cross-sectional image designated by a predetermined planar cursor after rotation and displaying the data on a terminal device in the operating room. Have at least.

[0011] According to the present invention, a three-dimensional image of a predetermined part suitable for a patient's surgery can be displayed on the surgeon's terminal screen. In addition, since the input operation is performed in a non-contact manner using a fingertip or the like as a point pointing object, the surgeon can easily and freely operate the screen without using an input device such as a mouse or a keyboard. In addition, the frequency of touching devices other than patients and treatment equipment is greatly reduced, so hygiene is greatly improved.

In one embodiment of the present invention, the predetermined plane cursor is an orthogonal three-plane cursor, whereby cross-sectional images corresponding to the three orthogonal planes are simultaneously displayed. Accordingly, by moving the orthogonal three-plane cursor, the three orthogonal cross sections inside the stereoscopic image of the predetermined part of the patient can be easily displayed.

In a preferred embodiment of the present invention, in response to the second operation command, image display data corresponding to an enlarged image of one cross section corresponding to the planar cursor specified by the command is transmitted. There is further provided an image processing apparatus that further includes a fourth image processing means and that can enlarge and display a desired cross-sectional image on the stereoscopic image with a point pointing object such as a fingertip. Thereby, the desired cross-sectional image on the stereoscopic image of the patient can be easily displayed on the screen of the image display.

[0014] In a preferred embodiment of the present invention, the display screen of the image display device includes a three-dimensional image, a cross-sectional image designated by each plane cursor of the three-dimensional image, and one enlarged cross-sectional image. Can be displayed by the area division method. In addition, a three-dimensional image with an orthogonal three-plane cursor and an enlarged image of one cross-section are displayed with almost the same size, and three cross-sectional images are displayed. As a result, since one stereoscopic image, three cross-sectional images, and one enlarged cross-sectional image are displayed simultaneously on one screen, each image can be referred to as a list. In addition, since the stereoscopic image and the enlarged cross-sectional image are displayed with almost the same size, it is easy to compare them. Therefore, it is difficult for errors in the range and size of the corresponding part to occur.

[0015] In a preferred embodiment of the present invention, the image processing device performs image processing that is designated in advance or designated each time by an operation command on one or more types of three-dimensional image data. Thus, image display data viewed from a predetermined viewpoint is generated, and the data is transmitted to the surgeon side terminal device. This allows the surgeon to store each 3D image By selecting the data, a three-dimensional image of a predetermined part of the patient suitable for the operation can be created, which can be used as a simulation when performing the operation, and a more effective operation can be performed.

In one preferred embodiment of the present invention, the one or more types of three-dimensional image data are obtained by imaging a predetermined part of a patient via a diagnostic apparatus such as a CT apparatus, an MRI apparatus, or a PET apparatus. Three types of 3D image data. This makes it possible to combine imaging data from diagnostic devices with different types of image data, and to display a 3D image of a specific part of the patient suitable for surgery, so the surgeon performs more accurate surgery. be able to.

[0017] The surgeon support system according to the present invention as seen from another corner diagnoses a predetermined part of a patient via a terminal device functioning as a man-machine interface and a diagnostic device such as a CT device, an MRI device, or a PET device. An image storage device storing the obtained 3D image data, and image processing that is read from the image storage device and is designated in advance or by an operation command for the patient's 3D image data. And an image processing device that generates image display data viewed from a predetermined viewpoint and displays an image corresponding to the operating room side terminal device.

[0018] The terminal device includes an image display having a predetermined display screen and an input operation acceptor for accepting an input operation.

[0019] The input operation accepting device is in a position that does not block the display screen of the image display device, and the position of a point pointing object such as a fingertip that is close to and parallel to the display screen in a non-contact manner. And a command generating means for generating a corresponding operation command based on the position of the point pointing object detected by the point pointing position detector and its movement mode.

[0020] The image processing apparatus generates a three-dimensional image with a cursor obtained by superimposing a predetermined planar cursor that is sequentially updated according to a cursor movement operation on a three-dimensional image data corresponding to three-dimensional image data related to a predetermined part of the patient. The first image processing means for generating corresponding image display data and displaying it on the image display device of the operating room terminal device, and the input operation acceptance power of the terminal device in response to the incoming first operation command , The stereoscopic image with the cursor Image display data corresponding to the state rotated by the direction and amount specified by the operation command is generated and displayed on the terminal device.

And at least second image processing means and third image processing means for generating image display data corresponding to a cross-sectional image designated by a predetermined planar cursor after rotation and displaying the data on a terminal device. Yes.

[0021] According to the present invention, a stereoscopic image of a predetermined part suitable for a patient's surgery can be displayed on the terminal screen. Also, since the input operation is performed without touching the position of the point indicating member with a fingertip or the like, the surgeon can operate without using an input device such as a mouse or a keyboard, and the screen operation can be performed easily and freely. It can be performed.

The invention's effect

[0022] According to the present invention, a patient's in-vivo simulation is performed using a stereoscopic image with a plane force and a cross-sectional image specified by each plane cursor via a diagnostic apparatus such as a CT apparatus, an MRI apparatus, or a PET apparatus. Can be realized more accurately. In addition, the input operation receiver on the surgeon's terminal device allows the surgeon to move the fingertip etc. along the display screen as a pointed object without using an input device such as a mouse or keyboard. Thus, it is possible to provide a surgeon support system that can display a desired cross-sectional image on a stereoscopic image of a patient on the screen of an image display in the operating room.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, an embodiment of a surgeon support system according to the present invention will be described in detail with reference to the accompanying drawings.

A system configuration diagram of the surgeon support system according to the present invention is shown in FIG. As shown in the figure, this surgeon support system comprises an image processing device, a surgeon's terminal device 2, a CT device 3, an MRI device 4, a PET device 5, an auxiliary storage device 6, and a data output device 7. Speak. These system elements are connected by a system bus or LAN cable (not shown).

[0025] The image processing apparatus 1 generates image display data by performing image processing on 3D data obtained by imaging a predetermined part of a patient via the diagnostic apparatus 3-5 described above. The created image display data is sent to the surgeon's terminal device. [0026] The surgeon-side terminal device 2 includes an image display having a predetermined display screen and an input operation accepting device for receiving an input operation. The image display is preferably a flat display such as LCD, PDP, or organic EL. In the image display, the image data sent from the image processing apparatus 1 is displayed. The input operation acceptor accepts an input operation such as rotation on the display image, and the operation command generated during the input operation is sent to the image processing apparatus 1 described above.

[0027] The contents of the operation command will be described in detail later. Further, although the embodiment of the present invention will be described with reference to the surgeon-side terminal device, it goes without saying that the present invention can also be implemented with a terminal device installed outside the operating room equipped with the input operation accepting device.

[0028] The CT apparatus 3, the MRI apparatus 4, and the PET apparatus 5 are medical diagnostic imaging apparatuses conventionally used. The basic operation of CT, MRI, and PET devices has already been widely known from various documents, and a description thereof will be omitted.

Next, FIG. 2 shows in detail an example of an input operation acceptor and an image display of the terminal device according to the present embodiment. As shown in the figure, the terminal device 2 includes an image display D, a pointed position detector 21, command generation means 22, a housing 23, a spacer 24, and a retroreflective member 25. .

[0030] As the image display D, a CRT image display or a liquid crystal image display is used. On the image display D, one stereoscopic image created by the image processing apparatus 1, three cross-sectional images, and one enlarged cross-sectional image are displayed.

[0031] The point indicating position detector 21 detects the position of a point indicating object such as a fingertip existing in a plane area close to and parallel to the display screen in a non-contact manner. Further, in order to improve the detection accuracy, one point pointing position detector 21 is attached to each of the upper left and right, and is connected to the command generating means 22.

[0032] The command generation means 22 calculates the coordinate position and the movement distance of the point pointing object from the signal detected by the point pointing position detector 21, creates an operation command based on the calculation result, Sends an operation command to the image processing apparatus 1.

Next, FIG. 3 shows a cross-sectional view of the input operation acceptor of the terminal device 2 of the present embodiment. As shown in the figure, on the screen display D, acrylic resin, polycarbonate resin A transparent input plane plate 26, which is also configured with equal force, is arranged in an overlapping manner. Further, the input flat plate 26 is fixed to the housing 23 via the spacer 24. Spacers 24 are arranged on the upper, lower, left and right sides of the housing 23. The retroreflective member 25 is disposed on the inner wall of each side of the spacer 24 so as to be perpendicular to the input flat plate 26.

[0034] The point indicating position detector 21 is fixed to the housing 23 via a spacer 24, and includes a light source 27 such as an infrared LED, a prism 28, an imaging lens 29, an imaging means 30 and the like. Yes. In the imaging means 30, the retroreflected light reflected by the optical axis L projected and received from the light source 27 by the operation of the point pointing object F such as the fingertip hits the retroreflective member 25 passes through the prism 28 and the imaging lens 29. Then, imaging is performed. The retroreflected light that has been imaged is converted into an electrical signal. From the electrical signal obtained by the imaging means 30, the command generation means 22 calculates the coordinate position and operation pattern of the point pointing object F, and a suitable operation command is created.

Next, FIG. 4, FIG. 5, and FIG. 8 to FIG. 14 are flow charts showing the parts related to the main part of the present invention. Hereinafter, these flowcharts, FIG. 6, FIG. 7, and FIG. —The operation of the system of this embodiment will be described systematically with reference to the screen display example shown in FIG.

FIG. 4 shows a general flow chart of processing of the surgeon support system constituting the system of the present embodiment. As shown in the figure, this processing includes start processing (step 401), stereoscopic image G1 creation processing (step 402), cross-sectional image G2 creation processing (step 403), and enlarged cross-sectional image G3 creation processing. (Step 404) and screen display processing (Step 40)

5), command analysis processing (step 407), image storage processing (step 408), and end processing (step 409).

[0037] When the display data is displayed on the screen (step 405), the image processing apparatus 1 is in a standby state until an operation command is received from the surgeon-side terminal apparatus 2 (step 4).

06).

Furthermore, in the image storage process (step 408), when the operation command from the surgeon-side terminal device 2 is “image storage”, the stereoscopic image is stored at the designated location. In the end process (step 409), if the operation command from the surgeon side terminal device 2 is “end”, the end process of the surgeon support system is performed and the display screen is closed. Details of the start process (step 401) are shown in FIG. As shown in the figure, when the surgeon support system is activated, a system menu screen is displayed (step 50 1). An example of this system menu screen is shown in FIG. As shown in the figure, the system menu screen SG1 includes a new creation button 601 and a button 602 for opening a created file. When the system menu screen SG1 is displayed, it enters a standby state until either the new creation button 601 or the created file open button 602 is pressed (step 502NO).

[0040] When any one of these buttons is pressed (step 502 YES), the type of the pressed button is identified, and when the pressed button is the new creation button 601 (step 503 new creation), a new one is created. The creation menu screen is displayed (step 504).

[0041] This new creation menu screen is shown in FIG. As shown in the figure, the new creation menu screen G2 includes an image creation source CT device button 701, an image creation source MRI device button 702, an image creation source PET device button 703, and an image creation source. Device button 704, X—Y plane cursor button 705, Y—Z plane cursor button 706, Z—X plane cursor button 707, all plane cursor buttons 708, create execution button 709, and created file Open button 710 is provided. The surgeon can select 3D image data for creating a stereoscopic image by pressing an image creation source designation button 701-704. Further, by pressing the plane cursor designation buttons 705 to 708, an orthogonal plane cursor to be superimposed on the stereoscopic image can be selected. When the new creation menu screen SG2 is displayed, the process waits until the creation execution button 709 is pressed (step 505 NO).

[0042] When the creation execution button 709 is pressed (step 505 YES), the type of the image creation source device selected on the new creation menu screen S G2 is temporarily stored in the image processing device 1 (step 506). The shooting direction of the selected 3D image data is temporarily stored in the image processing apparatus 1 (step 507). Further, the coordinates where the plane cursor force selected on the newly created menu screen SG2 intersects are set, and the setting contents are temporarily stored in the image processing apparatus 1 (step 508).

[0043] System menu screen When the button 602 to open the created file of SG1 is pressed (Step 503 has been created), an image selection dialog (not shown) is displayed (Step 509), and the process waits until a stereoscopic image file is selected (Step 510).

When a stereoscopic image file is selected (step 510 YES), image information including 3D image data type, stereoscopic image direction, and planar cursor coordinates is read (step 511). Image information is temporarily stored in the image processing apparatus 1 (step 512).

Next, details of the stereoscopic image G1 creation process (step 402) are shown in FIG. As shown in the figure, when the stereoscopic image G1 creation process is executed, the image processing apparatus 1 determines whether or not it is an initial operation (step 801). In the case of the initial operation (step 801F 1 = “0”), the 3D image data of the original diagnostic device temporarily stored in the start process (step 401) is read, and the stereoscopic image G1 is read in a predetermined direction. Created (step 80 2). When the stereoscopic image G1 is created, similarly, the planar cursor information set in the start process (step 401) is read, and a planar cursor passing through predetermined coordinates is additionally created in the stereoscopic image G1 ( Step 803). When the processing of the initial operation is completed, the flag F1 is set to “1” (step 804).

In the second and subsequent operations (step 801F1 = “1”), the operation command analyzed in the command analysis process (step 407) described later is determined (step 805). When the command is “rotation of a solid image” (step 805 rotation), a stereoscopic image is recreated according to the angle calculated from the movement distance of the point pointing object described above (step 806). When the command is “change stereoscopic image data” (step 805, change stereoscopic image data), a stereoscopic image is recreated from the three-dimensional image of the designated diagnostic device (step 807). In the case of commander S “move / jump” (step 805 move, jump), the 3D image is recreated so that the location specified by the above-mentioned point pointing object is in front (step 808). ). When the stereoscopic image G1 is re-created, a plane cursor passing through the specified coordinates is additionally created (step 809).

Next, details of the cross-sectional image G2 creation process (step 403) are shown in FIG. As shown in the figure, when the cross-sectional image G2 creation process is started, the image processing apparatus 1 determines the coordinates through which the XY plane created in the stereoscopic image G1 passes and the 3 selected in the start process. A sectional image G21 is created from the dimensional image data (step 901). Furthermore, a cross-sectional image G22 is created from the coordinates through which the Y—Z plane created in the stereoscopic image G1 passes and the 3D image data selected in the above-mentioned start processing (step 902), and similarly the Z—X plane. A cross-sectional image G23 is created from the coordinates through which and pass and the selected 3D image data (step 903).

Next, details of the enlarged cross-sectional image G3 creation process (step 404) are shown in FIG.

As shown in the figure, when the enlarged cross-sectional image G3 creation process is executed, the image processing apparatus 1 determines whether or not the initial operating force is sufficient (step 1001). In the case of the initial operation (step 1001F3 = “0”), the X-Y plane cross-sectional image G21 is enlarged (step 1002). When the initial operation processing is completed, the F3 flag is set to "1" (step 1003).

[0049] In the case of the second and subsequent operations (step 1001F3 = "1"), command analysis processing described later

The cross-sectional image selection command analyzed in (Step 407) is determined (Step 1004). When the command is “X—Y plane” (step 1004G21), the cross-sectional image G21 of the XY plane is enlarged (step 1005). When the command is “Y-Z plane” (step 1004G22), the cross-sectional image G22 of the Υ-Ζ plane is enlarged (step 1006). When the command is “Ζ-Χ plane” (step 1004G23), the cross-sectional image G23 of the Ζ-Χ plane is enlarged (step 1007). If the content of the operation command is other than a cross-sectional image (step 1004 is not specified), the cross-sectional image selected previously is enlarged and created (step 10 08).

Next, FIG. 11 shows details of the command creation processing (step 407). As shown in the figure, the image processing apparatus 1 is in a standby state until an operation command signal from the input operation acceptor power of the surgeon-side terminal apparatus 2 is received (step 1101). When the movement of the point pointing object is detected by the point pointing position detector (step 1101 YES), the type of movement force operation of the point pointing object is determined (step 1102). When the operation type is “touch operation” (step 1102 touch), touch processing is executed (step 1103). When the operation type is “drag operation” (step 1102 drag), the drag process is executed (step 1104). When the operation type is “rotation operation”, image rotation processing is executed (step 1105). Next, details of the touch process (step 1103) are shown in FIG. As shown in the figure, when the touch process is executed, the touch location is detected from the detected touch coordinates (step 1201). When the touch location is “cross-sectional image” (step 1201 cross-sectional image), it is analyzed as an enlarged cross-sectional image display switching command of the selected cross-sectional image (step 1202). When the touch location is “stereoscopic image” (step 1201 stereoscopic image), it is analyzed as a jump display command of a stereoscopic image that displays and moves the specified location of the stereoscopic image to the front (step 1203). When the touch location is “stereoscopic image source switching button” (switching of step 1201), it is analyzed as a stereoscopic image creation source switching command for selecting a diagnostic device as a stereoscopic image creation source (step 1204). In the case of the touch location force ^ save button (step 1201 save), it is analyzed as an image save command for saving the current 3D image information with a plane cursor and cross-sectional image information (step 1205). When the touch location is the “end button” (step 1201 end), it is analyzed as an operation end command to end the surgeon support system (step 1206).

Next, FIG. 13 shows details of the drag process (step 1104). As shown in the figure, when the drag process is executed, the drag location is detected from the detected drag coordinates and the movement distance of the point pointing object (step 1301). If the drag location is “plane cursor” (step 1301 plane cursor), the specified plane cursor is analyzed as a plane cursor movement command to move in the drag direction (step 1302). When the drag location is “stereoscopic image” (step 1301 stereoscopic image), the stereoscopic image is analyzed as a stereoscopic image movement command for moving in the drag direction (step 1303).

Next, details of the image rotation processing (step 1105) are shown in FIG. As shown in the figure, when the image rotation process is executed, the rotation of the image in the left-right direction is detected from the detected movement and movement distance of the pointed object (step 1401). When a horizontal rotation is detected, a vertical rotation is detected (step 1402 and step 1405). As a result of detection of left / right and up / down motions, a right forward rotation command is analyzed in the case of a right forward rotation (step 1403). In the case of right rear rotation, it is analyzed as a right rear rotation command (step 1404). Similarly, in the case of left front rotation, it is analyzed as a left front rotation command (step 1406), and in the case of left rear rotation, it is analyzed as a left rear rotation command. (Step 1407).

Finally, a specific screen display example according to the present invention is shown in FIG. As shown in the figure, the stereoscopic image G1 with a plane cursor, the cross-sectional image G21 specified with the X—Y plane cursor, the cross-sectional image G22 specified with the Y—Z plane cursor, and the Z—X plane The cross-sectional image G23 designated by the cursor, the enlarged image of one designated cross-section, and each operation button are simultaneously displayed on the screen G. In addition, the stereoscopic image G1 with the cursor and the enlarged image G3 of one cross section are displayed almost the same size as the patient. Each operation button includes a button 1501 for opening a frequently used folder, a save button 1502, an end button 1503, an image information button 1504 for displaying stereoscopic image information, and a display of the color of the stereoscopic image. Adjustment buttons such as an automatic density button 1505 for performing adjustment, a contrast high / low button 1506, a light / dark button 1507, and an enlarge / reduce button 1508 are provided. Furthermore, an example in which screen G is displayed on the surgeon's terminal device 2 is shown in FIG. As shown in the figure, the screen G described above is displayed on the screen display of the surgeon-side terminal device 2. The point pointing position detector 21 attached to the upper part of the surgeon-side terminal device 2 uses a point pointing object such as a finger to touch each image on the screen G or a portion where each button is displayed. By dragging or rotating, the position information of the pointed object is detected.

Next, an example in which the present surgeon support system is arranged in the operating room is shown in FIG. The surgeon's terminal 2 is attached to the ceiling in the operating room R via an arm. Since the position of the surgeon-side terminal device 2 is moved by the arm, the surgeon-side terminal device 2 can be positioned at a place where the surgeon Do can easily operate. Therefore, even when performing surgery on patient P lying on operating table T, it can be easily operated.

As described above, the present surgeon support system according to the above-described embodiment is a three-dimensional image data obtained by diagnosing a predetermined site of a patient via a diagnostic apparatus such as a CT apparatus, an MRI apparatus, or a PET apparatus. By combining the data, a life-size stereoscopic image suitable for surgery can be displayed. In addition, a cross-sectional image corresponding to each of the three orthogonal planes and a life-size enlarged image of one cross-section are displayed on one screen at the same time, together with a three-dimensional image with an orthogonal three-plane cursor. It can be realized more accurately.

[0057] Further, the surgeon support system does not use an input device such as a keyboard or a mouse. In addition, the touch operation, the drag operation, and the rotation operation are performed by a point indicating object such as a finger, so that operability is improved and hygiene is improved.

Industrial applicability

[0058] According to the present invention, a patient's in-vivo simulation is performed using a stereoscopic image with a plane force and a cross-sectional image specified by each plane cursor via a diagnostic apparatus such as a CT apparatus, an MRI apparatus, or a PET apparatus. Can be realized more accurately. In addition, the input operation receiver on the surgeon's terminal device allows the surgeon to move the fingertip etc. along the display screen as a pointed object without using an input device such as a mouse or keyboard. Thus, it is possible to provide a surgeon support system that can display a desired cross-sectional image on a stereoscopic image of a patient on the screen of an image display in the operating room.

Brief Description of Drawings

FIG. 1 is a block diagram showing a hardware configuration of the system according to the present embodiment.

FIG. 2 is an explanatory diagram showing a terminal device according to the present embodiment.

FIG. 3 is a cross-sectional view showing an input operation acceptor of the terminal device of the present embodiment.

FIG. 4 is a general flowchart showing a software configuration of the system of the present embodiment.

FIG. 5 is a flowchart showing details of start processing.

FIG. 6 is an explanatory diagram showing a display example of a menu screen of the terminal device.

FIG. 7 is an explanatory diagram showing a display example of a new image menu screen of the terminal device.

FIG. 8 is a flowchart showing details of a stereoscopic image G1 creation process.

FIG. 9 is a flowchart showing details of a cross-sectional image G2 creation process.

FIG. 10 is a flowchart showing details of an enlarged cross-sectional image G3 creation process.

FIG. 11 is a flowchart showing details of command analysis processing.

FIG. 12 is a flowchart showing details of touch processing in command analysis processing.

FIG. 13 is a flowchart showing details of a drag process in the command analysis process.

FIG. 14 is a flowchart showing details of image rotation processing in command analysis processing.

FIG. 15 is an explanatory diagram showing a display example of a simulation screen.

FIG. 16 is an explanatory diagram showing a display example in which a simulation screen is displayed on the terminal device of the present embodiment. [17] It is an explanatory diagram showing an example of an operating room in which the present surgeon support system is introduced. Explanation of symbols

1 Image processing device

2 Surgeon's terminal

3 CT device

4 MRI equipment

5 PET equipment

6 Auxiliary storage

7 Data output device

21-point indication position detector

22 Command generation means

23 Housing

24 Spacer

25 Retroreflective members

26 Input flat plate

27 Infrared light source

28 Prism

29 Imaging lens

30 Imaging means

601 New creation button

602 Button to open created file

701 CT device button

702 MRI device button

703 PET device button

704 All device buttons

705 X—Y plane cursor button

706 Y—Z plane cursor button

707 Z—X plane cursor button 708 All plane cursor buttons

709 Create button

710 Open button for created file

1501 Open folder button

1502 Save button

1503 Exit button

1504 Image information button

1505 Automatic density button

1506 Contrast strong 'weak' button

1507 Bright '喑 button

1508 Zoom button

D Image display

F point indicating object

G Simulation screen

G1 3D image with planar cursor

G21 X—Y plane cross-sectional image

G22 Y—Z-plane cross-sectional image

G23 Z—Section image of the X plane

SG1 menu screen

SG2 New menu screen

R operating room

Do surgeon

P patient

T operating table

Claims

The scope of the claims

An intra-operative terminal device installed near the operating table in the operating room and functioning as a man-machine interface;

An image storage device that stores 3D image data obtained by diagnosing a predetermined part of a patient via a diagnostic device such as a CT device, an MRI device, or a PET device;

Image display data viewed from a predetermined viewpoint is applied to the patient's three-dimensional image data read from the image storage device by performing image processing specified in advance or specified each time by an operation command. An image processing device for generating and displaying an image corresponding to the terminal device in the operating room

A surgeon support system including:

The operating room terminal device has an image display device having a predetermined display screen, an input operation accepting device for receiving an input operation, and

The input operation acceptor is in a position that does not obstruct the display screen of the image display, and can detect the position of a pointed object such as a fingertip that is close to and parallel to the display screen in a non-contact manner. A point pointing position detector;

Command generating means for generating a corresponding operation command based on the position of the point pointing object detected by the point pointing position detector and its movement mode, and

The image processing device

Generates image display data corresponding to a stereoscopic image with a cursor obtained by superimposing a predetermined planar cursor that is sequentially updated according to the cursor movement operation on the stereoscopic image corresponding to the three-dimensional image data related to the predetermined part of the patient. First image processing means for displaying on the image display device of the operating room terminal device,

Input operation acceptor force of the operating room terminal device In response to the first operation command that arrives, image display data corresponding to the state in which the stereoscopic image with the cursor is rotated by the direction and amount specified by the operation command Second image processing means for generating and displaying on the surgeon's terminal device;

Third image processing means for generating image display data corresponding to a cross-sectional image designated by a predetermined planar cursor after rotation and displaying it on the terminal device in the operating room is reduced. Have tochi,

As a result, the body image can be rotated through an operation of moving the fingertip or the like along the display screen as a point pointing object so that a desired cross-sectional image on the stereoscopic image of the patient can be displayed on the screen of the image display in the operating room. Surgeon support system characterized by that.

[2] The surgeon assistance according to claim 1, wherein the predetermined plane cursor is an orthogonal three-plane cursor, whereby cross-sectional images corresponding to each of the three orthogonal planes are simultaneously displayed. system.

[3] Input operation acceptor force of the operating room terminal device In response to the incoming second operation command, image display data corresponding to an enlarged image of one cross section corresponding to the planar cursor specified by the command is generated. And a fourth image processing means for displaying on the terminal device in the operating room.

As a result, the desired cross-sectional image on the patient's stereoscopic image can be enlarged and displayed on the screen of the image display device through the operation of moving the fingertip or the like along the display screen as a point indicating object. The surgeon support system according to claim 2.

[4] A feature is that a stereoscopic image with an orthogonal three-plane cursor and three cross-sectional images specified by each plane cursor are displayed on the display screen of the image display device using the area division method. The surgeon support system according to claim 2.

[5] Three-dimensional image with orthogonal three-plane cursor, three cross-sectional images specified by each plane cursor, and enlarged image of one cross-section are displayed on the display screen of the image display by the region division method. The surgeon support system according to claim 2, wherein:

[6] A three-dimensional image with an orthogonal three-plane cursor and an enlarged image of one cross-section are displayed almost in life, and the three cross-section images specified by each plane cursor are displayed smaller than that. The surgeon support system according to claim 5, wherein:

[7] The image processing apparatus reads out the image storage power and performs image processing that is pre-designated or specified each time by an operation command for one or more types of 3D image data related to the same part of the same patient. The surgeon support system according to any one of claims 1 to 6, characterized in that image display data viewed from a predetermined viewpoint is generated and sent to a terminal device in the operating room.

[8] One or more types of 3D image data are three types of 3D image data obtained by imaging a predetermined part of a patient via a CT, MRI, or PET diagnostic device. The surgeon support system according to claim 7 characterized by.

[9] A terminal device that functions as a man-machine interface;

Image display data viewed from a predetermined viewpoint can be obtained by performing image processing specified in advance or by an operation command on the patient's 3D image data read from the image storage device. An image processing device that generates and displays a corresponding image on the operating room terminal device;

A surgeon support system including:

The terminal device has an image display having a predetermined display screen, an input operation accepting device for accepting an input operation, and

The image processing device

In response to the incoming first operation command, the terminal device generates image display data corresponding to the state in which the stereoscopic image with the cursor is rotated by the direction and amount specified by the operation command. A second image processing means for displaying on the terminal device,

Image display corresponding to a cross-sectional image specified by a predetermined planar cursor after rotation At least a third image processing means for generating data and displaying it on the terminal device, so that the body image is rotated through an operation of moving the fingertip or the like along the display screen as a point pointing object. A surgeon support system characterized in that a desired cross-sectional image on a stereoscopic image can be displayed on the screen of an image display of a terminal device.