JP2012223363A - Surgical imaging system and surgical robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a surgical imaging system and a surgical robot, allowing high-accuracy drive control of an imaging means to a desired position according to the motion of an operator in any work environment.SOLUTION: This surgical imaging system 1 including the imaging means 23 imaging the inside of a body cavity and an image display means 11 on which an image imaged by the imaging means 23 is displayed includes: an imaging position control means 22 controlling a drive position of the imaging means 23; and a marker detection means 12 detecting positions of markers 11b worn on the operator 14. A drive amount and a drive direction of the imaging means 23 is determined based on movement amounts and movement directions of the markers 11b detected by the marker detection means 13a, and the imaging position control means 22 controls the drive position of the imaging means 23 based on the determined drive amount and drive direction.

Description

  The present invention relates to a surgical imaging system and a surgical robot.

  Minimally invasive surgical operations are widely performed for the purpose of reducing pain to patients and making surgical scars after surgery inconspicuous. In minimally invasive surgery, for example, an endoscope and a surgical manipulator (for example, forceps) are used by an operator or the like. Specifically, the surgeon or the like inserts and drives an endoscope, a surgical manipulator or the like from an insertion hole provided on the body surface of the patient, and the surgeon performs the surgery.

  Specifically, the surgeon operates the surgical manipulator while performing an operation while observing the inside of the body cavity with an endoscope inserted into the body cavity. In relation to such a technique, for example, Patent Document 1 describes a surgical manipulator system that uses a magnetic sensor to control the position of a scope (endoscope) according to the movement of the surgeon's head. . Further, for example, Patent Document 2 describes an in-vivo observation device whose tip can be bent.

JP-A-8-71072 JP-A-10-309258

  In the technique described in Patent Document 1, as described above, the motion of the head is detected using the magnetic sensor. However, in this technique, for example, in the vicinity of an apparatus that generates a magnetic field such as a nuclear magnetic resonance imaging apparatus (MRI), accurate endoscope drive control may not be performed due to the disturbance of the magnetic field.

  In addition, since a metal instrument is usually used for surgery, the magnetic field in the vicinity of the head may be disturbed by such a metal instrument. For this reason, there is a case where accurate endoscope drive control cannot be performed due to such disturbance of the magnetic field.

  Furthermore, if an obstacle (magnetic shield) exists between the signal transmission unit and the signal reception unit of the magnetic sensor, magnetic disturbance may occur, or the signal reception unit may not receive the magnetic signal accurately. is there. As a result, the endoscope drive control may not be accurate.

  In addition, when a magnetic sensor is used, there is a problem that the zero point of the measured value may unintentionally move due to long-term use, that is, zero point drift may occur.

  That is, in the conventional technology, the magnetic field is disturbed depending on the work environment such as the presence of the magnetic field, the shielding object, and the metal object, and the endoscope drive control may not be performed with high accuracy. In addition, when drive control is performed in such an environment, many calibrations must be performed, and surgery may be complicated.

  Moreover, in the technique described in Patent Document 2, since the driving direction of the distal end portion of the endoscope inserted into the body cavity is limited, a desired position in the body cavity may not be observed.

  The present invention has been made in view of the above problems, and its purpose is to be able to drive and control the imaging means to a desired position with high accuracy according to the movement of the operator regardless of the working environment. An object is to provide a surgical imaging system and a surgical robot.

  As a result of intensive studies to solve the above problems, the present inventors have found that the above problems can be solved by detecting the movement of the surgeon with a marker, and have completed the present invention.

  That is, the gist of the present invention is a surgical imaging system including an imaging unit that images a body cavity and an image display unit that displays an image captured by the imaging unit, and a driving position of the imaging unit Imaging position control means for controlling the position of the marker and marker detection means for detecting the position of the marker attached to the surgeon, and the imaging means based on the amount and direction of movement of the marker detected by the marker detection means A driving amount and a driving direction are determined, and the imaging position control unit controls a driving position of the imaging unit based on the determined driving amount and driving direction. Claim 1).

  According to such a surgical imaging system, it is possible to accurately perform drive control of the imaging means (for example, an endoscope) in the body cavity without being affected by a magnetic field, a shield, or the like.

  At this time, the marker detection means is an optical system detection means. According to such a configuration, a surgeon's movement can be detected more easily.

  At this time, the optical system detection means is a stereoscopic camera that detects a three-dimensional position of the marker. According to such a configuration, the three-dimensional movement of the operator can be reliably detected.

  Further, at this time, a plurality of the marker detection means are provided. According to such a configuration, it is possible to control the driving of the imaging unit more reliably and accurately.

  Further, the surgical imaging system according to the present embodiment includes an imaging switching trigger for controlling on and off of a driving mode for driving the imaging means, and the imaging position control means by the imaging position control means in a state where the driving mode is on. The driving of the imaging unit is stopped by the imaging position control unit when the imaging unit is driven and the drive mode is off (Claim 5).

  According to such a configuration, when the drive mode is turned off, the imaging unit is not driven, and the image displayed on the image display unit becomes a still image. Therefore, even if the operator moves the marker unintentionally, for example, by operating a manipulator, the displayed image can remain as a still image, and a stable operation can be performed.

  Further, the image display means is a head mounted display attached to the surgeon's head (claim 6). According to such a configuration, the inside of the body cavity can be observed in a state closer to the naked eye.

  Further, a plurality of the markers are provided (Claim 7). According to such a configuration, the operation of the operator can be detected more reliably.

  The imaging means is configured such that the imaging means is integrated with a surgical manipulator (claim 8). According to such a configuration, since the imaging means and the surgical manipulator can be configured integrally, the apparatus can be simplified.

  Another aspect of the present invention relates to a surgical robot to which the surgical imaging system is applied. According to the surgical robot according to the present embodiment, surgery using an imaging means (for example, an endoscope) or the like can be performed more easily than in the past.

  According to the present invention, it is possible to provide a surgical imaging system and a surgical robot capable of driving and controlling the imaging means to a desired position with high accuracy in accordance with the movement of the surgeon in any work environment. .

1 is a diagram illustrating an overall configuration of a surgical imaging system according to the present embodiment. It is a perspective view of a head mounted display provided with a marker applicable to the imaging system for operation concerning this embodiment. It is a figure which shows three operation | movement of the operator who mounted | wore the head mounted display. It is a flowchart at the time of the driving | operation of the imaging system for a surgery which concerns on this embodiment. It is a graph which shows the position resolution of the horizontal direction measured by applying the imaging system for surgery concerning this embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention (this embodiment) will be described by giving specific embodiments with appropriate reference to the drawings. However, the present embodiment is not limited to the following contents, and can be arbitrarily modified and implemented without departing from the gist thereof.

  In the present specification, unless otherwise specified, “driving the endoscope” does not represent driving of the imaging function of the endoscope but represents physical movement of the endoscope itself. In FIG. 1, for convenience of illustration, the master side system 10 includes the operator 14 and the slave side system 20 includes the patient 26. Furthermore, each function, each unit, and each unit described below can be realized by known hardware (such as a Central Processing Unit (CPU)), software, or a physical control mechanism.

[1. Constitution]
As shown in FIG. 1, the surgical imaging system 1 according to the present embodiment includes a master side system 10 and a slave side system 20. The master side system 10 and the slave side system 20 are connected to each other in a wired or wireless form (details will be described later). Since the surgical imaging system 1 has such a configuration, the operation on the slave side system 20 is performed by the operation on the master side system 10. At the same time, the result (taken image or the like) obtained by the slave side system 20 is transmitted to the master side system 10.

  First, the master side system 10 will be described.

  As shown in FIG. 1, the master side system 10 includes a head mounted display (HMD) 11 (image display means), a stereoscopic camera 12 (marker detection means), and a stereoscopic camera 12 mounted on the head of the operator 14. And a foot switch SW (imaging switching trigger) connected to the master side device 13.

  The HMD 11 is attached to the head of the operator 14 as described above. As described above, by using the HMD as the image display means, the movement of the head of the operator 14 and the movement of the field of view of the operator 14 can be more reliably synchronized. That is, in the surgical imaging system 1, since there is no movement of the head accompanying the movement of the line of sight, it can be considered that the movement of the head matches the direction of the line of sight.

  An enlarged perspective view of the HMD 11 is shown in FIG. As shown in FIGS. 1 and 2, the HMD 11 includes an image display unit 11 a in which an image photographed by an endoscope 23 (imaging unit) described later is displayed in color, and a marker 11 b detected by the stereoscopic camera 12. , Are provided. And HMD11 is comprised by attaching the marker 11b with respect to arbitrary well-known head mounted displays. That is, the marker 11 b is attached to the operator 14 via the HMD 11. The HMD 11 (more specifically, the image display unit 11a) functions as an image display unit that displays an image picked up by the endoscope 23.

  The HMD 11 includes four markers 11b. In the present embodiment, markers of different colors (for example, blue, red, yellow, and green) are provided. These markers 11b preferably have colors that do not overlap with or are not similar to the background color.

  Further, the number of markers is not particularly limited, and may be only one or plural. However, from the viewpoint of more accurately grasping the operation of the surgeon 14, the number of markers 11b is preferably 3 or more.

  Further, the shape and size of the marker 11b are not particularly limited as long as the marker detection unit can detect the marker 11b. However, the shape of the marker 11b is preferably spherical. Further, the size of the marker 11b is preferably larger than the pixel size of the image.

  As shown in FIGS. 1 and 2, there are four positions of the marker 11 b provided in the HMD 11, that is, the top of the operator 14, the left and right head sides, and the frontal region. Thus, by providing a certain amount of space between the markers 11b while covering the entire head, the movement of the head of the operator 14 can be grasped more accurately. The installation location of the marker 11b is not limited to the main body of the HMD 11, and may be directly provided (fixed) on the body of the operator 14 (for example, a plurality of locations on the upper body).

  Here, the relationship between the operation of the HMD 11, that is, the operation of the operator 14 and the driving of the endoscope 23 will be described with reference to FIG. Note that “front and rear, left and right” shown in FIG. 3 represents “front and rear, right and left” as viewed from the operator 14.

  In the present embodiment, in FIG. 3A, when the operator 14 turns his head to the right, the endoscope 23 is also driven to the right, and the image displayed on the HMD 11 also moves to the right. It is supposed to be. On the other hand, when the operator 14 turns his head leftward, the endoscope 23 is also driven leftward, and the image displayed on the HMD 11 is also moved leftward. That is, the endoscope 23 is driven in the direction of the line of sight of the operator 13. These rotation amounts correspond to the drive amount of the endoscope 23.

  In addition to the case shown in FIG. 3A, as shown in FIG. 3B, when the position of the entire body of the operator 14 is moved in the right direction, the endoscope 23 is moved in the right direction. The endoscope 23 is driven in the left direction even when the whole is moved in the left direction. In this case, the movement amount of the entire body corresponds to the driving amount of the endoscope 23. Note that the endoscope 23 in this embodiment is driven as described above because the distal end portion thereof does not have a bending mechanism. Therefore, when an endoscope having a bending mechanism is used, the left-right translation of the endoscope is performed without changing the direction of the endoscope in accordance with the left-right translation of the operator as shown in FIG. Can be done.

  Furthermore, as shown in FIG. 3C, when the head is tilted forward, the endoscope 23 is driven downward, and the image displayed on the HMD 11 also moves downward. On the other hand, when the head is tilted backward, the endoscope 23 is driven upward and the image displayed on the HMD 11 also moves upward.

  As shown in FIGS. 3A to 3C, the driving amount and the driving direction of the endoscope 23 can be controlled based on the rotation (movement) of only the head and the movement of the entire body. It can be done.

  Although not shown, when the operator 14 moves the entire body in the forward direction, the endoscope 23 moves in the forward direction, and the size of the image displayed on the HMD 11 increases (that is, The image is magnified). On the other hand, when the operator 14 moves the entire body in the backward direction, the endoscope 23 moves in the backward direction, and the size of the image displayed on the HMD 11 is reduced (that is, the image is reduced).

  As described above, since the driving of the endoscope 23 is controlled based on an intuitive method, it is possible to reduce the complexity of the operation of the endoscope 23 under extremely tense situations such as during surgery. it can.

  The description will be given again with reference to FIG. The stereoscopic camera 12 (marker detection means) detects the position of the marker 11b attached to the operator 14. In the present embodiment, the stereoscopic camera 12 detects the marker 11b provided in the HMD 11 worn by the operator 14. And the operation | movement of the operator 14 can be grasped | ascertained when the three-dimensional camera 12 detects the three-dimensional position of the marker 11b.

  In the present embodiment, a plurality of stereoscopic cameras 12 are installed, although not shown, from the viewpoint of fail safe. Thus, by installing a plurality of stereoscopic cameras 12, it is possible to grasp the operation of the surgeon 14 more accurately. Further, for example, even when an unintended shielding object is generated between one stereoscopic camera 12 and the operator 14, the movement of the operator 14 can be tracked by the other stereoscopic camera 12.

  The specific configuration of the stereoscopic camera 12 is not particularly limited, and any stereoscopic camera can be applied. Further, as the marker detection means, a stereoscopic camera is used in the present embodiment, but other optical system detection means (for example, a plurality of two-dimensional cameras) or another detection means may be used. Even if the marker detection unit is configured in this manner, the same effect as that obtained when a stereoscopic camera is used can be obtained.

  The master side device 13 is connected to the stereoscopic camera 12. Specifically, the master side device 13 includes an image analysis unit 13a, a signal transmission unit 13b, a signal reception unit 13c, a signal conversion unit 13d, and a calculation unit (not shown). And the image analysis part 13a of the master side apparatus 13 is connected to the stereo camera 12 as mentioned above. Further, the signal conversion unit 13 d of the master side device 13 is connected to the HMD 11.

  The image analysis unit 13 a measures the amount of movement of the marker 11 b detected by the stereoscopic camera 12. Based on the measured movement amount, the calculation unit calculates the drive amount of the endoscope 23.

  As a specific method for determining the movement amount of the marker 11b, the movement amount of the marker 11b moved at the time is measured by tracking the position of the marker 11b in the image taken by the stereoscopic camera 12 for a predetermined time. It has become. More specifically, for example, when the marker 11b that was at the position of coordinates (X1, Y1, Z1) at time t0 has moved to the position of coordinates (X2, Y2, Z2) at time t0 + Δt, coordinates (X1, The distance from Y1, Z1) to coordinates (X2, Y2, Z2) is the amount of movement. The direction from the coordinates (X1, Y1, Z1) to the coordinates (X2, Y2, Z2) is the moving direction.

  Such movement amount measurement is performed for each marker 11b, and the image analysis unit 13a determines the movement amount of each marker 11b. Then, based on the determined moving amount and moving direction, the calculation unit determines the driving amount and driving direction of the endoscope 23. And the filter process mentioned later is also performed by the calculating part.

  In the present embodiment, the movement amount of the marker 11b and the driving amount of the endoscope 23 are set to be the same amount. However, these relationships do not necessarily have to be set to the same amount, and from the viewpoint of avoiding rapid driving of the endoscope 23, the endoscope 23 is driven once for the amount of movement 5 of the marker 11b. It may be set.

  The signal transmission unit 13b transmits the driving amount and driving direction of the endoscope 23 determined by the calculation unit as a signal to a signal reception unit 21a provided in the slave system 20 described later.

  The signal transmission unit 13b is connected to the foot switch SW. The foot switch SW controls on / off of driving of the endoscope 23 by the imaging position control unit 22, in other words, on / off of a driving mode for driving the endoscope 23. That is, the endoscope 23 is driven while the foot switch SW is on. On the other hand, when the foot switch SW is OFF, the endoscope 23 is not driven (stopped). By providing such a foot switch SW, it is possible to prevent the endoscope 23 from being driven in an unintended direction even when the surgeon 14 moves unintentionally during the operation.

  The foot switch SW is normally in an off state, and is switched on when the operator 14 presses it. And when the operator 14 removes a foot | leg, it will switch to an OFF state again. That is, in the present embodiment, the endoscope 23 is driven only while the operator 14 continues to press the foot switch SW. During normal surgery using an endoscope, the operator 14 operates the manipulator 24 with both hands. Therefore, by using the foot switch SW as the imaging switching trigger, it is possible to prevent the operation of the manipulator 24 of the operator 14 from being hindered.

  The signal receiving unit 13c receives a signal about a state inside the body cavity imaged by the endoscope 23 from a signal transmitting unit 21c described later. The signal converting unit 13d converts the signal received by the signal receiving unit 13c into a format to be displayed on the HMD 11. The received signal is converted in this way, and a color image (video) captured by the endoscope 23 is displayed on the HMD 11.

  Next, the slave side system 20 will be described.

  The slave-side system includes a slave-side device 21, an imaging position control unit 22, an endoscope 23, a manipulator 24 (surgical manipulator), and a support base 25.

  The slave side device 21 includes a signal reception unit 21a, an image processing unit 21b, and a signal transmission unit 21c. The signal receiving unit 21a receives a signal related to the driving amount and driving direction of the endoscope 23 from the signal transmitting unit 13b. And the signal receiving part 21a is connected to the imaging position control means 22 mentioned later.

  The image processing unit 21b converts an image photographed by the endoscope 23 into a signal that can be transmitted. The signal converted by the image processing unit 21b is transmitted to the signal transmission unit 21c. The signal transmission unit 21c transmits the signal transmitted from the image processing unit 21b to the signal reception unit 13c. In this way, a signal from the slave side device 21 is transmitted to the master side device 13.

  The imaging position control means 22 is connected to the signal receiving unit 21a and controls the driving amount and driving direction of the endoscope 23. That is, the imaging position control means 22 controls the drive position of the endoscope 23 based on the drive amount and the drive direction calculated by the above-described calculation unit.

  The imaging position control means 22 in this embodiment is configured to control the drive amount and the drive direction by a piston-type device using air pressure. The imaging position control means 22 can drive the endoscope 23 in three directions (left-right direction, up-down direction, and front-rear direction with the insertion port as a rotation center) following the marker 11 b attached to the operator 14. It is possible. That is, the imaging position control means 22 is configured so that the endoscope 23 can capture an arbitrary position in a three-dimensional space within a conventional endoscope visual field.

  The manipulator 24 is called a so-called forceps. In the present embodiment, two manipulators 24 are provided, and are inserted into a body cavity from an insertion hole 26 a provided on the body surface of the patient 26. The manipulator 24 is connected to remote control means attached to the hand of the operator 14 by an electric signal line (not shown). For this reason, the manipulator 24 can follow the movement of the hand of the operator 14 and drive the distal end portion thereof.

  The manipulator 24 in the present embodiment is configured to be driven by air pressure. That is, the manipulator 24 according to the present embodiment includes a pneumatic swing actuator and a pneumatic cylinder (both not shown). Since the manipulator 24 has such a configuration, the manipulator 24 can be translated, moved up and down, and moved back and forth.

  The support base 25 integrally supports the endoscope 23, the manipulator 24, and the like, and is installed so as to surround the patient 26. However, the endoscope 23 is supported by the support base 25 so that it can be driven in any direction. Further, the support base 25 is made of a lightweight and highly rigid metal such as an aluminum alloy.

  According to the surgical imaging system 1 having the above configuration, the movement of the head of the operator 14 can be accurately grasped by the marker 11b attached to the HMD 11. That is, the line of sight of the operator 14 can be detected by detecting the movement of the head of the operator 14. Therefore, by controlling the direction of the endoscope 23 based on the line of sight of the surgeon 14, it is possible to control the part in the body cavity imaged by the endoscope 23. And the said site | part imaged with the endoscope 23 can be displayed on HMD11. As a result, the operator 14 is released from the operation of the endoscope 23 during the operation, and can easily observe the state in the body cavity by driving the endoscope 23 to a desired position.

[2. Control method]
Next, drive control of the endoscope 23 in the surgical imaging system 1 will be described with reference to FIGS. 4 and 5.

  When the surgical imaging system 1 is activated, the surgeon 14 wearing the HMD 11 photographed by the stereoscopic camera 12 is photographed by the stereoscopic camera 12. Since the stereoscopic camera 12 is composed of two two-dimensional cameras (that is, a left-eye camera and a right-eye camera) as shown in FIG. 1, an operator 14 photographed by each two-dimensional camera can monitor a monitor (not shown). To be displayed together. At this time, electric power is also supplied to the endoscope 23, and an image currently captured by the endoscope 23 is displayed on the HMD 11.

  On the monitor, the operator 14 wearing the marker 11b as described above is displayed. Then, the marker 11b to be tracked among the markers 11b displayed on the monitor is designated on the monitor screen (step S101). At this time, the number of markers 11b to be tracked can be arbitrarily changed, but three or more markers 11b are designated from the viewpoint of more accurate driving. Also, two screens photographed by two two-dimensional cameras are displayed on the monitor, and the same corresponding marker 11b is designated on each screen.

  The image analysis unit 13a tracks the marker 11b specified in step S101 using a color information histogram (steps S102 and S103). This tracking is performed for both images taken by the left eye camera and the right eye camera. As a specific tracking method, a histogram of color information is used in the present embodiment.

  At the time of this tracking, for example, the operator 14 may be unable to track the marker 11b due to a frame out of the stereoscopic camera 12 or a shield appearing between the stereoscopic camera 12 and the operator 14 (in step S104). Yes direction). In such a case, steps S102 and S103 are repeated until the designated marker 11b can be traced again.

  When the image analysis unit 13a determines that all the designated markers 11b can be traced (No direction in step S104), the image analysis unit 13a displays the three-dimensional position of the designated marker 11b on the left-eye screen and the right-eye screen ( That is, calculation is performed based on the position of the left and right screens (step S105). Specifically, the position and rotation center of the operator 14, the posture of the operator 14, and the like are calculated based on the trajectory of the marker 11 b as the operator 14 moves the head up and down. (Step S106).

  As described above, the initial setting is completed and a standby state is entered. Next, control during operation of the surgical imaging system 1 after step S106 is completed will be described.

  When the operator 14 shifts or rotates the head position in the standby state, the image analysis unit 13a determines the translation amount and the rotation amount of the head based on the translation amount and the rotation amount of the designated marker 11b. Calculate (step S107). A calculation unit (not shown) calculates the driving amount and driving direction of the endoscope 23 based on the calculated translation amount and rotation amount (that is, the moving amount and the moving direction).

  Then, a filtering process is performed on the driving amount and the driving direction (that is, the operation command value transmitted to the imaging position control means 22) calculated by the calculation unit (step S108). Specifically, two processes are performed on the operation command value: application of a low-pass filter that cuts a high-frequency component and cut of a minute operation command value that is equal to or less than a predetermined threshold. By performing such processing on the operation command value, the driving of the endoscope 23 becomes smooth.

  When the foot switch SW is pressed (turned on, Yes in step S109), the signal transmission unit 13b makes an endoscope 23 to the signal reception unit 21a of the slave side system 20. The driving amount and driving direction (operation command value) are transmitted (step S110). Then, the endoscope 23 is driven based on the driving amount and the driving direction calculated by the calculation unit. When the operator 14 removes his / her foot from the foot switch SW and the foot switch SW is not depressed (turned off, No direction in step S109), the driving of the endoscope 23 is stopped. .

  Thereafter, when the operation of the endoscope 23 becomes unnecessary because the operation by the operator 14 is completed, for example, when the system stop button is pressed (Yes direction in step S111), the surgical imaging system 1 is finish.

  In this manner, the driving of the endoscope 23 is controlled following the operation of the head of the operator 14.

[3. Application]
The surgical imaging system according to the present embodiment can be applied to any application. For example, the present invention can be applied to a surgical robot or the like that integrally holds an endoscope 23 and a manipulator 23 that constitute the slave system 20.

[4. effect]
Since the surgical imaging system according to the present embodiment does not use a magnetic sensor, there is an advantage that the surgical imaging system is hardly affected by the surrounding environment. Therefore, according to the surgical imaging system according to the present embodiment, it is possible to accurately drive and control the imaging unit at a desired position in any work environment. In particular, even in a place where a strong magnetic field such as MRI is generated or a place where a large amount of metal such as a surgical instrument is present, the endoscope can be placed at a desired position without causing a decrease in accuracy of the endoscope driving position. Can be driven.

  In particular, in the case of a system using a conventional magnetic sensor, there is a problem that the distance between the magnetic field transmission unit and the reception unit is limited in addition to the above problem. That is, there is a problem that the accuracy decreases as the distance between them increases. Specifically, it is known that the amount of magnetic field decreases in proportion to the cube of the distance, and as a result, there is a problem that these must be installed as close as possible.

  However, according to the surgical imaging system according to the present embodiment, there is no such limitation. That is, even if the distance is long, for example, if the telephoto lens is attached to the stereoscopic camera 12 or the resolution of the stereoscopic camera 12 is increased, the position of the marker 11b can be accurately detected. As a result, even when the distance is long, the endoscope 23 can be driven with high accuracy.

  Furthermore, unlike the case of using a magnetic sensor, the number of markers 11b can be arbitrarily increased in the surgical imaging system according to the present embodiment. And the detection accuracy can be made higher as the number of markers 11b increases. Furthermore, even if the marker 11b is arbitrarily increased, the hardware change is small and the detection accuracy can be increased very easily. Moreover, since the marker 11b is usually inexpensive, the detection accuracy can be increased at a low cost.

  Further, according to the surgical imaging system according to the present embodiment, even if there is a shielding object between the stereoscopic camera 12 and the marker 11b, the presence of the shielding object can be corrected on the software. Therefore, even if such a shield is present, the endoscope 23 can be driven with the same operation and accuracy as when there is no such shield. Further, by adding a commercially available camera as necessary, the endoscope 23 can be driven with the same accuracy.

  Furthermore, conventionally, at least two people, an operator who performs an operation on a patient, and an assistant who performs an endoscope operation have been engaged during the operation. Therefore, the direction of the endoscope may be in a direction different from the surgeon's intention, which hinders high efficiency of the operation.

  In order to solve such a problem, it was considered that the surgeon himself operated the endoscope. Specifically, there has been a technique in which an operator presses a foot pedal to switch modes during an operation using a manipulator, and the operator manually controls the position of the endoscope. However, according to this technique, the operation of the manipulator and the driving control of the endoscope cannot be performed simultaneously and synchronously.

  However, according to the surgical imaging system according to the present embodiment, the surgeon himself can perform an endoscope operation. This eliminates the need for a conventional assistant. Further, the direction of the endoscope can be controlled as intended by the operator 14 while operating the manipulator (surgery). Therefore, the endoscope can be operated more efficiently than before, and the efficiency of the operation such as shortening of the operation time can be improved.

  Furthermore, according to the surgical imaging system according to the present embodiment, the endoscope 23 can be driven in any direction up, down, front, back, left, and right as desired by the operator 14. Therefore, since the endoscope is automatically driven according to the direction of view of the surgeon 14, an intuitive operation can be performed.

  Here, the resolution of the endoscope 23 when the surgical system according to the present embodiment is applied will be described with reference to FIG. FIG. 5 is a graph obtained by the study of the present inventors. That is, FIG. 5 shows image recognition of the position of the marker 11b when the operator 14 wearing the marker 11b moves the head sideways with respect to the stereoscopic camera 12 (the operation shown in FIG. 3B). It is the experimental result acquired by.

  In the experiment shown in FIG. 5, as the stereoscopic camera 12, the comprehensive angle in the horizontal direction is 84.9 °, the comprehensive angle in the vertical direction is 68.9 °, the resolution is 320 pixels × 240 pixels, and the baseline length (two A three-dimensional camera having a distance of 300 mm between the cameras was used. And it is a result obtained when the operator 14 is arranged at a position 0.5 m away from the stereoscopic camera 12.

  As shown in FIG. 5, according to the surgical imaging system according to the present embodiment, it can be seen that the position resolution is about 1 mm and has high position resolution (driving accuracy). Further, according to the study by the present inventors, although not shown, a graph showing that the followability (that is, synchronization) of the endoscope 23 with respect to the position of the HMD 11 is also excellent is obtained.

[5. Example of change]
The present embodiment has been described with reference to a specific embodiment. However, the present embodiment is not limited to the above-described content, and can be implemented with any modifications without departing from the scope of the present invention. It is.

  For example, in the present embodiment, the endoscope 23 and the manipulator 24 are integrally fixed to the support base 25. However, the endoscope 23 and the manipulator 24 are integrally configured, and the endoscope configured integrally is configured. The mirror 23 and the manipulator 24 may be fixed to the support base 25. By comprising in this way, the number of the insertion holes 26a formed in the body surface of the patient 26 can be reduced. As a result, a further minimally invasive surgical operation is possible.

  Further, in this case, for example, by configuring the tip portion of the manipulator 24 with a pneumatic oscillation actuator and a pneumatic cylinder, the fail safety by the endoscope can be enhanced. That is, when an endoscope is attached to the manipulator, when the endoscope comes into contact with an organ, the contact can be detected from the differential pressure of the pneumatic actuator. In such a case, further driving of the endoscope can be stopped, and organ damage caused by the endoscope can be more reliably prevented.

  Furthermore, in the present embodiment, the signal transmission units 13b and 21c and the signal reception units 13c and 21a are provided in the master side device 13 and the slave side device 21, respectively. The signal transmitting / receiving unit of the master side device 13 and the signal transmitting / receiving unit of the slave side device 21 may communicate with each other.

  In the embodiment shown in FIG. 1, the stereoscopic camera 12 and the operator 14 are shown to be relatively close to each other. A camera may be installed so that the surgeon 14 can perform an operation at an arbitrary location in the room.

  Furthermore, the master side system 10 and the slave side system 20 do not necessarily have to be close to each other. For example, even when the respective systems are at remote positions such as remote medical care, the surgical imaging system according to this embodiment is provided. Applicable.

  In the present embodiment, the marker 11b is attached to the HMD 11. However, the marker 11b may be attached directly to the head (for example, ear) of the operator 14. Further, the mounting position of the marker 11b is not limited in any way, for example, it is mounted on the shoulder of the operator 14 or the like. Therefore, the marker 11b can be attached to any part as long as the part can be estimated by the operator 14 and the surgeon's line of sight can be estimated.

  Furthermore, in the present embodiment, the endoscope 23 is driven when the foot switch SW is pressed, but without providing such a switch, for example, the surgeon 14 quickly turns the head twice. It may be set so that the endoscope 23 is driven when it is moved, and it is set so that the endoscope 23 is not driven when it is further rotated twice. In such a case, an operation for quickly moving the head twice serves as an imaging switching trigger.

  In addition, a voice of an operator or the like may be used as an imaging switching trigger. In other words, the mode may be such that the endoscope 23 is driven when the surgeon utters “switch”, and the endoscope 23 is stopped when the surgeon utters “stop”.

  Further, when the moving speed of the marker 11b is faster (or lower) than the predetermined speed, it is considered that the movement is not intended by the operator 14, and in such a case, control is performed so that the endoscope 23 is not driven. May be.

  In the embodiment shown in FIG. 1, each unit may be connected by wire or may be connected wirelessly. In addition, wired and wireless may be connected in an appropriate combination.

DESCRIPTION OF SYMBOLS 1 Surgical imaging system 10 Master side system 11 Head mounted display (HMD, image display means)
11a Image display unit 11b Marker 12 Stereo camera (marker detection means)
13 Master side device 13a Image analysis unit 13b Signal transmission unit 13c Signal reception unit 13d Signal conversion unit 14 Surgeon 20 Slave side system 21 Slave side device 21a Signal reception unit 21b Image processing unit 21c Signal transmission unit 22 Imaging position control means 23 Inside Endoscope (imaging means)
24 Manipulator 25 Support base 26 Patient 26a Insertion hole SW Foot switch (imaging switching trigger)

Claims (9)

Imaging means for imaging the body cavity;
Image display means for displaying an image picked up by the image pickup means;
A surgical imaging system comprising:
Imaging position control means for controlling the drive position of the imaging means;
Marker detection means for detecting the position of the marker worn by the surgeon;
With
Based on the movement amount and movement direction of the marker detected by the marker detection means, the driving amount and the driving direction of the imaging means are determined,
The imaging system for surgery, wherein the imaging position control means controls the driving position of the imaging means based on the determined drive amount and drive direction. The surgical imaging system according to claim 1, wherein the marker detection unit is an optical system detection unit. The surgical imaging system according to claim 2, wherein the optical system detection means is a stereoscopic camera that detects a three-dimensional position of the marker. The surgical imaging system according to any one of claims 1 to 3, wherein a plurality of the marker detection means are provided. An imaging switching trigger for controlling on and off of a driving mode for driving the imaging means;
In the state where the drive mode is on, the imaging unit is driven by the imaging position control unit,
5. The surgical imaging system according to claim 1, wherein driving of the imaging unit by the imaging position control unit is stopped in a state in which the driving mode is off. The surgical imaging system according to any one of claims 1 to 5, wherein the image display means is a head-mounted display attached to the surgeon's head. The surgical imaging system according to any one of claims 1 to 6, wherein a plurality of the markers are provided. The surgical imaging system according to claim 1, wherein the imaging unit is configured integrally with a surgical manipulator. A surgical robot to which the surgical imaging system according to any one of claims 1 to 8 is applied.

Images