JP5800610B2 - Medical master-slave manipulator - Google Patents

Description

  The present invention relates to a medical master-slave manipulator including a master input device and a slave arm.

In the medical field, a medical robot system (medical master-slave manipulator) including a master input device and a slave arm has been proposed. In the medical robot system described in Patent Document 1, the master input device is provided with a control switch mechanism for selecting a slave arm for controlling the operation. That is, the first slave arm is controlled by the master input device in the first mode and the second slave arm is controlled by the master input device in the second mode, and which one is controlled is switched by the control switch mechanism.
Examples of the control switch mechanism include voice commands, physically arranged switches, foot pedals, icons displayed on a display, and graphic user interface selection means.

Furthermore, the medical robot system of Patent Document 1 is provided with a control switch mechanism different from that described above. The control switch mechanism controls the operation by a master input device, displays an auxiliary image by controlling a laparoscopic ultrasound (LUS) probe, and displays the auxiliary image on the original display image. Switch to an image operation mode such as
In the image operation mode, another auxiliary image such as an ultrasonic diagnostic image acquired by an LUS probe or the like can be overlaid on an image of an endoscope or the like, and the master input device can function as a pointing device.

Special table 2009-512514 gazette

  When performing a medical procedure or the like via a master / slave manipulator, an image of a site to be treated is acquired by an endoscope or the like attached to the slave arm, and the operator performs the procedure while viewing the image displayed on the display. . At this time, the moving direction of the slave arm in the displayed image does not match the moving direction of the master arm that is actually operated by the operator among the master input devices, and it becomes difficult for the operator to perform master input intuitively. There is a problem.

  In the medical robot system disclosed in Patent Literature 1, image information is manipulated by switching a displayed image using a control switch mechanism and associating a master input device with a pointing device. Thus, switching the control method of the slave manipulator is not considered, and the above-described problem still remains.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a medical master-slave manipulator in which an operator can operate a master manipulator intuitively to operate a slave manipulator.

  The medical master slave manipulator of the present invention includes a master input unit for issuing an operation command, a slave manipulator operated by the master input unit, and a control unit that transmits an operation signal to the slave manipulator based on the operation command An image processing unit that generates a first video signal related to the video of the subject, and a second video signal obtained by subjecting the first video signal to image processing, the first video signal, and the second video signal A display unit that displays one of the video signals, and a switching unit that switches the video signal displayed on the display unit, and when the switching unit switches the video signal displayed on the display unit, The control unit performs a conversion process associated with the video signal displayed on the display unit in response to the operation command.

  The master input unit includes an operation unit, the slave manipulator includes slave arms, the number of which is greater than the number of the operation units, a conversion process associated with the first video signal, and the first In the conversion process associated with the second video signal, the correspondence between the operation unit and the slave arm may be different from other conversion processes.

The medical master-slave manipulator of the present invention further includes an observation unit that has a drive unit and acquires the first video signal, and performs the predetermined update input after driving the drive unit, It is capable update the contents of the conversion processing associated with one of the video signal and the first video signal would look like.

  According to the medical master-slave manipulator of the present invention, the operator can operate the master input device intuitively to operate the slave manipulator.

It is a figure which shows the medical manipulator system to which the medical master slave manipulator of 1st embodiment of this invention is applied. It is a functional block diagram of the medical master slave manipulator. It is a flowchart which shows the flow of operation | movement of the medical master slave manipulator. (A) And (b) is a figure which shows the master input part in the medical master slave manipulator of 2nd embodiment of this invention. It is a flowchart which shows the flow of operation | movement of the medical master slave manipulator. It is a functional block diagram of the medical master slave manipulator of 3rd embodiment of this invention. It is a flowchart which shows the flow of operation | movement at the time of updating an initial image in the medical master slave manipulator.

  Hereinafter, a first embodiment of the present invention will be described. FIG. 1 is a diagram showing a medical manipulator system to which a medical master-slave manipulator (hereinafter simply referred to as “master-slave manipulator”) 1 of the present embodiment is applied. The master-slave manipulator 1 includes a master arm 21, and includes a master input unit 2 for issuing an operation command and a slave manipulator 3 having a slave arm 31, and an operation of the master arm 21 by an operator (operator) Op. The slave arm 31 is remotely controlled so as to follow. An operation command via the master arm 21 is transmitted to the master control unit 41 of the control unit 4, and after being subjected to conversion processing described later as necessary, is input to the manipulator control unit 42. Thereafter, an operation signal is sent from the manipulator control unit 42 to the slave manipulator 3, and the slave arm 31 operates.

  As shown in FIG. 1, the slave manipulator 3 is installed on an operating table 100 on which a patient P is placed and has a plurality of slave arms 31. Each slave arm has a plurality of multi-degree-of-freedom joints, and can perform multi-axis operation. Each multi-degree-of-freedom joint is individually driven by a power unit (not shown). As the power unit, for example, a motor (servo motor) having a servo mechanism including an incremental encoder, a speed reducer, or the like can be used.

  One of the plurality of slave arms 31 is provided with an observation means (not shown) such as an endoscope for acquiring an image of the operative field related to the procedure target region (subject), and the other slave arms perform various treatments. A treatment tool (not shown) is attached. Known observation means and treatment tools can be appropriately selected and used. Each slave arm also has a plurality of power units (not shown) for driving the attached treatment instrument and the like. For example, a servo motor can be used as the power unit. In the following description, a slave arm to which a treatment tool is attached among the slave arms may be referred to as a “treatment slave arm”.

  The master input unit 2 includes a plurality of master arms 21 operated by the surgeon Op, and a display unit 22 on which an image acquired by the above observation means is displayed. Each master arm 21 has a known configuration capable of multi-axis operation, and is provided with a grasping portion (operation portion) 21A for grasping by the operator on the distal end side close to the operator Op.

FIG. 2 is a functional block diagram of the master / slave manipulator 1. As shown in FIGS. 1 and 2, the display unit 22 includes a display 23 on which an image is displayed and a switching unit 24 for switching the image displayed on the display 23.
The display 23 displays an image of the surgical field acquired by the observation means. In the present embodiment, two images having a shift corresponding to parallax are projected on the display 23, and the operator Op views the images three-dimensionally through the known 3D glasses 101 having a polarization mechanism, a shutter mechanism, and the like. Can be seen.

  The switching unit 24 causes a predetermined video to be displayed on the display 23 by selectively connecting a video signal transmitted from an image processing unit described later to the display 23. In the present embodiment, the switching unit 24 operates when the operator Op steps on the foot switch 25 shown in FIG. 1, but the input mechanism for operating the switching unit 24 is not particularly limited, and various switches and the like are used. It can be appropriately selected and used.

  An image processing unit 50 is provided between the observation unit 32 and the display unit 22. The image processing unit 50 is a known processing circuit or processing program, and the video signal transmitted from the observation unit 32 is processed into a video displayable state by the image processing unit 50 to generate a video signal. The processed video signal is displayed on the display 23 of the display unit 22.

  An input conversion switching unit 55 is provided between the foot switch 25 and the master control unit 41. Based on the information transmitted from the foot switch 25, the input conversion switching unit 55 transmits the conversion processing content to the master control unit 41 so that the slave arm 31 displayed on the display 23 can be operated intuitively by the master arm 21. To do.

The operation of the master-slave manipulator 1 configured as described above will be described.
FIG. 3 is a flowchart showing an operation flow of the master / slave manipulator 1.
First, in step S10, the surgeon Op sets a condition of an image to be displayed when the foot switch 25 is pressed. Here, as an example, a video (switching video) obtained by rotating the video of the operative field (initial video) as acquired by the observation means 90 degrees clockwise is displayed. In this case, the operator Op inputs the rotation angle θ (90 degrees clockwise) to the master input unit 2 via an interface (not shown), and sets the generation condition of the switching video.
The input switching video generation conditions are transmitted to the image processing unit 50 and the input conversion switching unit 55. In response to this, the input conversion switching unit 55 performs a conversion process for matching the movement direction of the master arm and the movement direction of the slave arm when the switching video is displayed based on the generation condition (hereinafter referred to as “switching conversion process”). Is automatically calculated and temporarily stored in a storage unit such as a ROM or a RAM.

When a video signal is sent from the observation means 32, in step S20, the image processing unit 50 performs a process for displaying the initial video on the video signal, and the initial video signal (first video signal). Is generated and displayed on the display 23 as it is. In parallel, the initial video signal is subjected to image processing according to the generation condition input in step S10, and a switching video signal (second video signal) for displaying the switching video is generated. That is, the image processing unit 50 constantly generates the initial video signal and the switching video signal, but only the initial video is initially displayed on the display 23 according to the setting of the switching unit 24.
The surgeon operates the master arm 21 while watching the video displayed on the display 23. An operation command to the master arm 21 is subjected to predetermined initial conversion by the master control unit 41 and sent to the manipulator control unit 42, and is sent to the slave manipulator 3 to operate the slave arm 31. The master control unit 41 continues the processing of the operation command by the above initial conversion until the foot switch 25 is pressed.

  When the operator Op wants to perform a procedure while watching the switching image, the operator Op presses the foot switch 25 in step S30. In step S <b> 31, the switching unit 24 changes the setting so that the switching video signal generated by the image processing unit 50 is sent to the display 23, and the switching image is displayed on the display 23.

The signal of the foot switch 25 is also transmitted to the input conversion switching unit 55. Thereby, in step S <b> 32, the input conversion switching unit 55 transmits the primarily stored switching conversion process to the master control unit 41.
In step S40, the master control unit 41 that has received information from the input conversion switching unit 55 determines the position and posture of the gripping unit 21A of the master input unit 2 and the distal end portion of the treatment slave arm 31 displayed in the switching video. Initialize to match the position and orientation. The master control unit 41 calculates a difference between the position and posture of the gripping unit 21A and the position and posture of the distal end portion of the treatment slave arm 31 in the switching video, and the master input unit 2 sets the difference so that the difference becomes zero. The drive unit is operated to initialize the gripping unit 21A. At this time, a message such as “Initializing. Please release your hand from the gripping part” may be displayed on the display unit 22 to alert the surgeon.
The “position” of the master arm and the slave arm is a three-dimensional position represented by an XYZ coordinate system of a predetermined part of the master arm and the slave arm (for example, the gripping part 21A and the distal end of the treatment instrument). “Position” means the direction of the tip with the predetermined portion as a base point. Position / posture alignment in the master-slave manipulator is a known technique, and the specific method is not limited to the above.

In subsequent step S41, the operation command conversion processing content in the master control unit 41 is updated to the switching conversion processing received from the input conversion switching unit 55. Thereby, the master control unit 41 converts the operation command so that the operation direction of the grasping unit 21A matches the operation direction of the distal end portion of the treatment slave arm 31 in the switching image selected by the operator Op. After updating the conversion processing content, in step S42, the master control unit 41 performs a conversion process associated with the screen selected by the operator Op in response to the operation command, transmits an operation signal to the manipulator control unit 42, and performs the processing. finish.
When the operator Op presses the foot switch 25 once again, the video on the display 23 is switched to the initial video, and after the master arm 21 is initialized, the conversion process of the master control unit 41 is updated to the initial conversion.

  According to the master-slave manipulator 1 of the present embodiment, when the surgeon Op presses the foot switch 25, the video on the display 23 is switched by the switching unit 24 to a switching video with conditions preset by the surgeon Op. Further, the conversion processing content of the master control unit 41 so as to perform the switching conversion processing associated with the switching video so that the moving direction of the distal end portion of the treatment slave arm 31 in the switching video is the same as the moving direction of the gripping portion 21A. Is updated. Therefore, the operator simply adjusts the conversion processing content of the master control unit by simply switching the video displayed on the display 23 with the foot switch 25, and intuitively operates the gripping unit while viewing the desired video. The slave manipulator can be operated intuitively without stress.

  Further, since the position and posture of the gripping portion 21A are initialized at the time of switching the image, not only the operation direction of the gripping portion 21A but also the position / posture relationship is automatically aligned with the distal end portion of the treatment slave arm 31. . Therefore, the operation can be performed more intuitively.

  Furthermore, since the switching video is generated by performing image processing on the video acquired by the observation unit 32, it is not necessary for the operator Op to drive the observation unit 32 in order to perform a procedure from different viewpoints. Therefore, it is possible to eliminate a time lag caused by the viewpoint switching and smoothly perform a series of procedures. Further, since the problem of interference with other slave arms due to driving of the observation means can be eliminated, the safety of the procedure is improved.

  In the present embodiment, an example has been described in which the master control unit 41 drives the master input unit 2 to perform initialization, but instead, it may be configured such that the operator can perform initialization manually. For example, the master input unit 2 is not provided with a drive unit, and a switch for switching on / off operation command transmission from the master input unit to the master control unit is provided. Then, after the switch is pressed, the initialization may be completed by moving the grasping portion 21A to a desired position that the operator wants to set as the initial position and pressing the foot switch 25 again. In this case, the position and posture of the gripping part and the position and posture of the slave arm tip may not be completely matched, but the initial position set by the operator is respected, and the difference value of the movement amount from the initial position Since the slave arm is operated based on this, the operation can be performed intuitively with the setting that is most easily operated by the surgeon.

  Further, the image processing that is the generation condition of the switching video is not limited to the above-described rotation processing, and may be enlargement, reduction, or the like. In this case, the intuitive operation can be continued by changing the motion scale ratio between the master arm and the slave arm when performing the procedure while viewing the initial image and when performing the procedure while viewing the switching image. Therefore, the information of motion scale ratio may be included in the switching conversion process. Here, it goes without saying that a condition for generating a switching video may be set by combining a plurality of image processes.

Furthermore, two or more types of switching images may be set. In this case, for example, each time the foot switch 25 is pressed, the initial video and each switching video may be switched in order.
Further, instead of always generating the switching video, it may be generated when the foot switch 25 is pressed and displayed on the display 23.

  Note that the switching image that has been subjected to the rotation processing by the image processing maintains the 3D display that the operator can visually recognize even after switching. For example, in 3D display, two images corresponding to the parallax of both eyes are displayed on the same screen, but the two images are reconverted so as to maintain the 3D display in consideration of the binocular parallax according to the rotation amount during image processing. Display on the display device.

  A second embodiment of the present invention will be described with reference to FIGS. 4 and 5. The difference between the master-slave manipulator 61 of the present embodiment and the above-described master-slave manipulator 1 is that the correspondence relationship between the master arm and the slave arm is switched when the video on the display unit is switched. In the following description, components that are the same as those already described are assigned the same reference numerals and redundant description is omitted.

FIG. 4 is a diagram showing the master input unit 2 of the master / slave manipulator 61. As shown in the display 23, the slave manipulator 3 is provided with three treatment slave arms 62A, 62B, and 62C. When the initial image is displayed on the display 23, as shown in FIG. 4A, the slave arms 62A and 62B are in a position where the operator Op can easily operate intuitively, and the initial image is rotated clockwise by 90. When the switching image rotated at a time is displayed on the display 23, as shown in FIG. 4B, the slave arms 62B and 62C are in a position where the operator Op can easily operate intuitively.
Since the master input unit 2 has only two master arms, in this embodiment, two treatment slave arms and a master arm that are easy to operate intuitively are associated with each other in the initial video and the switching video. That is, the correspondence between the master arm and the treatment slave arm differs depending on the video displayed on the display 23 of the display unit 22.

FIG. 5 is a flowchart showing an operation flow of the master / slave manipulator 61. In step S10A, as in the first embodiment, the conversion angle θ, which is a switching video generation condition, is set, and two treatment slave arms operated by the master arm 21 in each of the initial video and the switching video are displayed. Set by the operator Op.
Thereafter, in step S40, initialization is performed so that the positions and postures of the two slave arm tips and the gripping parts set in step S10A are aligned. In subsequent step S41A, the master arm and the slave arm The correspondence and operation command conversion processing contents are updated.
Other points are substantially the same as in the first embodiment.

Also in the master slave manipulator 61 of the present embodiment, the conversion processing content of the master control unit is automatically updated simply by switching the image displayed on the display 23, so the surgeon can watch the selected video while watching the selected video. By operating the arm intuitively, the slave arm can be operated intuitively without stress.
Furthermore, since the correspondence relationship between the master arm and the slave arm is also automatically updated, the procedure can be performed by operating the optimum slave arm on each screen.

In the present embodiment, an example in which there are three treatment slave arms has been described, but there may be two treatment slave arms or four or more treatment slave arms. Further, instead of setting the generation condition of the switching image and the slave arm to be operated while watching each video in step S10A, the master control unit or the like is based on the positional relationship between the observation means and each treatment slave arm. It may be configured to automatically calculate a rotation angle at which two slave arms of different combinations can be easily operated and determine the conversion angle θ. At this time, the positional relationship between the observation unit and each treatment slave arm may be calculated from the initial image and the switching image acquired by the observation unit, or may be calculated from position information based on the value of the encoder of the slave arm. Good.
Alternatively, the optimum correspondence between the slave arm and the master arm may be automatically calculated from the conversion angle θ determined in step S10A based on the positional relationship information between the observation means and the slave arm. Also in this case, as described above, the positional relationship between the observation unit and each slave arm may be calculated from the video acquired by the observation unit, or may be calculated from the position information based on the value of the encoder of the slave arm. Also good.
As another method, the correspondence relationship between the slave arm and the master arm may be re-selected by the operator after the switching video is displayed in step S31. The selection method may be performed by an input switch such as an interface.

  A third embodiment of the present invention will be described with reference to FIGS. The difference between the master-slave manipulator 71 of the present embodiment and the above-described master-slave manipulator 1 is that when the observation means is driven, a switching video is generated based on the initial video state before driving.

FIG. 6 is a functional block diagram of the master / slave manipulator 71. The overall configuration of the master / slave manipulator 71 is substantially the same as that of the master / slave manipulator 1 of the first embodiment, but the observation means 72 includes a drive unit 73, and an image acquired by driving the drive unit 73. The field of view can be rotated. The observation unit 72 also includes a rotation amount detection unit 74 formed of a rotary encoder or the like, and can detect the rotation angle of the observation unit 72 when the driving unit 73 is driven.
In the present embodiment, the signal of the foot switch 25 is transmitted to the image processing unit 50 in addition to the switching unit 24 and the input conversion switching unit 55.

  The operation of the master-slave manipulator 71 configured as described above will be described. The operation that does not involve driving the observation means 72 is generally the same as in the first embodiment. In a case where it is desired to perform a procedure using an image having a rotation angle different from both the initial image and the switching image set in step S10, the operator Op drives the drive unit 73 according to an operation command from the master arm 21 or the like, The observation means 72 is rotated to search for a video that is a suitable surgical field, and the initial video is updated.

FIG. 7 is a flowchart showing a flow of initial video update. First, in step S51, the observation means 72 is rotated, and an image suitable for an operative field is searched.
When the image of the observation means 72 is in a desired state suitable for the procedure, the surgeon Op performs a predetermined update input that combines the operation of the foot switch 25 and other input means in step S52.

In step S53, the image processing unit 50 that has received the update input updates its setting so that the video acquired by the observation unit 72 at the time of receiving the update input becomes the initial video.
Subsequently, in step S54, the input conversion switching unit 55 calculates the input conversion processing content corresponding to the updated initial video and transmits it to the master control unit 41. The master control unit 41 receives the received input conversion processing content. Update the contents of the initial conversion so that becomes the initial conversion.

Further, in the subsequent step S55, the image processing unit 50 updates the generation condition based on the information of the rotation amount detection unit 74 so that the difference between the rotation angles of the initial video before and after the update is used as the switching video generation condition. Thereafter, image processing is performed on the updated initial video signal, and a signal capable of displaying substantially the same video as the initial video before update is generated as a switching video signal.
In step S56, the input conversion switching unit 55 calculates a conversion process corresponding to the updated switching video, and temporarily stores it in the storage unit as the switching conversion process. In this example, since the initial image before the update becomes the switched video after the update, the contents of the initial conversion are temporarily stored as the switching conversion process. After that, in step S57, the master arm 21 is initialized so as to correspond to the updated initial video, and a series of processing ends.
In addition, there is no restriction | limiting in particular in the order of the process of step S53 to S56, You may change suitably. Also, these steps may be performed after step S57.

Other points are substantially the same as in the first embodiment. That is, when the operator Op presses the foot switch 25, a switching video having the same angle as the initial video before the update, which is generated by performing image processing on the updated initial video, is displayed on the display. The corresponding switching conversion process is transmitted to the master control unit 41 and applied.
When the foot switch 25 is pressed without performing a predetermined operation after driving the observation means 72, the observation means 72 stops there, and an initial video generated by image processing of the video acquired by the observation means 72 The setting of the initial video is updated so that the video with the same angle as the initial video becomes the initial video.

  In the master-slave manipulator 71 of this embodiment, the drive unit 73 is driven, and a predetermined operation is performed when a desired video is obtained, so that the most frequently used initial video in the procedure is appropriately updated to the desired video. Can do.

  In addition, since the image processing unit 50 and the input conversion switching unit 55 are automatically set so that the initial video before the update becomes the switching video, it is substantially the same as the initial video before the update only by operating the foot switch 25. It is possible to easily return to the same image.

  In the present embodiment, instead of rotating the observation unit, a predetermined operation may be performed after the enlargement / reduction operation of driving the observation optical system. For example, if a predetermined operation is performed when the initial video is multiplied by n, the n-fold video is updated to the initial video, and the updated initial video obtained by image processing becomes 1 / n times as the switching video. In this way, a plurality of suitable magnifications can be set while driving the observation optical system, and the procedure can be performed while suitably switching the images of the plurality of magnifications.

  In the present embodiment, an example in which the switching video is one has been described. However, a plurality of switching videos may be set by performing the above-described predetermined operation a plurality of times. In this way, it is possible to perform a procedure with an image in a state prior to the most recent state before driving, and it is possible to perform a procedure more appropriately by using a plurality of images.

  The embodiments of the present invention have been described above. However, the technical scope of the present invention is not limited to the above-described embodiments, and various modifications may be made to each component without departing from the spirit of the present invention. It is possible to delete or combine the configurations of the embodiments.

  First, in the present invention, the configuration of the display of the display unit can be variously changed. For example, a head mounted display may be used, and a display having two or more screens or display areas may be used. Especially in the latter case, when a plurality of switching videos are set, a part of them can be displayed in advance on the display as the next candidate (and the nth candidate), so that the switching of images can be performed more smoothly. Can do. If two or more screens or display areas are adjacent to each other, it is more preferable because the next candidate switching video can be roughly viewed in the peripheral visual field even during the procedure using the initial video.

In addition, by setting a large number of switching images, it is possible to create an environment in which the observation means is driven in a pseudo manner. For example, if 11 switching images are set for every 30 degrees of rotation angle, and this is sequentially displayed by an input mechanism such as a rotary knob, image adjustment close to the state in which the observation means is rotationally driven in real time can be performed. Can do. Since the switching video is generated by image processing, the load on the entire master-slave manipulator does not increase so much even if the number of switching images increases within a certain range. Further, the present invention has an advantage that the angle of the image can be adjusted without receiving the restriction even when the observation means cannot be rotated 360 degrees due to mechanical restrictions.
In each of the above-described embodiments, the example in which the gripping unit as the operation unit is connected to the master arm has been described. However, the operation unit may be configured without the master arm. For example, the configuration may be such that the position / posture of the hand of the operation unit or the operator is acquired by a motion sensor such as a camera or a position sensor.

1, 61, 71 Master-slave manipulator 2 Master input unit 3 Slave manipulator 4 Control unit 21 Master arm 21A Grasping unit (operation unit)
22 display unit 24 switching unit 31, 62A, 62B, 62C slave arm 50 image processing unit 72 observation means 73 drive unit

Claims (3)

A master input unit for issuing an operation command;
A slave manipulator operated by the master input unit;
A control unit for transmitting an operation signal to the slave manipulator based on the operation command;
An image processing unit that generates a first video signal related to a video of a subject and a second video signal obtained by performing image processing on the first video signal;
A display unit for displaying one of the first video signal and the second video signal;
A switching unit that switches a video signal displayed on the display unit;
With
When the switching unit switches the video signal displayed on the display unit, the control unit performs a conversion process associated with the video signal displayed on the display unit in response to the operation command. A master slave manipulator for medical use. The master input unit has an operation unit,
The slave manipulator has slave arms, the number of which is greater than the number of the operation units, a conversion process associated with the first video signal, and a conversion process associated with the second video signal. The medical master-slave manipulator according to claim 1, wherein the correspondence between the operation unit and the slave arm is different. A driving unit, further comprising observation means for acquiring the first video signal;
The content of the conversion process associated with the first video signal and the first video signal can be updated by performing predetermined update input after driving the driving unit. surgical master slave manipulators over data according to 1.

Images