JP7115493B2 - Surgical arm system and surgical arm control system - Google Patents

Description

The present disclosure relates to strabismus control devices and medical systems.

Conventionally, for example, in Patent Document 1 below, it is possible to display an endoscopic image on a monitor in which the up, down, left, and right directions of the endoscopic image match the operator's up, down, left, and right operation directions so as not to cause discomfort during work. A technique that assumes this is described.

JP 2006-218027 A

In recent years, strabismus scopes have been used as rigid scopes to be inserted into the human body. However, since oblique rotation by the oblique scope rotates around the scope axis, there is a problem that the field of view after rotation is less intuitive than simple horizontal or vertical movement. As a result, oblique rotation causes problems such as the object to be observed moving out of the field of view of the screen, or losing sight of the instrument in the field of view.

Therefore, it has been demanded not to lose sight of the observed object from the field of view of the monitor image.

According to the present disclosure, a multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end; a control system for controlling an arm, wherein the control system controls the speed of rotation or the speed of movement of the strabismus within a field of view imaged through the strabismus based on the position of the observed object within the field of view; A surgical arm system is provided that controls at least one of the

Further, according to the present disclosure, a multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end is controlled to change the position and posture of the squintoscope. a surgical arm control system for controlling the articulated arm in a field of view imaged through the squintoscope, based on the position of the observed object within the field of view, the speed of rotation or the speed of movement of the squintoscope A surgical arm control system is provided for controlling at least one of the

According to the present disclosure, it is possible to prevent the observation target from being lost from the field of view of the monitor image.
In addition, the above effects are not necessarily limited, and in addition to the above effects or instead of the above effects, any of the effects shown in this specification, or other effects that can be grasped from this specification may be played.

1 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which a medical support arm device according to the present disclosure can be applied; FIG. 2 is a block diagram showing an example of functional configurations of a camera head and a CCU shown in FIG. 1; FIG. 1 is a perspective view showing a configuration example of a medical support arm device according to an embodiment of the present disclosure; FIG. It is a block diagram which shows the structural example of a medical support arm apparatus. 1 is a schematic diagram showing the appearance of a perspective mirror 100. FIG. FIG. 10 is a schematic diagram showing how the field of view 200 changes due to oblique rotation. FIG. 10 is a schematic diagram showing how a field of view displayed on a monitor changes due to oblique rotation; FIG. 4 is a schematic diagram showing a method of calculating the distance from the center of the field of view 200 to the observed object 210; FIG. 10 is a schematic diagram showing an example of a map that defines the oblique rotation speed according to the subject position within the screen; FIG. 10 is a characteristic diagram showing the relationship between the distance (horizontal axis) from the screen center O to the observed object 210 and the oblique rotation speed (vertical axis). 4 is a schematic diagram showing how the direction of the optical axis of the objective optical system changes due to oblique rotation of the oblique mirror 100. FIG. FIG. 10 is a schematic diagram showing a case where movement in the XY directions is performed by giving rotation of the optical axis centered on the trocar point or a movement close thereto. It is a schematic diagram showing an example of movement in the Y direction, and showing a case where oblique rotation is performed along with the movement in the Y direction. FIG. 4 is a schematic diagram showing an example of rotation about a constraint point 300; FIG. 4 is a diagram for explaining control of an arm that moves an observation target to the center of the screen; FIG. 4 is a diagram for explaining control of an arm that moves an observation target to the center of the screen; FIG. 10 is a diagram for explaining an overview of control processing for moving an observation target to the center of the screen; 4 is a flowchart showing an example of the flow of target calculation processing; FIG. 4 is a diagram for explaining target position calculation; FIG. 10 is a schematic diagram for explaining a specific example when the squint is turned to the left by controlling the squint mirror according to the present embodiment; FIG. 10 is a schematic diagram for explaining a specific example when the squint is turned to the left by controlling the squint mirror according to the present embodiment; FIG. 10 is a schematic diagram for explaining a specific example when the squint is turned to the left by controlling the squint mirror according to the present embodiment; FIG. 10 is a schematic diagram for explaining a specific example when the squint is turned to the left by controlling the squint mirror according to the present embodiment; FIG. 4 is a schematic diagram showing the configuration of a control unit of a support arm device for performing oblique rotation and follow-up operations; FIG. 4 is a schematic diagram showing a configuration in which a holding unit for independently controlling rotation of the oblique mirror and rotation of the camera head is provided at the tip of the arm in the configuration example shown in FIG. 3 ; FIG. 23 is a schematic diagram showing configurations of a support arm device including an arm section and a control device, an endoscope unit, and a CCU in the configuration example shown in FIG. 22; FIG. 24 is a sequence diagram showing the flow of processing in the configurations shown in FIGS. 22 and 23; FIG.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description.

In addition, description shall be performed in the following order.
1. Basic configuration 1.1. Configuration example of endoscope system 1.2. Concrete Configuration Example of Medical Support Arm Device 1.3. Configuration example of control device 2 . Oblique Rotation Operation and Observation Observation Observation Operation by Observation Mirror 2.1. Rotational movement of oblique scope 2.2. Change in oblique rotation speed according to the distance from the center of the visual field 2.3. Follow-up operation to observed object 2.4. Configuration example of control unit for oblique rotation motion and follow-up motion 2.5. Details of tracking operation 2.6. Concrete example of follow-up operation3. 4. An example in which a holding unit is provided to independently control the rotation of the squint scope; summary

<<1. Basic configuration>>
First, the basic configuration of an endoscope system according to an embodiment of the present disclosure will be described with reference to FIGS. 1 to 4. FIG.

<1.1. Configuration example of endoscope system>
FIG. 1 is a diagram showing an example of a schematic configuration of an endoscopic surgery system 5000 to which technology according to the present disclosure can be applied. FIG. 1 shows an operator (doctor) 5067 performing surgery on a patient 5071 on a patient bed 5069 using an endoscopic surgery system 5000 . As illustrated, the endoscopic surgery system 5000 includes an endoscope 5001, other surgical tools 5017, a support arm device 5027 for supporting the endoscope 5001, and various devices for endoscopic surgery. and a cart 5037 on which is mounted.

In endoscopic surgery, instead of cutting the abdominal wall and laparotomy, tubular piercing instruments called trocars 5025a to 5025d are punctured into the abdominal wall multiple times. Then, the barrel 5003 of the endoscope 5001 and other surgical instruments 5017 are inserted into the body cavity of the patient 5071 from the trocars 5025a to 5025d. In the illustrated example, a pneumoperitoneum tube 5019 , an energy treatment instrument 5021 and forceps 5023 are inserted into the patient's 5071 body cavity as other surgical instruments 5017 . Also, the energy treatment tool 5021 is a treatment tool that performs tissue incision and ablation, blood vessel sealing, or the like, using high-frequency current or ultrasonic vibration. However, the illustrated surgical tool 5017 is merely an example, and various surgical tools generally used in endoscopic surgery, such as a forceps and a retractor, may be used as the surgical tool 5017 .

A display device 5041 displays an image of a surgical site in a body cavity of a patient 5071 photographed by the endoscope 5001 . The operator 5067 uses the energy treatment tool 5021 and the forceps 5023 to perform treatment such as excision of the affected area while viewing the image of the operated area displayed on the display device 5041 in real time. Although not shown, the pneumoperitoneum tube 5019, the energy treatment instrument 5021, and the forceps 5023 are supported by the operator 5067, an assistant, or the like during surgery.

(support arm device)
The support arm device 5027 has an arm portion 5031 extending from the base portion 5029 . In the illustrated example, the arm portion 5031 is composed of joint portions 5033a, 5033b, 5033c and links 5035a, 5035b, and is driven under the control of the arm control device 5045. The arm 5031 supports the endoscope 5001 and controls its position and orientation. As a result, stable position fixation of the endoscope 5001 can be achieved.

(Endoscope)
The endoscope 5001 is composed of a lens barrel 5003 having a predetermined length from its distal end inserted into a body cavity of a patient 5071 and a camera head 5005 connected to the proximal end of the lens barrel 5003 . In the illustrated example, an endoscope 5001 configured as a so-called rigid scope having a rigid barrel 5003 is illustrated, but the endoscope 5001 is configured as a so-called flexible scope having a flexible barrel 5003. good too.

The tip of the lens barrel 5003 is provided with an opening into which an objective lens is fitted. A light source device 5043 is connected to the endoscope 5001, and light generated by the light source device 5043 is guided to the tip of the lens barrel 5003 by a light guide extending inside the lens barrel 5003, and reaches the objective. The light is irradiated through the lens toward an observation target within the body cavity of the patient 5071 . Note that the endoscope 5001 is assumed to be a perspective scope.

An optical system and an imaging element are provided inside the camera head 5005, and reflected light (observation light) from an observation target is converged on the imaging element by the optical system. The imaging device photoelectrically converts the observation light to generate an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. The image signal is transmitted to a camera control unit (CCU: Camera Control Unit) 5039 as RAW data. The camera head 5005 has a function of adjusting the magnification and focal length by appropriately driving the optical system.

Note that the camera head 5005 may be provided with a plurality of imaging elements, for example, in order to support stereoscopic viewing (3D display). In this case, a plurality of relay optical systems are provided inside the lens barrel 5003 to guide the observation light to each of the plurality of imaging elements.

(various devices mounted on the cart)
The CCU 5039 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and controls the operations of the endoscope 5001 and the display device 5041 in an integrated manner. Specifically, the CCU 5039 subjects the image signal received from the camera head 5005 to various image processing such as development processing (demosaicing) for displaying an image based on the image signal. The CCU 5039 provides the image signal subjected to the image processing to the display device 5041 . Also, the CCU 5039 transmits a control signal to the camera head 5005 to control its driving. The control signal may include information regarding imaging conditions such as magnification and focal length.

The display device 5041 displays an image based on an image signal subjected to image processing by the CCU 5039 under the control of the CCU 5039 . When the endoscope 5001 is compatible with high-resolution imaging such as 4K (horizontal pixel number 3840×vertical pixel number 2160) or 8K (horizontal pixel number 7680×vertical pixel number 4320), and/or 3D display , the display device 5041 may be one capable of high-resolution display and/or one capable of 3D display. In the case of 4K or 8K high-resolution imaging, using a display device 5041 with a size of 55 inches or more provides a more immersive feeling. Also, a plurality of display devices 5041 having different resolutions and sizes may be provided depending on the application.

The light source device 5043 is composed of a light source such as an LED (light emitting diode), for example, and supplies the endoscope 5001 with irradiation light for imaging the surgical site.

The arm control device 5045 is configured by a processor such as a CPU, for example, and operates according to a predetermined program to control driving of the arm portion 5031 of the support arm device 5027 according to a predetermined control method.

The input device 5047 is an input interface for the endoscopic surgery system 5000. FIG. The user can input various information and instructions to the endoscopic surgery system 5000 via the input device 5047 . For example, through the input device 5047, the user inputs various types of information regarding surgery, such as patient's physical information and information about the surgical technique. Further, for example, the user, via the input device 5047, instructs to drive the arm unit 5031, or instructs to change the imaging conditions (type of irradiation light, magnification, focal length, etc.) by the endoscope 5001. , an instruction to drive the energy treatment instrument 5021, or the like.

The type of the input device 5047 is not limited, and the input device 5047 may be various known input devices. As the input device 5047, for example, a mouse, keyboard, touch panel, switch, footswitch 5057 and/or lever can be applied. When a touch panel is used as the input device 5047 , the touch panel may be provided on the display surface of the display device 5041 .

Alternatively, the input device 5047 is a device worn by the user, such as a glasses-type wearable device or an HMD (Head Mounted Display). is done. Also, the input device 5047 includes a camera capable of detecting the movement of the user, and performs various inputs according to the user's gestures and line of sight detected from images captured by the camera. Furthermore, the input device 5047 includes a microphone capable of picking up the user's voice, and various voice inputs are performed via the microphone. As described above, the input device 5047 is configured to be capable of inputting various kinds of information in a non-contact manner, so that a user belonging to a clean area (for example, an operator 5067) can operate a device belonging to an unclean area without contact. becomes possible. In addition, since the user can operate the device without taking his/her hands off the surgical tool, the user's convenience is improved.

The treatment instrument control device 5049 controls driving of the energy treatment instrument 5021 for tissue cauterization, incision, blood vessel sealing, or the like. The pneumoperitoneum device 5051 inflates the body cavity of the patient 5071 for the purpose of securing the visual field of the endoscope 5001 and securing the operator's working space, and injects gas into the body cavity through the pneumoperitoneum tube 5019 . send in. The recorder 5053 is a device capable of recording various types of information regarding surgery. The printer 5055 is a device capable of printing various types of information regarding surgery in various formats such as text, images, and graphs.

A particularly characteristic configuration of the endoscopic surgery system 5000 will be described in more detail below.

(support arm device)
The support arm device 5027 includes a base portion 5029 as a base and an arm portion 5031 extending from the base portion 5029 . In the illustrated example, the arm portion 5031 is composed of a plurality of joint portions 5033a, 5033b, 5033c and a plurality of links 5035a, 5035b connected by the joint portion 5033b. , the configuration of the arm portion 5031 is simplified. In practice, the shape, number and arrangement of the joints 5033a to 5033c and the links 5035a and 5035b, the directions of the rotation axes of the joints 5033a to 5033c, and the like are appropriately set so that the arm 5031 has a desired degree of freedom. obtain. For example, the arm portion 5031 may preferably be configured to have 6 or more degrees of freedom. As a result, the endoscope 5001 can be freely moved within the movable range of the arm portion 5031, so that the barrel 5003 of the endoscope 5001 can be inserted into the body cavity of the patient 5071 from a desired direction. be possible.

The joints 5033a to 5033c are provided with actuators, and the joints 5033a to 5033c are configured to be rotatable around a predetermined rotation axis by driving the actuators. By controlling the driving of the actuator by the arm control device 5045, the rotation angles of the joints 5033a to 5033c are controlled, and the driving of the arm 5031 is controlled. Thereby, control of the position and attitude of the endoscope 5001 can be achieved. At this time, the arm control device 5045 can control the driving of the arm section 5031 by various known control methods such as force control or position control.

For example, when the operator 5067 appropriately performs an operation input via the input device 5047 (including the foot switch 5057), the arm control device 5045 appropriately controls the driving of the arm section 5031 in accordance with the operation input. The position and orientation of the scope 5001 may be controlled. With this control, the endoscope 5001 at the distal end of the arm section 5031 can be moved from an arbitrary position to an arbitrary position, and then fixedly supported at the position after the movement. Note that the arm portion 5031 may be operated by a so-called master-slave method. In this case, the arm portion 5031 can be remotely operated by the user via an input device 5047 installed at a location remote from the operating room.

When force control is applied, the arm control device 5045 receives an external force from the user and operates the actuators of the joints 5033a to 5033c so that the arm 5031 moves smoothly according to the external force. A so-called power assist control for driving may be performed. Accordingly, when the user moves the arm portion 5031 while directly touching the arm portion 5031, the arm portion 5031 can be moved with a relatively light force. Therefore, it becomes possible to move the endoscope 5001 more intuitively and with a simpler operation, and the user's convenience can be improved.

Here, generally, in endoscopic surgery, the endoscope 5001 is supported by a doctor called a scopist. On the other hand, the use of the support arm device 5027 makes it possible to more reliably fix the position of the endoscope 5001 without manual intervention, so that an image of the surgical site can be stably obtained. , the operation can be performed smoothly.

Note that the arm control device 5045 does not necessarily have to be provided on the cart 5037 . Also, the arm control device 5045 does not necessarily have to be one device. For example, the arm control device 5045 may be provided in each of the joints 5033a to 5033c of the arm portion 5031 of the support arm device 5027, and the arm portion 5031 is driven by a plurality of arm control devices 5045 cooperating with each other. Control may be implemented.

(light source device)
The light source device 5043 supplies irradiation light to the endoscope 5001 when imaging the surgical site. The light source device 5043 is composed of, for example, a white light source composed of an LED, a laser light source, or a combination thereof. At this time, when a white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. can be adjusted. In this case, the observation target is irradiated with laser light from each of the RGB laser light sources in a time division manner, and by controlling the drive of the imaging device of the camera head 5005 in synchronization with the irradiation timing, each of the RGB can be handled. It is also possible to pick up images by time division. According to this method, a color image can be obtained without providing a color filter in the imaging device.

Further, the driving of the light source device 5043 may be controlled so as to change the intensity of the output light every predetermined time. By controlling the drive of the imaging device of the camera head 5005 in synchronism with the timing of the change in the intensity of the light to acquire images in a time division manner and synthesizing the images, a high dynamic A range of images can be generated.

Also, the light source device 5043 may be configured to be capable of supplying light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissues is used to irradiate a narrower band of light than the irradiation light (i.e., white light) used during normal observation, thereby observing the mucosal surface layer. So-called Narrow Band Imaging is performed in which a predetermined tissue such as a blood vessel is imaged with high contrast. Alternatively, in special light observation, fluorescence observation may be performed in which an image is obtained from fluorescence generated by irradiation with excitation light. Fluorescence observation involves irradiating body tissue with excitation light and observing fluorescence from the body tissue (autofluorescence observation), or locally injecting a reagent such as indocyanine green (ICG) into the body tissue and observing the body tissue. A fluorescent image may be obtained by irradiating excitation light corresponding to the fluorescent wavelength of the reagent. The light source device 5043 can be configured to be able to supply narrowband light and/or excitation light corresponding to such special light observation.

(camera head and CCU)
Referring to FIG. 2, the functions of camera head 5005 and CCU 5039 of endoscope 5001 are described in more detail. FIG. 2 is a block diagram showing an example of functional configurations of the camera head 5005 and CCU 5039 shown in FIG.

Referring to FIG. 2, the camera head 5005 has a lens unit 5007, an imaging section 5009, a drive section 5011, a communication section 5013, and a camera head control section 5015 as its functions. The CCU 5039 also has a communication unit 5059, an image processing unit 5061, and a control unit 5063 as its functions. The camera head 5005 and CCU 5039 are connected by a transmission cable 5065 so as to be able to communicate bidirectionally.

First, the functional configuration of the camera head 5005 will be described. A lens unit 5007 is an optical system provided at a connection portion with the lens barrel 5003 . Observation light captured from the tip of the lens barrel 5003 is guided to the camera head 5005 and enters the lens unit 5007 . A lens unit 5007 is configured by combining a plurality of lenses including a zoom lens and a focus lens. The optical characteristics of the lens unit 5007 are adjusted so that the observation light is condensed on the light receiving surface of the imaging device of the imaging unit 5009 . Also, the zoom lens and the focus lens are configured so that their positions on the optical axis can be moved in order to adjust the magnification and focus of the captured image.

The image pickup unit 5009 is configured by an image pickup element and arranged after the lens unit 5007 . Observation light that has passed through the lens unit 5007 is condensed on the light receiving surface of the image sensor, and an image signal corresponding to the observation image is generated by photoelectric conversion. An image signal generated by the imaging unit 5009 is provided to the communication unit 5013 .

As an imaging device constituting the imaging unit 5009, for example, a CMOS (Complementary Metal Oxide Semiconductor) type image sensor, which has a Bayer arrangement and is capable of color imaging, is used. As the imaging element, for example, one capable of capturing a high-resolution image of 4K or higher may be used. By obtaining a high-resolution image of the surgical site, the operator 5067 can grasp the state of the surgical site in more detail, and the surgery can proceed more smoothly.

In addition, the imaging device constituting the imaging unit 5009 is configured to have a pair of imaging devices for acquiring image signals for right eye and left eye corresponding to 3D display. The 3D display enables the operator 5067 to more accurately grasp the depth of the living tissue in the surgical site. Note that when the imaging unit 5009 is configured as a multi-plate type, a plurality of systems of lens units 5007 are provided corresponding to each imaging element.

Also, the imaging unit 5009 does not necessarily have to be provided in the camera head 5005 . For example, the imaging unit 5009 may be provided inside the lens barrel 5003 immediately after the objective lens.

The drive unit 5011 is configured by an actuator, and moves the zoom lens and focus lens of the lens unit 5007 by a predetermined distance along the optical axis under control from the camera head control unit 5015 . Thereby, the magnification and focus of the image captured by the imaging unit 5009 can be appropriately adjusted.

Communication unit 5013 is configured by a communication device for transmitting and receiving various information to and from CCU 5039 . The communication unit 5013 transmits the image signal obtained from the imaging unit 5009 as RAW data to the CCU 5039 via the transmission cable 5065 . At this time, the image signal is preferably transmitted by optical communication in order to display the captured image of the surgical site with low latency. During surgery, the operator 5067 performs surgery while observing the state of the affected area using captured images. Therefore, for safer and more reliable surgery, moving images of the operated area are displayed in real time as much as possible. This is because it is required. When optical communication is performed, the communication unit 5013 is provided with a photoelectric conversion module that converts an electrical signal into an optical signal. After the image signal is converted into an optical signal by the photoelectric conversion module, it is transmitted to the CCU 5039 via the transmission cable 5065 .

The communication unit 5013 also receives a control signal for controlling driving of the camera head 5005 from the CCU 5039 . The control signal includes, for example, information to specify the frame rate of the captured image, information to specify the exposure value at the time of imaging, and/or information to specify the magnification and focus of the captured image. Contains information about conditions. The communication section 5013 provides the received control signal to the camera head control section 5015 . Note that the control signal from the CCU 5039 may also be transmitted by optical communication. In this case, the communication unit 5013 is provided with a photoelectric conversion module that converts an optical signal into an electrical signal, and the control signal is provided to the camera head control unit 5015 after being converted into an electrical signal by the photoelectric conversion module.

Note that the imaging conditions such as the frame rate, exposure value, magnification, and focus are automatically set by the control unit 5063 of the CCU 5039 based on the acquired image signal. That is, the endoscope 5001 is equipped with so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function.

The camera head control unit 5015 controls driving of the camera head 5005 based on control signals from the CCU 5039 received via the communication unit 5013 . For example, the camera head control unit 5015 controls the driving of the imaging element of the imaging unit 5009 based on the information specifying the frame rate of the captured image and/or the information specifying the exposure during imaging. Also, for example, the camera head control unit 5015 appropriately moves the zoom lens and the focus lens of the lens unit 5007 via the driving unit 5011 based on information specifying the magnification and focus of the captured image. The camera head control unit 5015 may further have a function of storing information for identifying the lens barrel 5003 and camera head 5005 .

By arranging the lens unit 5007, the imaging unit 5009, and the like in a sealed structure with high airtightness and waterproofness, the camera head 5005 can be made resistant to autoclave sterilization.

Next, the functional configuration of the CCU 5039 will be explained. A communication unit 5059 is configured by a communication device for transmitting and receiving various information to and from the camera head 5005 . The communication unit 5059 receives image signals transmitted from the camera head 5005 via the transmission cable 5065 . At this time, as described above, the image signal can be preferably transmitted by optical communication. In this case, the communication unit 5059 is provided with a photoelectric conversion module that converts an optical signal into an electrical signal for optical communication. The communication unit 5059 provides the image processing unit 5061 with the image signal converted into the electric signal.

The communication unit 5059 also transmits a control signal for controlling driving of the camera head 5005 to the camera head 5005 . The control signal may also be transmitted by optical communication.

An image processing unit 5061 performs various types of image processing on the image signal, which is RAW data transmitted from the camera head 5005 . The image processing includes, for example, development processing, image quality improvement processing (band enhancement processing, super-resolution processing, NR (noise reduction) processing and/or camera shake correction processing, etc.), and/or enlargement processing (electronic zoom processing). etc., various known signal processing is included. Also, the image processing unit 5061 performs detection processing on the image signal for performing AE, AF, and AWB.

The image processing unit 5061 is configured by a processor such as a CPU or GPU, and the processor operates according to a predetermined program to perform the image processing and detection processing described above. Note that when the image processing unit 5061 is composed of a plurality of GPUs, the image processing unit 5061 appropriately divides information related to image signals and performs image processing in parallel by the plurality of GPUs.

The control unit 5063 performs various controls related to the imaging of the surgical site by the endoscope 5001 and the display of the captured image. For example, the control unit 5063 generates control signals for controlling driving of the camera head 5005 . At this time, if the imaging condition is input by the user, the control unit 5063 generates a control signal based on the input by the user. Alternatively, when the endoscope 5001 is equipped with the AE function, the AF function, and the AWB function, the control unit 5063 optimizes the exposure value, focal length, and A white balance is calculated appropriately and a control signal is generated.

Further, the control unit 5063 causes the display device 5041 to display an image of the surgical site based on the image signal subjected to image processing by the image processing unit 5061 . At this time, the control unit 5063 recognizes various objects in the surgical site image using various image recognition techniques. For example, the control unit 5063 detects the shape, color, and the like of the edges of objects included in the surgical site image, thereby detecting surgical tools such as forceps, specific body parts, bleeding, mist when using the energy treatment tool 5021, and the like. can recognize. When displaying the image of the surgical site on the display device 5041, the control unit 5063 uses the recognition result to superimpose and display various surgical assistance information on the image of the surgical site. By superimposing and displaying the surgery support information and presenting it to the operator 5067, it becomes possible to proceed with the surgery more safely and reliably.

A transmission cable 5065 connecting the camera head 5005 and the CCU 5039 is an electrical signal cable compatible with electrical signal communication, an optical fiber compatible with optical communication, or a composite cable of these.

Here, in the illustrated example, wired communication is performed using the transmission cable 5065, but communication between the camera head 5005 and the CCU 5039 may be performed wirelessly. If the communication between them is performed wirelessly, it is not necessary to lay the transmission cable 5065 in the operating room, so the situation that the transmission cable 5065 interferes with the movement of the medical staff in the operating room can be resolved.

An example of the endoscopic surgery system 5000 to which the technology according to the present disclosure can be applied has been described above. Although the endoscopic surgery system 5000 has been described as an example here, the system to which the technique according to the present disclosure can be applied is not limited to such an example. For example, the technology according to the present disclosure may be applied to a flexible endoscope system for inspection and a microsurgery system.

<1.2. Specific Configuration Example of Medical Support Arm Device>
Next, a specific configuration example of the medical support arm device according to the embodiment of the present disclosure will be described in detail. The support arm device described below is an example configured as a support arm device that supports an endoscope at the distal end of an arm portion, but the present embodiment is not limited to such an example.

First, referring to FIG. 3, a schematic configuration of a support arm device 1400 according to this embodiment will be described. FIG. 3 is a schematic diagram showing the appearance of the support arm device 1400 according to this embodiment.

A support arm device 1400 according to this embodiment includes a base portion 1410 and an arm portion 1420 . A base portion 1410 is a base of the support arm device 1400 and an arm portion 1420 extends from the base portion 1410 . In addition, although not shown in FIG. 3, a control unit that integrally controls the support arm device 1400 may be provided in the base unit 1410, and the driving of the arm unit 1420 may be controlled by the control unit. good. The control unit is composed of various signal processing circuits such as a CPU and a DSP, for example.

The arm section 1420 has a plurality of active joint sections 1421a to 1421f, a plurality of links 1422a to 1422f, and an endoscope device 423 as a distal end unit provided at the distal end of the arm section 1420. FIG.

Links 1422a to 1422f are substantially bar-shaped members. One end of the link 1422a is connected to the base portion 1410 via the active joint portion 1421a, the other end of the link 1422a is connected to one end of the link 1422b via the active joint portion 1421b, and the other end of the link 1422b is the active joint. It is connected to one end of the link 1422c via the portion 1421c. The other end of link 1422c is connected via passive slide mechanism 1100 to link 1422d, and the other end of link 1422d is connected via passive joint 200 to one end of link 1422e. The other end of link 1422e is connected to one end of link 1422f via active joints 1421d and 1421e. The endoscope device 1423 is connected to the distal end of the arm portion 1420, that is, the other end of the link 1422f via the active joint portion 1421f. In this way, with the base portion 1410 as a fulcrum, the ends of the plurality of links 1422a to 1422f are connected to each other by the active joint portions 1421a to 1421f, the passive slide mechanism 1100, and the passive joint portion 1200. An elongated arm shape is configured.

The position and posture of the endoscope device 1423 are controlled by controlling the actuators provided in the active joints 1421a to 1421f of the arm 1420. As shown in FIG. In this embodiment, the endoscope device 1423 enters into the body cavity of the patient whose distal end is the site to be treated, and photographs a partial area of the site to be treated. However, the distal end unit provided at the distal end of the arm portion 1420 is not limited to the endoscope device 1423, and various medical instruments may be connected to the distal end of the arm portion 1420 as the distal end unit. Thus, the support arm device 1400 according to this embodiment is configured as a medical support arm device equipped with medical instruments.

Below, the support arm device 1400 will be described by defining coordinate axes as shown in FIG. Also, the up-down direction, front-rear direction, and left-right direction are defined according to the coordinate axes. That is, the vertical direction with respect to the base portion 1410 installed on the floor is defined as the z-axis direction and the vertical direction. Also, the direction perpendicular to the z-axis and the direction in which the arm portion 1420 extends from the base portion 1410 (that is, the direction in which the endoscope device 1423 is positioned with respect to the base portion 1410) is the y-axis. Define direction and front-to-rear direction. Further, directions orthogonal to the y-axis and z-axis are defined as x-axis direction and left-right direction.

The active joints 1421a-1421f rotatably connect the links to each other. The active joints 1421a to 1421f have actuators, and have rotation mechanisms that are driven to rotate about a predetermined rotation axis by driving the actuators. By controlling the rotational drive of each of the active joints 1421a to 1421f, it is possible to control the driving of the arm 1420, such as extending and contracting (folding) the arm 1420, for example. Here, the driving of the active joints 1421a to 1421f can be controlled by, for example, well-known systemic coordinated control and ideal joint control. As described above, since the active joints 1421a to 1421f have a rotation mechanism, in the following description, drive control of the active joints 1421a to 1421f specifically refers to the rotation angle and /or means that the generated torque (torque generated by the active joints 1421a to 1421f) is controlled.

The passive slide mechanism 1100 is one aspect of the passive form changing mechanism, and connects the link 1422c and the link 1422d so as to move forward and backward along a predetermined direction. For example, the passive slide mechanism 1100 may connect the link 1422c and the link 1422d so as to be able to move linearly with each other. However, the forward/backward motion of the link 1422c and the link 1422d is not limited to linear motion, and may be forward/backward motion in an arc-shaped direction. The passive slide mechanism 1100 is, for example, operated to advance and retreat by a user, and makes the distance between the active joint portion 1421c on one end side of the link 1422c and the passive joint portion 1200 variable. This can change the overall shape of the arm portion 1420 . The details of the configuration of the passive slide mechanism 1100 will be described later.

The passive joint section 1200 is one aspect of the passive form changing mechanism, and rotatably connects the link 1422d and the link 1422e to each other. The passive joint section 1200 is, for example, rotated by a user to change the angle formed by the link 1422d and the link 1422e. This can change the overall shape of the arm portion 1420 . The details of the configuration of the passive joint section 1200 will be described later.

In this specification, the “posture of the arm” means that the active joints 1421a to 1421f are moved by the control unit while the distance between the active joints adjacent to each other across one or more links is constant. It refers to the state of the arm that can be changed by the drive control of the actuator provided in the arm. In addition, the "shape of the arm" refers to the distance between the active joints adjacent to each other across the link, or the distance between the active joints that connect the adjacent active joints, as the passive shape changing mechanism is operated. It refers to the state of the arms that can change by changing the angle between them.

The support arm device 1400 according to this embodiment has six active joints 1421a to 1421f, and the arm 1420 is driven with six degrees of freedom. In other words, drive control of the support arm device 1400 is realized by drive control of the six active joints 1421a to 1421f by the control unit, while the passive slide mechanism 1100 and the passive joints 1200 are not subject to drive control by the control unit. is not.

Specifically, as shown in FIG. 3, the active joints 1421a, 1421d, and 1421f rotate the longitudinal direction of the connected links 1422a and 1422e and the imaging direction of the connected endoscope device 1423. It is provided so as to be in the axial direction. The active joints 1421b, 1421c, 1421e define the connection angle of the connected links 1422a to 1422c, 1422e, 1422f and the endoscope device 423 on the yz plane (the plane defined by the y-axis and the z-axis). It is provided so that the x-axis direction, which is the direction of change inside, is the rotation axis direction. Thus, in this embodiment, the active joint portions 1421a, 1421d, and 1421f have a so-called yawing function, and the active joint portions 1421b, 1421c, and 1421e have a so-called pitching function.

By having such a configuration of the arm portion 1420, the support arm device 1400 according to the present embodiment realizes six degrees of freedom for driving the arm portion 1420. The mirror device 1423 can be freely moved. FIG. 3 illustrates a hemisphere as an example of the movable range of the endoscope device 1423 . Assuming that the center point RCM (remote motion center) of the hemisphere is the imaging center of the treatment site imaged by the endoscope device 1423, with the imaging center of the endoscope device 1423 fixed at the center point of the hemisphere, By moving the endoscope device 1423 on the spherical surface of the hemisphere, the treatment site can be imaged from various angles.

In addition to the above degrees of freedom, the arm portion 1420 may have a degree of freedom 1421g for rotating the endoscope device 1423 coaxially with the link 1422f. As a result, the endoscope device 1423 can be rotated around the longitudinal axis of the link 1422f.

<1.3. Configuration example of control device>>
So far, the configuration of the support arm device 1400 according to the present embodiment has been described. A configuration example of a control device for controlling the driving of the arm portion 1420 in the support arm device 1400 according to the present embodiment, that is, the rotational driving of the actuator 1430 provided in the active joint portions 1421a to 1421f will be described below.

FIG. 4 is a block diagram showing an overall configuration example of the support arm device 1400 including the control device 1350. As shown in FIG. The control device 1350 includes a control section 1351 , a storage section 1357 and an input section 1359 .

The control unit 1351 is composed of various signal processing circuits such as a CPU and a DSP. The control unit 1351 integrally controls the control device 1350 and performs various calculations for controlling the driving of the arm unit 1420 in the support arm device 1400 . Specifically, the control unit 1351 has a whole body coordination control unit 1353 and an ideal joint control unit 1355 . The whole body coordination control section 1353 performs various calculations in the whole body coordination control in order to drive and control the actuators 1430 provided in the active joint sections 1421a to 1421f of the arm section 1420 of the support arm device 1400. FIG. The ideal joint control unit 1355 performs various calculations in ideal joint control that realizes an ideal response to whole-body cooperative control by correcting the influence of disturbance. The storage unit 1357 may be, for example, a storage element such as a RAM (Random Access Memory) or a ROM (Read Only Memory), or may be a semiconductor memory, a hard disk, or an external storage device.

The input unit 359 is an input interface for the user to input information, instructions, etc. regarding drive control of the support arm device 400 to the control unit 351 . The input unit 359 has operation means such as a lever or a pedal operated by a user, and the position, speed, etc. of each component of the arm unit 420 changes instantaneously according to the operation of the lever, pedal, or the like. may be set as a goal. The input unit 359 may have operation means operated by the user, such as a mouse, a keyboard, a touch panel, buttons, and switches, in addition to levers and pedals.

Also, the arm portion 1420 controlled by the control device 1350 includes an active joint portion 1421 . The active joint section 1421 (1421a to 1421f) has various configurations necessary for driving the arm section 1420, such as supporting members for connecting or supporting the links 1422a to 1422f and the endoscope device 1423. In the description up to this point and the description below, driving the joints of the arm 1420 may mean driving the actuators 430 in the active joints 1421a to 1421f.

Active joint 1421 comprises torque sensor 1428 , encoder 1427 and actuator 1430 . Although actuator 1430 , encoder 1427 and torque sensor 1428 are shown separately in FIG. 4 , encoder 1427 and torque sensor 1428 may be included in actuator 1430 .

Actuator 1430 is composed of a motor, a motor driver, and a speed reducer. Actuator 1430 is, for example, an actuator corresponding to force control. In actuator 1430, the rotation of the motor is reduced at a predetermined reduction ratio by the speed reducer, and transmitted to another member in the subsequent stage via the output shaft, thereby driving the other member.

A motor is a driving mechanism that produces rotational driving force. The motor is controlled by the motor driver so as to generate torque corresponding to the torque command value from the controller. A brushless motor, for example, is used as the motor. However, the present embodiment is not limited to such an example, and various known types of motors may be used as the motor.

A motor driver is a driver circuit (driver IC (Integrated Circuit)) that rotates a motor by supplying current to the motor, and controls the number of revolutions of the motor by adjusting the amount of current supplied to the motor. can be done. The motor driver drives the motor by supplying the motor with a current corresponding to the torque command value τ from the control unit.

Also, the motor driver can adjust the viscous resistance coefficient in the rotary motion of the actuator 1430 by adjusting the amount of current supplied to the motor. This makes it possible to apply a predetermined resistance to the rotational movement of the actuator 1430, that is, the rotational movement of the active joints 1421a to 1421f. For example, the active joints 1421a to 1421f can be put in a state in which they are easily rotated by a force applied from the outside (that is, in a state in which the arm 1420 is easily moved manually). A state in which it is difficult to rotate against a force (ie, a state in which it is difficult to manually move the arm portion 1420) can be made.

A speed reducer is connected to the rotation shaft (drive shaft) of the motor. The speed reducer reduces the rotation speed of the rotation shaft of the connected motor (that is, the rotation speed of the input shaft) at a predetermined reduction ratio and transmits the reduced speed to the output shaft. In this embodiment, the configuration of the speed reducer is not limited to a specific one, and various known types of speed reducer may be used as the speed reducer. However, as the speed reducer, it is preferable to use a speed reducer such as Harmonic Drive (registered trademark) whose speed reduction ratio can be set with high accuracy. Also, the speed reduction ratio of the speed reducer can be appropriately set according to the application of the actuator 1430 . For example, if the actuator 1430 is applied to the active joints 1421a to 1421f of the support arm device 400 as in this embodiment, a speed reducer having a speed reduction ratio of about 1:100 can be preferably used.

Encoder 1427 detects the rotation angle of the input shaft (that is, the rotation angle of the rotation shaft of the motor). Based on the rotation speed of the input shaft detected by the encoder 1427 and the speed reduction ratio of the speed reducer, information such as the rotation angles, rotation angular velocities, and rotation angular accelerations of the active joints 1421a to 1421f can be obtained. As the encoder 1427, various known rotary encoders such as a magnetic encoder and an optical encoder may be used. The encoder 1427 may be provided only on the input shaft of the actuator 1430, or an encoder for detecting the rotation angle and the like of the output shaft of the actuator 1430 may be provided after the speed reducer.

Torque sensor 1428 is connected to the output shaft of actuator 1430 and detects torque acting on actuator 1430 . Torque sensor 1428 detects the torque (generated torque) output by actuator 1430 . Torque sensor 1428 can also detect an external torque applied to actuator 1430 from the outside.

The configuration of the active joint portion 1421 has been described above. Here, in this embodiment, the motion of the arm portion 1420 is controlled by force control. In the force control, in the support arm device 1400, the rotation angles of the active joints 1421a to 1421f and the torque acting on the active joints 1421a to 1421f are determined by the encoders 1427 and the torque sensors 1428 provided in the actuators 1430. are detected respectively. At this time, the torque acting on each of the active joints 1421a to 1421f detected by the torque sensor 1428 may include force acting on the arm 1420 and/or the endoscope device 1423.

Also, based on the rotation angle detected by the encoder 1427 and the torque value detected by the torque sensor 1428, the current state (position, speed, etc.) of the arm section 1420 can be acquired. In the support arm device 1400, based on the acquired state of the arm portion 1420 (arm state), the actuators provided in the active joint portions 1421a to 1421f necessary for the arm portion 1420 to perform the desired exercise purpose are controlled. The torque to be generated by 1430 is calculated, and the actuators 1430 of the active joints 1421a to 1421f are driven using this torque as a control value.

As the actuator 1430, various known actuators generally used in various devices whose operations are controlled by force control can be used. For example, as the actuator 1430, those described in JP-A-2009-269102, JP-A-2011-209099, etc., which are prior patent applications filed by the applicant of the present application, can be suitably used.

In the support arm device 1400 according to the present embodiment, the configurations of the actuator 1430 and the components that make up the actuator are not limited to the configurations described above, and may be other configurations.

The basic configuration of the endoscope system has been described above. Specific embodiments of the endoscope system described above will be described below.

<<2. Oblique Rotation Operation and Observation Observation Observation Operation by Observation Mirror>>
<2.1. Rotational Movement of Periscope>
In this embodiment, a perspective scope is used as the endoscope 5001 (endoscope device 423) described above. FIG. 5 is a schematic diagram showing the appearance of the perspective mirror 100. As shown in FIG. In the perspective mirror 100, the orientation (C1) of the objective lens toward the subject forms a predetermined angle φ with respect to the longitudinal direction of the perspective mirror 100 (scope axis C2). That is, in the oblique scope 100, the objective optical system forms an angle with respect to the ocular optical system of the scope. In the squinting scope 100, an operation of rotating the squinting scope 100 around the scope axis C2 as a rotation axis (hereinafter referred to as oblique rotation) is performed for observation. By performing oblique rotation, it is possible to obtain a wraparound field of view and up, down, left, and right peripheral vision.

FIG. 6 is a schematic diagram showing how the field of view 200 displayed on the display device 5041 changes due to oblique rotation. As shown in FIG. 6, in a state where the field of view 200 is displayed on the monitor, when the perspective mirror is obliquely rotated by the oblique rotation angle α and the vertical direction of the field of view 200 is controlled, the field of view displayed on the monitor is the field of view 200. to the field of view 200'. Therefore, by controlling oblique rotation and two axes in the vertical direction of the camera, it is possible to obtain peripheral vision in the vertical and horizontal directions.

FIG. 7 is a schematic diagram showing how the field of view displayed on the monitor changes due to oblique rotation. FIG. 7 shows how the inside of the body is observed with a squintoscope, and shows a state in which an observation object 210 that the user (operator) wants to see exists among various organs 215 . In FIG. 7, the field of view 200 of the monitor is indicated by a rectangular area, and together with the field of view 200, the observation object 210 that the user (operator) intends to see and the scope axis C2 are shown. FIG. 7 also shows a state in which the operator brings a surgical instrument 220 such as forceps into contact with the observation object 210 and holds the observation object 210 .

As shown in FIG. 7, the observation object 210 is positioned below the field of view 200 (on the side of the operator who operates the surgical instrument 220) with respect to the position of the scope axis C2. FIG. 7 shows how the observed object 210 moves with respect to the field of view 200 when the oblique scope 100 is rotated about the scope axis C2.

7 shows a state in which the optical axis of the oblique mirror 100 objective optical system is directed downward in the vertical direction of the field of view 200. FIG. In this case, the observation object 210 enters the field of view 200, and the operator can visually recognize the observation object 210 on the monitor.

7 shows a state in which the optical axis of the objective optical system of the oblique mirror 100 is oriented rightward in the lateral direction of the field of view 200. FIG. In this case, the observation object 210 is out of the field of view 200, and the operator cannot visually recognize the observation object 210 on the monitor.

7 shows a state in which the optical axis of the objective optical system of the oblique mirror 100 is directed upward in the vertical direction of the field of view 200. FIG. 7 shows a state in which the optical axis of the objective optical system of the oblique mirror 100 is oriented leftward in the horizontal direction of the field of view 200. FIG. In these cases as well, the observation object 210 is out of the field of view 200, and the operator cannot visually recognize the observation object 210 on the monitor.

As described above, with the perspective scope 100, by performing oblique rotation, the position of the field of view with respect to the observed object 210 can be changed, and a relatively wide range around the scope axis C2 can be visually recognized. On the other hand, as shown in FIG. 7, there is a possibility that the portion or instrument that the user wants to see will be out of the field of view 200 of the monitor due to the oblique rotation, and the user will lose sight of those places.

Therefore, in this embodiment, the position of the observed object 210 is detected, and the arm is controlled so that the observed object 210 moves to the center of the screen. Further, in the present embodiment, the rotation speed during oblique rotation is changed according to the position of the observation object 210 with respect to the field of view 200 . Specifically, control is performed so that the rotation speed during oblique rotation becomes slower as the observation object 210 is further away from the center of the field of view 200 . The farther the observation object 210 is from the center of the field of view 200, the more likely the observation object 210 will be out of the field of view 200 when the oblique rotation is performed. For this reason, the more the observation object 210 moves away from the center of the field of view 200, the slower the rotation speed during oblique rotation. It will be suppressed. However, since the tracking function and the squint rotation speed are different, if you don't control the movement speed of the degree of freedom to move the screen up, down, left and right for tracking and the squint rotation speed in coordination, it may be out of the screen. have a nature. Therefore, while controlling the observation object 210 to come to the center of the screen, the oblique rotation speed is controlled to be reduced according to the distance L from the screen center to the object 21 or the relative position.

<2.2. Change in oblique rotation speed according to distance from center of visual field>
Here, the change of the oblique rotation speed according to the distance from the center of the field of view 200 to the observed object 210 will be described. FIG. 8 is a schematic diagram showing a method of calculating the distance from the center of the field of view 200 to the observed object 210. As shown in FIG. As shown in FIG. 8, when the oblique rotation speed is ω [rad/s] and the number of pixels from the screen center O to the position p of the observation object 210 is (x, y) [pixels], observation from the screen center is performed. The number of pixels l to the position of the object 210 is given by l=√(x ² +y ² ). From here, it is realized by adjusting the oblique rotation speed ω=f(l) as a function. For example, when the number of pixels from the center of the screen to the diagonal screen vertex is lmax [pixels], and the maximum oblique rotation speed is ωmax,
ω=ωmax*(lmax−l)=ωmax*(lmax−√(x2+y2))
and
As a result, ω=ωmax when the position of the observation object 210 is at the center of the screen, and ω=0 when the position of the observation object 210 is at the diagonal screen vertex. In this example, a linear calculation is used, but a higher-order function or the like may be used. Also, if the distance Z [mm] from the camera to the subject and the angle of view θ are known, the distance L [mm] from the center of the screen to the observed object 210 may be calculated to set ω=f(L). .

FIG. 9 is a schematic diagram showing an example of a map that defines the oblique rotation speed according to the subject position within the screen. Based on FIG. 8, the map 400 shows an example in which the oblique rotation speed decreases as the distance from the screen center O increases. The density of dots in map 400 corresponds to the speed of oblique rotation, and map 410 shows the relationship between the density of dots in map 400 and the speed of oblique rotation. As shown in map 410, the darker the dot, the faster the oblique rotation. Therefore, in the map 400, the speed of oblique rotation decreases as the distance from the screen center O increases.

It also shows the relationship between the density of dots, the oblique rotation speed, and the distance to the subject (observation object 210). According to the map 420, the closer the distance to the subject, the lower the oblique rotation speed. The closer the distance to the subject, the greater the amount of movement of the observation object 210 within the field of view 200 in accordance with the movement of the oblique mirror 100. Losing sight of the observation object 210 from the observation object 200 can be suppressed.

FIG. 10 is a characteristic diagram showing the relationship between the distance from the screen center O to the observed object 210 (horizontal axis) and the oblique rotation speed (vertical axis). As described above, control is performed so that the oblique rotation speed decreases as the distance from the screen center O to the observation object 210 increases, but variations such as those shown in FIG. 10 can be assumed as control methods.

In the case of characteristic 1 shown in FIG. 10, control is performed with priority given to the oblique rotation speed. If the movement of the arm in the XY directions is sufficiently fast, it is possible to follow the observed object 210 even with such characteristics. Since the oblique mirror rotation speed is not 0 even at the screen edge (Lmax), the observation object 210 may be out of the screen depending on the position of the observation object 210 at the start of oblique rotation.

In the case of characteristics 2 and 3 shown in FIG. 10, it is assumed that the speed of the XY movement of the arm is slower than the development of the field of view due to oblique rotation. When the observation object 210 is close to the center O of the screen, the oblique rotation is fast, and the oblique mirror rotation speed is not 0 even at the edge of the screen (Lmax). Since the observation object 210 has a speed even when it is separated from the screen center O by a certain distance or more, the observation object 210 may be out of the screen depending on the position of the observation object 210 at the start of oblique rotation.

Further, in the case of characteristic 4 shown in FIG. 10, it is assumed that the XY movement is considerably slow and the observation object 210 is never removed from the field of view 200 . When the observation object 210 deviates from the screen center O to some extent, the oblique rotation is completely stopped and only the XY movement is performed.

In any of the properties 1 to 4, the properties change depending on the relationship between the maximum oblique rotation speed and the XY movement speed of the arm, whether or not the observation object 210 is permitted to move outside the field of view 200, and the like. Become.
<2.3. Follow-Up Operation to Observation Object>
[How to move the arm to move the observation target to the center of the screen]
FIG. 11 shows how the orientation of the optical axis of the objective optical system changes due to the rotation of the perspective mirror 100 about the long axis (perspective rotation). As shown in FIG. 10, it is possible to change the viewing direction of the operator by oblique rotation, but the field of view also shifts as described with reference to FIG. On the other hand, FIG. 12 shows a case where movement in the XY directions is performed by giving a rotation of the optical axis centered on the trocar point or a movement close thereto. Note that the trocar point is the position where the trocar is inserted into the human body. As shown in FIG. 12, applying a rotational movement (perspective scope pivot movement) to the perspective scope 100 about the trocar point can change the perspective rotation angle without losing the field of view. When such a movement is performed, the oblique rotation angle and the XY movement are interlocked.

In the following, an example of how the oblique mirror 100 moves with respect to the field of view in combination with the oblique rotation of the oblique mirror 100 will be described. FIG. 13 shows an example of parallel movement without changing the posture of the oblique mirror, and is a schematic diagram showing a case where oblique rotation is performed along with the movement of parallel movement. In this case, the movement is performed without changing the attitude of the oblique mirror 100 with respect to the camera field of view.

Oblique angle α (around the Y axis), oblique rotation angle ψ (around the Z axis), rigid scope length L, distance R to the observation object, point V at the center of the visual field, amount of translation (ax, ay, az) , the simultaneous transformation matrix representing translation is as follows.

Also, a simultaneous conversion matrix for oblique angles and oblique rotation angles can be expressed as follows. For simplicity, the initial position of the base of the rigid scope is the origin of (0,0,0).

At this time, the point V at the center of the field of view is as follows.

As is clear from this formula above, the field of view can be moved by the same amount of movement as the translation of the rigid scope. Conversely, by controlling ax and ay according to the value of ψ so as to keep V constant, it is possible to keep looking at the target point while rotating obliquely as shown in FIG. Furthermore, if the amount of change in (ax, ay) to keep V at a fixed position is sufficiently faster than the amount of change in (Rcosψα, Rsinψsinα), the rotation of the oblique mirror will not cause the object to move out of the field of view. If the oblique rotation speed is controlled as described above, the oblique rotation can be performed without losing sight of the object on the screen. Also, since the relative difference between (Rcosψα, Rsinψsinα) and (ax, ay) affects the amount of change, the same movement can be realized by adjusting the velocity (ax, ay) in addition to the velocity of the oblique rotation angle. Such movement is limited to the case of no trocar point constraint, for example, strabiscopy with a small thoracotomy/small incision in the chest. Since the motion is in the Y direction, the distance between the field of view 200 and the camera is maintained.

Also, FIG. 14 shows an example in which the field of view is moved vertically and horizontally by rotation about the constraint point 300 . Constraint points 300 correspond to trocar points, for example. In this case, the arm can be configured with three degrees of freedom around the constraint point 300 . Let φ be the rotation about the X-axis and θ be the rotation about the Y-axis with the constraint point 300 as the center, then each rotation matrix is as follows.

Also, the simultaneous transformation matrix for rotation about the constraint point 300 is as follows.

At this time, the point V at the center of the field of view can be expressed as follows.

If θ, φ, and ψ are adjusted so that the above V is maintained, the field of view does not deviate. When the target oblique angle ψ is determined, the target values of θ and φ are selected so that the value of V is maintained. When controlling θ, φ, and ψ using these target values as command values, by adjusting the oblique rotation angle described above, it is possible to control the object without removing it from the field of view even in the process of following the target values. It becomes possible.

Also, since the amount of change in V relative to the change in θ and φ changes depending on the attitude of ψ, the rotational speed of θ and φ may be adjusted so that the amount of change in X and Y becomes the same according to the value of the oblique rotation angle ψ. . Since the above calculations are complicated, for simple implementation, α=0 and ψ=0 may be set and the rigid scope axis may be rotated by θ and φ regardless of the oblique rotation angle to simplify the movement of the field of view.

[Control of the arm that moves the object to be observed to the center of the screen]
In the following, an example of following the observation object 210 so as to be positioned in the center of the field of view 200 when the oblique scope 100 is moved around the constraint point 300 will be described. An arm control method may be used in which the position of an object within the screen is recognized by image recognition or the like, and the observation object is moved within the screen based on that information. It is also possible to determine the operator's observation target point by line-of-sight detection and move the observation target point to the center of the field of view.

According to this embodiment, the support arm device is controlled so that the object existing inside the patient's body is aligned with the optical axis of the endoscope attached to the support arm device and inserted into the patient's body. and a control unit that moves the endoscope by moving the endoscope. By recognizing the target using image recognition, etc., and controlling the arm so that the target appears in the center of the screen, it is possible to capture the target, such as a surgical instrument or a tumor, in the center of the image obtained by the endoscope. As a result, convenience for the operator is improved. Such an information processing device may be configured separately from the endoscopic surgery system 5000 or may be configured as an arbitrary device included in the endoscopic surgery system 5000 .

The outline of this embodiment will be described below with reference to FIGS. 15 and 16. FIG.

15 and 16 are diagrams for explaining the outline of this embodiment. FIG. 15 shows how the barrel 5003 of the endoscope 5001 is inserted into the body cavity of the patient 5071 through the trocar 5025a punctured in the abdominal wall of the patient 5071. FIG. The endoscope 5001 indicated by the solid line indicates the current position and orientation, and the endoscope 5001 indicated by the dashed line indicates the destination (that is, the future position) of the endoscope control processing according to the present embodiment. ) indicates the position and orientation. A forceps 5023 is inserted from a trocar 5025d punctured in the abdominal wall of the patient 5071. FIG. FIG. 16 shows images obtained by the endoscope 5001 shown in FIG. 15 (hereinafter also referred to as endoscopic images). It is an image obtained after movement by endoscope control processing according to the present embodiment.

Referring to FIG. 15, in the current position and orientation of the endoscope 5001, the tip of the forceps 5023 is captured within the field of view 6052 of the endoscope 5001, but not on the central axis (that is, the optical axis) 6051. . Therefore, as shown in the left diagram of FIG. 16, an endoscopic image is obtained in which the tip of the forceps 5023 is not shown in the center. In such a situation, the endoscopic surgery system 5000 according to the present embodiment performs a process of moving the endoscope 5001 so that the forceps 5023 and other surgical instruments are shown in the center of the screen. Specifically, the endoscopic surgery system 5000 moves the endoscope 5001 by a support arm device 5027 (not shown) so that the tip of the forceps 5023 is positioned on the central axis 6051 . As a result, as shown in the right diagram of FIG. 16, an endoscopic image is obtained in which the tip of the forceps 5023 is reflected in the center. An example in which the tip of the forceps 5023 is positioned on the central axis 6051 ignoring the oblique angle and oblique rotation angle will be described below. It can be performed. In addition, although an example in which the tip of the forceps 5023 is positioned on the central axis 6051 is shown, the observation object 210 can also be positioned on the central axis 6051 as a target by recognizing the image of the observation object 210. be able to.

In this way, the endoscopic surgery system 5000 can automatically follow the surgical tool and provide an endoscopic image in which the surgical tool is shown in the center of the screen. Therefore, the operator can comfortably continue the operation without operating the endoscope 5001 .

<2.4. Configuration Example of Control Unit for Oblique Rotation Operation and Follow-up Operation>
FIG. 21 is a schematic diagram showing the configuration of the control section 1351 of the support arm device 1400 for performing the oblique rotation motion and the follow-up motion described above. As shown in FIG. 21, the control unit 1351 includes a distance acquisition unit 1351a that acquires the distance from the center of the field of view to the position of the observation object 210 within the field of view 200 obtained by imaging the observation object 210, and a speed calculation unit 1351b that calculates the speed of oblique rotation of the squint mirror 100 or the moving speed of the squint mirror 100 based on the distance from the center of the observation object 210 to the position of the observation object 210 . Acquisition of the distance from the center of the field of view to the position of the observation object 210 by the distance acquisition unit 1351a is performed based on the result of image recognition of the observation object 210 by the control unit 5063 of the CCU 5039 .

Further, the control unit 1351 controls the support arm device 1400 so that the observation object 210 is positioned at the center of the field of view 200 based on the image recognition result of the observation object 210 by the control unit 5063 of the CCU 5039 . At this time, the control unit 1351 controls at least one of the oblique rotation speed of the squint mirror 100 and the movement speed of the squint mirror 100 according to the position of the observation object 210 within the field of view 200 . The control unit 1351 controls the rotation angle and rotation speed of the oblique mirror 100 about the major axis according to the position of the observation object 210 within the field of view 200 . The control unit 5063 of the CCU 5039 can recognize the features of the image of the observed object 210 and the position within the field of view 200 using various image recognition techniques, as described above. The control unit 1351 acquires information about the position of the observed object 210 within the field of view 200 from the CCU 5039 .

The operator can specify the observation object 210 from the operative site displayed on the display device 5041 by operating the input device 5047 while viewing the image of the operative site displayed on the display device 5041 in real time. . The control unit 5063 recognizes the features of the image of the observed object 210 and the position within the field of view 200 based on the specified observed object 210 .

<2.5. Details of follow-up operation>
Next, with reference to FIG. 17, processing for realizing the above-described endoscope control will be described.

FIG. 17 is a diagram for explaining an overview of endoscope control processing according to the present embodiment. Each block shown in FIG. 17 represents a process, and the endoscope control process consists of a plurality of processes. As shown in FIG. 17, the endoscopic surgery system 5000 performs image processing to detect targets such as surgical instruments. The endoscopic surgery system 5000 then calculates the position of the target based on the detection result. Next, the endoscopic surgery system 5000 calculates the current orientation of the endoscope based on the calculated target position and trocar position. Next, the endoscopic surgery system 5000 calculates the target distal end position of the endoscope based on the calculated current endoscope posture and setting information of the virtual plane (flat or curved plane). Next, the endoscopic surgery system 5000 calculates the amount of change in the posture of the endoscope based on the current posture of the endoscope and the target position of the distal end of the endoscope, and determines the amount of change according to the calculated amount of change. It generates arm control information (that is, commands) for realizing posture changes. The endoscopic surgery system 5000 then controls the support arm (for example, the arm section 5031) to operate according to the generated command. The endoscopic surgery system 5000 repeats the series of processes described above.

Details of the endoscope control process according to the present embodiment will be described below.

(1) Introduction According to this embodiment, the endoscopic surgery system 5000 can realize a function of recognizing a surgical tool from an endoscopic image and automatically following it. A calculation method for operating the endoscope by the arm based on the image processing portion (marker detection) and the detection result to move the surgical instrument to the center of the screen while considering the trocar point will be described below.

In the following, after explaining the functional requirements, the method of detecting the surgical tool (marker) by image processing is described, and then the calculation method of converting the detection result into target movement information and posture information and moving it is described. do.

Here, an example in which the oblique angle and oblique rotation angle are ignored and the arm is moved around the constraint point is shown.

(2) Image Processing The endoscopic surgery system 5000 detects a surgical tool (for example, the tip position and/or posture of the surgical tool) by image processing.

For example, the position of the surgical instrument may be detected by image processing based on endoscopic images by attaching a marker to the distal end of the surgical instrument. Markers should be easily detectable. For example, the marker may be a color such as blue or green that stands out from the color of the organ or blood vessel in the body cavity (eg, the color that is on the opposite side of the color wheel from the color of the organ or blood vessel). Alternatively, the marker may be a specific pattern such as a two-dimensional code or barcode.

For example, by attaching a marker to the part of the surgical instrument that is outside the body, the position of the surgical instrument can be detected based on information such as the marker detection result by an external sensor, the length and posture of the surgical instrument. may

Note that detection of the surgical tool may be performed by a method other than image processing.

For example, by creating a special trocar, the position of the surgical instrument may be calculated based on the insertion amount of the surgical instrument and the angle of the trocar.

For example, by attaching the surgical tool to a support arm device different from the endoscope, the position of the surgical tool may be calculated from the position and orientation information of the support arm device.

(3) Target Calculation The endoscopic surgery system 5000 performs target calculation. The target calculation is a calculation for instructing movement by calculating both position and orientation.

Specifically, the endoscopic surgery system 5000 first obtains the target position from the image processing result, and then determines the amount of change in posture based on the current posture with the trocar point as the starting point and the posture when the target position is reached. do. The endoscopic surgery system 5000 performs target calculation based on the current position/orientation obtained from the encoder while the movement amount is obtained from the result of image processing. Add the calculated value to the command value. The reason for this is that there is a gap between the current value and the command value due to a control error. This is because a problem occurs in which the error increases.

An example of the flow of the target calculation process will be described below with reference to FIG.

FIG. 18 is a flowchart showing an example of the flow of target calculation processing by the endoscopic surgery system 5000 according to this embodiment. As shown in FIG. 18, the endoscopic surgery system 5000 first performs coordinate calculation.

In the coordinate calculation, the endoscopic surgery system 5000 first calculates the coordinates based on the current value. Specifically, the endoscopic surgery system 5000 acquires the image processing result (step S402). Next, the endoscopic surgery system 5000 transforms the detected position into the camera coordinate system (that is, transforms from 2D to 3D) (step S404). Next, the endoscopic surgery system 5000 transforms from the camera coordinate system to the World coordinate system (step S406). Next, the endoscopic surgery system 5000 converts the trocar points to unit vectors (step S408). Next, the endoscopic surgery system 5000 obtains the length up to the intersection with the predetermined plane (that is, the virtual plane) (step S410). Next, the endoscopic surgery system 5000 transforms the vector from the trocar point to the predetermined plane into the World coordinate system (step S412).

After calculating the coordinates based on the current value, the endoscopic surgery system 5000 calculates the coordinates based on the command value. Specifically, the endoscopic surgery system 5000 converts the length of the endoscope into the insertion depth (step S414).

After coordinate calculation, the endoscopic surgery system 5000 performs attitude calculation.

In posture calculation, the endoscopic surgery system 5000 first calculates the posture based on the current value. Specifically, the endoscopic surgery system 5000 acquires the current posture vector (step S416). Next, the endoscopic surgery system 5000 obtains the posture of the calculated new target vector (step S418). Next, the endoscopic surgery system 5000 obtains a relative posture change amount with respect to the calculated new target vector (step S420).

After calculating the posture based on the current value, the endoscopic surgery system 5000 calculates the posture based on the command value. Specifically, the endoscopic surgery system 5000 converts the posture change amount from the posture of the final command value (step S422).

Through the processing described above, the endoscopic surgery system 5000 obtains the target position and target orientation.

(4) Target Position Calculation FIG. 19 is a diagram for explaining target position calculation according to this embodiment. As shown in FIG. 19, in the image processing result, the position viewed from the camera coordinate system with the center of the screen at the tip of the camera as (0.5, 0.5) is normalized to [0.0-1.0]. specified value. Due to the dimensionless values as is, the endoscopic surgical system 5000 first converts to the metric unit system. However, in the endoscopic surgery system 5000, the image processing result is 2D and there is no information in the depth direction. do.

The reason why the depth is assumed to be 50 [mm] will be explained. The first reason is that if the assumed values are larger than the actual values, the amount of movement of (x, y) becomes larger than the actual (x, y) and overruns (oscillates). The second reason is that the photographing distance in the assumed technique is set to 50 [mm] to 100 [mm], which is the shortest distance. The third reason is that when the actual distance is larger, the movement is newly determined from the residual in the next image processing result, so that the goal can be finally reached.

(5) Desired Posture Calculation The endoscopic surgery system 5000 obtains the desired posture after the target position is determined. The control unit 1351 controls the arm unit 1420 based on the target position and the target posture.

<2.6. Specific example of follow-up operation>
FIGS. 20A to 20D are schematic diagrams for explaining a specific example when the squint is turned to the left by controlling the squint mirror according to the present embodiment. Similar to FIG. 7, FIGS. 20A to 20D also show states in which observation objects 210 that the user (operator) wants to see exist in various organs 215. FIG. Here, FIG. 20A shows a state (perspective state) in which the optical axis of the objective optical system faces the lower side of the field of view 200 (toward the operator who operates the surgical tool 220) with respect to the position of the scope axis C. showing. FIG. 20B shows an enlarged view of the field of view 200 of FIG. 20A. In this case, when oblique rotation is performed to look at the left side of the field of view 200, as shown in FIG. Although the left side of the object 210 can be observed, the observed object 210 becomes invisible.

On the other hand, in this embodiment, as shown in FIG. 20D, the arm is controlled to follow the observation object 210 so that the observation object 210 is positioned at the center of the screen. It is possible to rotate the squint scope 100 to the left without any movement.

<<3. Example of providing a holding unit that independently controls the rotation of the perspective scope>>
In the configuration example shown in FIG. 3 described above, an example in which the attitude and position of the oblique scope 100 are changed only by controlling the active joints 1421a to 1421f of the arm 1420 is shown. On the other hand, a holding unit with a built-in actuator may be provided at the tip of the arm portion 1420 to independently control the rotation of the oblique mirror. In this case, the rotation of the oblique scope 100 is performed by the holding unit, and the position of the oblique scope as a whole and the posture with respect to the surgical site can be controlled by the active joint portion of the arm.

FIG. 22 is a schematic diagram showing a configuration in which a holding unit 7000 for independently controlling the rotation of the perspective mirror 100 and the rotation of the camera head 7100 is provided at the tip of the arm section 1420 in the configuration example shown in FIG. . The holding unit 7000 is attached to the endoscope unit mounting portion 7300 at the tip of the arm portion 1420, and includes a camera head mounting portion 7100, a camera head rotation driving portion 7200, a perspective mirror mounting portion 7400, and a perspective mirror rotation driving portion 7500. there is

As shown in FIG. 22, the camera head 5005 is attached to the camera head rotation drive section 7200 via the camera head attachment section 7100 . The camera head rotation drive section 7200 includes an actuator 7210 such as a motor, and rotates the camera head 5005 with respect to the body of the endoscope unit attachment section 7300 and the holding unit 7000 .

Also, the oblique mirror 100 is attached to the oblique mirror rotation driving section 7500 via the oblique mirror mounting section 7400 . The squint scope rotation driving section 7500 includes an actuator 7510 such as a motor, and rotates the squint scope 100 about its axis with respect to the body of the endoscope unit mounting section 7300 and the holding unit 7000 .

FIG. 23 is a schematic diagram showing the configuration of the support arm device 1400 including the arm section 1420 and the control device 1350, the endoscope unit 5006, and the CCU 5039 in the configuration example shown in FIG. The control device 1350 includes a CCU communication unit 1358 that communicates with the CCU 5039 in addition to the configuration shown in FIG. The CCU 5039 has an arm communication section 5064 that communicates with the control device 1350 in addition to the configuration shown in FIG. The control device 1350 and the CCU 5039 can exchange information with each other through communication between the CCU communication section 1358 and the arm communication section 5064 . An endoscope unit 5006 is obtained by adding the function of the holding unit 7000 to the camera head 5005 shown in FIG. It has The endoscope unit 5006 includes, in addition to the configuration of the camera head 5005 shown in FIG. 7500 and a camera head rotation drive unit 7200 . The perspective controller 5016 and camera head controller 5017 may be provided in the holding unit 7000 . Also, the functions of the endoscope unit control section 5014, the perspective control section 5016, and the camera head control section 5017 may be provided in the control section 1351 of the control device 1350. FIG.

The oblique view control section 5016 drives the actuator 7510 based on a command from the endoscope unit control section 5014 . The actuator 7510 and the perspective rotation section encoder 7520 are provided in the perspective mirror rotation drive section 7500 . The endoscope unit control section 5014 drives the actuator 7510 based on the rotation angle of the actuator 7510 detected by the perspective rotation section encoder 7520 to control the rotation of the perspective mirror 100 around the axis.

Also, the camera head control section 5017 drives the actuator 7210 based on a command from the endoscope unit control section 5014 . Actuator 7210 and camera head rotation section encoder 7220 are provided in camera head rotation drive section 7200 . The endoscope unit control section 5014 controls the rotation of the camera head 5005 about the axis based on the rotation angle of the actuator 7210 detected by the camera head rotation section encoder 7220 .

With the above configuration, the oblique scope 100 and the camera head 5005 can be independently rotated with respect to the endoscope unit mounting portion 7300 . As a result, it is possible to rotate the perspective mirror 100 to visually recognize a desired observation object, and to rotate the camera head 5005 to properly control the orientation of the image.

24 is a sequence diagram showing the flow of processing in the configuration shown in FIGS. 22 and 23. FIG. First, in step S502, the endoscope unit sends to the CCU 5039 information on oblique rotation of the oblique scope 100 and orientation of the camera head 5005 for vertical control. In the next step S504, information regarding oblique rotation of the squint mirror 100 and orientation for vertical control of the camera head 5005 is sent from the CCU 5039 to the arm control device.

In the next step S506, arm control commands and endoscope unit drive command values are calculated. In the next step S508, the drive command value for the endoscope unit is sent from the arm control device to the CCU 5039 together with the drive command. Also, in step S510, the arm is controlled based on the arm control command.

In step S512, the drive command value for the endoscope unit is sent from the CCU 5039 to the endoscope unit together with the drive command. In the next step S514, the endoscope unit is controlled based on the drive command value.

In the next step S516, information on oblique rotation of the oblique scope 100 and orientation of the camera head 5005 for vertical control is sent from the endoscope unit to the CCU 5039. FIG. In the next step S504, information regarding oblique rotation of the squint mirror 100 and orientation for vertical control of the camera head 5005 is sent from the CCU 5039 to the arm control device.
In the next step S510,

<<4. Summary>>
As described above, according to the present embodiment, the speed of oblique rotation is decreased as the distance from the center of the field of view 200 of the monitor increases. It is possible to prevent the observed object 210 from moving out of the field of view 200 .

Further, according to the present embodiment, the observation object 210 existing inside the human body of the patient is image-recognized, and the position of the observation object 210 is aligned with the optical axis of the objective optical system of the squintoscope 100 . The endoscope is moved while controlling the arm to constrain the tip of the endoscope on a virtual plane. As a result, when oblique rotation is performed, it is possible to capture an observation object 210 such as a surgical instrument or a tumor in the center of the field of view 200, thereby improving the operator's convenience.

Therefore, the operator can perform oblique rotation while keeping the desired part, instrument, etc. within the screen at all times, and can suppress losing sight of the observation object 210 due to oblique rotation. Further, it is possible to easily realize the posture control based on the distance information as in the case of the pivot movement centering on the subject.

Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is obvious that those who have ordinary knowledge in the technical field of the present disclosure can conceive of various modifications or modifications within the scope of the technical idea described in the claims. is naturally within the technical scope of the present disclosure.

A series of processes by each device described in this specification may be realized using any of software, hardware, and a combination of software and hardware. Programs constituting software are stored in advance in a storage medium (non-transitory media) provided inside or outside each device, for example. Each program, for example, is read into a RAM when executed by a computer, and executed by a processor such as a CPU.

Further, the processes described using flowcharts and the like in this specification do not necessarily have to be executed in the illustrated order. Some processing steps may be performed in parallel. Also, additional processing steps may be employed, and some processing steps may be omitted.

Also, the effects described herein are merely illustrative or exemplary, and are not limiting. In other words, the technology according to the present disclosure can produce other effects that are obvious to those skilled in the art from the description of this specification, in addition to or instead of the above effects.

Note that the following configuration also belongs to the technical scope of the present disclosure.
(1) a multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end;
a control system for controlling the articulated arm to change the position and orientation of the oblique scope;
with
The surgical arm system, wherein the control system controls at least one of a speed of rotation or a speed of translation of the squintoscope within a field of view imaged through the squintoscope based on a position of an object viewed within the field of view.
(2) The surgical arm system according to (1), wherein the control system controls the rotational angle and rotational speed of the oblique scope about its longitudinal axis.
(3) The surgical arm system according to (1) or (2) above, wherein the control system controls the posture and movement speed of the articulated arm so as to move the position of the observation object within the field of view. .
(4) The control system controls at least one of the rotational speed and the moving speed of the oblique mirror based on the distance from the center of the field of view to the position of the observed object within the field of view, ) to (3), the surgical arm system.
(5) The surgical arm system according to any one of (1) to (4), wherein the control system reduces the rotation speed as the distance from the center of the field of view to the position of the observation object increases.
(6) The control system controls the articulated arm that supports the squintoscope so that the observation object is positioned at the center of the field of view, based on the result of determining the position of the observation object. A surgical arm system according to any one of (1) to (5).
(7) The control system controls the articulated arm that supports the squintoscope so that the observation object is positioned at the center of the field of view based on the result of image recognition of the observation object, The surgical arm system according to 6).
(8) The surgical arm system according to any one of (1) to (7), wherein the control system sets the rotation speed to 0 when the observation object is positioned at the edge of the field of view.
(9) The surgical arm system according to (5) above, wherein the control system controls the rotational speed so that the amount of decrease in rotational speed with respect to the amount of increase in distance has a linear relationship.
(10) The control system according to (5) above, wherein, when the distance is within a predetermined range, the rotational speed is the same as the rotational speed when the center and the position of the observation target match. surgical arm system.
(11) The surgical arm system according to (5), wherein the control system sets the rotation speed to a constant value when the distance exceeds a predetermined range.
(12) The control system according to any one of (1) to (11) above, wherein the control system includes a first control section that controls rotation of the oblique scope and a second control section that controls the articulated arm. surgical arm system.
(13) The surgical arm system according to (12), wherein the articulated arm includes a holding unit that holds the oblique scope and the camera head independently of each other so as to be rotationally controllable.
(14) The holding unit includes the first control section,
The second control unit acquires the rotation angle information of the squint mirror acquired by the first control unit via an image processing unit connected to the camera head, and based on the rotation angle information, The surgical arm system according to (13) above, which controls an articulated arm.
(15) A multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a strabismus at its distal end is controlled to change the position and posture of the strabismus. A surgical arm control system for controlling
A surgical arm control system for controlling, within a field of view imaged through the oblique scope, at least one of a speed of rotation or a speed of movement of the oblique scope based on a position of an object to be observed within the field of view.

REFERENCE SIGNS LIST 100 oblique mirror 200 field of view 210 observation object 350 control device 351 control unit 351a distance acquisition unit 351b speed calculation unit 400 support arm device

Claims (15)

a multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end;
a control system for controlling the articulated arm to change the position and orientation of the oblique scope;
with
The control system is
controlling at least one of a rotational speed or a moving speed of the oblique mirror based on a position of an object to be observed within the visual field imaged through the oblique mirror;
The surgical arm system , wherein the longer the distance from the center of the field of view to the position of the observation object, the slower the rotation speed . a multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end;
a control system for controlling the articulated arm to change the position and orientation of the oblique scope;
with
The control system is
controlling at least one of a rotational speed or a moving speed of the oblique mirror based on a position of an object to be observed within the visual field imaged through the oblique mirror;
The surgical arm system, wherein the rotation speed is set to 0 when the observation object is positioned at the edge of the field of view. 3. The surgical arm system of claim 1 or 2 , wherein the control system controls the rotation angle and speed of rotation about the longitudinal axis of the oblique scope. 4. The surgical arm system according to any one of claims 1 to 3 , wherein said control system controls the attitude and movement speed of said articulated arm so as to move the position of said observation object within said field of view. 5. The control system of claims 1 to 4, wherein the control system controls at least one of rotation speed and movement speed of the oblique mirror based on the distance from the center of the field of view to the position of the observation object within the field of view. A surgical arm system according to any one of the preceding paragraphs . 2. From claim 1, wherein the control system controls the articulated arm that supports the squintoscope so that the observation object is positioned at the center of the field of view based on the result of determining the position of the observation object. 6. The surgical arm system according to any one of 5 . 7. The control system according to claim 6, wherein the control system controls the articulated arm that supports the squintoscope so that the observation object is positioned at the center of the field of view, based on a result of image recognition of the observation object. surgical arm system. 2. The surgical arm system according to claim 1 , wherein said control system controls said rotational speed such that said rotational speed decreases with respect to said distance increments in a linear relationship. 2. The surgical arm system according to claim 1 , wherein when said distance is within a predetermined range, said control system makes said rotational speed the same as said rotational speed when said center coincides with said position of said observation object. . 2. The surgical arm system according to claim 1 , wherein said control system keeps said rotational speed constant when said distance exceeds a predetermined range. The surgical arm system according to any one of claims 1 to 10 , wherein said control system comprises a first controller for controlling rotation of said oblique scope and a second controller for controlling said articulated arm. . 12. The surgical arm system according to claim 11 , wherein the articulated arm includes a holding unit that holds the oblique scope and camera head independently and rotatably. The holding unit includes the first control section,
The second control unit acquires the rotation angle information of the squint mirror acquired by the first control unit via an image processing unit connected to the camera head, and based on the rotation angle information, 13. The surgical arm system of claim 12 , which controls an articulated arm. A multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end is controlled, and the multi-joint arm is controlled to change the position and posture of the squintoscope. A surgical arm control system comprising:
Within the field of view imaged through the squinting scope, at least one of the rotational speed and the moving speed of the squinting scope is controlled based on the position of the observation object within the field of view, and the observation object is moved from the center of the field of view. A surgical arm control system , wherein the longer the distance to the position, the slower the rotational speed . A multi-joint arm having a plurality of joints rotatably connected by a plurality of links and capable of supporting a squintoscope at its distal end is controlled, and the multi-joint arm is controlled to change the position and posture of the squintoscope. A surgical arm control system comprising:
controlling at least one of a rotational speed and a moving speed of the oblique mirror based on a position of an observation object within the field of view in a field of view imaged through the oblique scope, and the observation object being positioned at an edge of the field of view; A surgical arm control system that, if positioned, zeroes the rotational speed.

Images