JP2017038285A - Medical treatment observation device, controller, and operation method and operation program of controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a medical treatment observation device, a controller, and an operation method and an operation program of the controller, which allow a user to easily and accurately grasp a position of a medical device even if the medical device is out of a visual field area of the medical treatment observation device with a simple structure.SOLUTION: A medical treatment observation device comprises: a first imaging section 21 for imaging a first visual field area; a second imaging section 22 for imaging a second visual field area which includes a first visual field area and is wider than the first visual field area by magnification lower than that of the first imaging section 21; a detection section 311 for detecting whether a medical device appears in a first image which the first imaging section 21 images; and a display control section 341 which performs one of a first display control which outputs a signal of the first image to a display device 4 in accordance with a detection result of the detection section 311 to display it on the display device 4 and a second display control which outputs a signal of a second image that at least the second imaging section 22 images to the display device 4 to display the second image on the display device 4.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a medical observation apparatus, a control apparatus, an operation method for a control apparatus, and an operation program for the control apparatus for observing a minute part of an object to be observed.

  2. Description of the Related Art Conventionally, for example, when performing surgery on a patient, a medical observation apparatus such as a surgical microscope is known in which a surgeon such as a doctor observes an enlarged image of an affected area of the patient. The surgical microscope has an enlarged image of the affected area displayed on the eyepiece, and the surgeon observes the affected area through the eyepiece, and an enlarged captured image displayed on the monitor by the surgeon. It is divided roughly into the type to observe. When the surgeon uses a surgical microscope, the affected part is magnified by directing his / her line of sight toward the eyepiece or monitor.

  In a surgical operation using a surgical microscope, when an operator introduces a medical instrument such as forceps (also referred to as a surgical instrument) into a visual field region (hereinafter simply referred to as a visual field region) of the surgical microscope, Once you move your eyes away from the monitor The medical instrument is introduced into the visual field region by performing observation.

  In general, the higher the zoom magnification of the surgical microscope, the more difficult it is for the operator to visually determine the visual field region on the object to be observed. In such a case, the surgeon moves the line of sight and repeats the indirect observation of the object to be observed through the eyepiece or the monitor and the direct observation of the object to be observed by the eye many times. Had to be introduced into the field of view. For example, when the operator first introduces a medical instrument into the visual field when starting an operation, or when the operator unintentionally moves the medical instrument out of the visual field during the operation. May occur. In the latter case, the surgeon must move the line of sight over and over again to reintroduce the medical instrument into the field of view, even during the operation.

  As a technique for easily introducing a medical instrument into the field of view of a surgical microscope, the medical instrument is included in the field of view by changing the zoom magnification of the zoom optical system of the imaging unit when the medical instrument is detected. A technique for doing so is disclosed (for example, see Patent Document 1).

JP 2004-117596 A

  However, in the technique described in Patent Document 1, a range that is wider than a variable range of zoom magnification that secures a field area of magnification observation in a normal surgical microscope for magnifying an object to be observed such as an affected area, that is, a medical instrument It is necessary to change the zoom magnification to a wide-angle visual field range that is easy to introduce. Here, if the variable range of the zoom magnification on the wide angle side is small and the visual field range on the wide angle side is narrow, the visual field region for introducing the medical device becomes narrow, and it is difficult to assist the introduction of the medical device. Also, if the variable range of zoom magnification on the wide-angle side is large and the viewing range on the wide-angle side is wide, it is easy to introduce medical instruments, but the configuration of the zoom optical system and the configuration for driving the zoom optical system must be complicated. I don't get it.

  Furthermore, with the technique described in Patent Document 1, when changing the zoom magnification, a time lag occurs between the operation of changing the magnification and the movement of the medical device, and the position of the medical device cannot be accurately grasped. There is.

  The present invention has been made in view of the above, and the user can easily and accurately grasp the position of the medical instrument even when the medical instrument deviates from the field of view of the medical observation apparatus with a simple configuration. It is an object of the present invention to provide a medical observation apparatus, a control apparatus, an operation method for the control apparatus, and an operation program for the control apparatus.

  In order to solve the above-described problems and achieve the object, a medical observation apparatus according to the present invention includes a first imaging unit that images a first visual field region, and the first visual field region. A second imaging unit that captures a wider second field-of-view region at a lower magnification than the first imaging unit, and detection that detects whether or not a medical instrument is included in the first image captured by the first imaging unit A first display control for outputting a signal of the first image to a display device and displaying the first image on the display device according to a detection result of the detection unit, and at least the second imaging unit. A display control unit that executes any one of a second display control that outputs a signal of the captured second image to the display device and causes the display device to display the second image. To do.

  In the medical observation apparatus according to the present invention, in the above invention, the display control unit executes the first display control when the detection unit detects that the medical instrument is reflected in the first image. And when the said detection part detects that the said medical device is not reflected in the said 1st image, said 2nd display control is performed.

  In the medical observation device according to the present invention, in the above invention, the display control unit displays the second image as a main screen on the display device as the second display control, and the first image is displayed as the main image. It is characterized in that it is displayed on the display device as a sub-screen displayed in an area smaller than the screen.

  In the medical observation device according to the present invention, in the above invention, when the display control unit displays the second image as the second display control, the information related to the first visual field region is superimposed on the second image. And displaying on the display device.

  The medical observation apparatus according to the present invention can set the first image display mode in which the first image continues to be displayed in the above invention, and the display control unit is set to the first image display mode. In this case, the first display control is executed.

  The medical observation apparatus according to the present invention further includes an input unit that is provided in an operation unit that receives an instruction to move the first imaging unit by a user and that receives a setting input of the first image display mode. It is characterized by.

  In the medical observation apparatus according to the present invention, in the above invention, the detection unit detects whether or not the medical instrument is reflected in the first image by performing image processing on the first image. It is characterized by.

  The control device according to the present invention includes a first imaging unit that captures an image of the first visual field area, and a second visual field area that includes the first visual field area and is wider than the first visual field area. A control unit for controlling an image acquisition device including a second image pickup unit that picks up an image with a detection unit that detects whether or not a medical instrument is reflected in the first image picked up by the first image pickup unit And a first display control for outputting a signal of the first image to a display device and displaying the first image on the display device according to a detection result of the detection unit, and at least the second imaging unit picks up an image. And a display control unit for switching and executing a second display control for outputting the second image signal to the display device and displaying the second image on the display device.

  The operation method of the control device according to the present invention includes: a first imaging unit that images a first visual field region; and a second visual field region that includes the first visual field region and is wider than the first visual field region from the first imaging unit. And a second imaging unit that captures images at a low magnification, and a method for operating the control device that controls the image acquisition device, wherein the detection unit includes a medical instrument in the first image captured by the first imaging unit. A detection step for detecting whether or not the image is captured, and a display control unit that outputs a signal of the first image to a display device and displays the first image on the display device according to a detection result of the detection unit; One of the first display control to be performed and the second display control to output at least the signal of the second image captured by the second imaging unit to the display device and display the second image on the display device. And a display control step to be executed.

  An operation program of a control device according to the present invention includes a first imaging unit that images a first visual field region, and a second visual field region that includes the first visual field region and is wider than the first visual field region, from the first imaging unit. Whether or not the control device that controls the image acquisition device including the second imaging unit that captures images at a lower magnification includes a medical instrument in the first image captured by the first imaging unit. And a display control unit that outputs a signal of the first image to a display device and displays the first image on the display device in accordance with a detection result of the detection unit. And at least one of a second display control for outputting a signal of a second image captured by the second imaging unit to the display device and displaying the second image on the display device. Are executed.

  ADVANTAGE OF THE INVENTION According to this invention, even if it is a case where a medical instrument remove | deviates from the visual field area | region of a medical observation apparatus by simple structure, a user can be made to grasp | ascertain the position of a medical instrument easily and correctly.

FIG. 1 is a diagram schematically showing an overall configuration of a surgical microscope system which is a medical observation system including a medical observation apparatus according to an embodiment of the present invention. FIG. 2 is an enlarged perspective view showing a configuration of a microscope unit and its surroundings included in the microscope apparatus. FIG. 3 is a block diagram showing a functional configuration of a surgical microscope system which is a medical observation system according to an embodiment of the present invention. FIG. 4 is a diagram schematically illustrating configurations of a first imaging unit and a second imaging unit included in the microscope apparatus. FIG. 5 is a diagram schematically showing a state of surgery using a surgical microscope system which is a medical observation system according to an embodiment of the present invention. FIG. 6 is a flowchart showing an outline of processing performed by the control device according to the embodiment of the present invention. FIG. 7 is a diagram schematically illustrating another display example (first example) when the display device displays a low-magnification image. FIG. 8 is a diagram schematically illustrating another display example (second example) when the display device displays a low-magnification image. FIG. 9 is a diagram schematically illustrating another display example (third example) when the display device displays a low-magnification image. FIG. 10 is a diagram illustrating another configuration example of the first and second imaging units. FIG. 11 is a side view of the second imaging unit in the direction of arrow A in FIG. FIG. 12 is a diagram illustrating still another configuration example of the second imaging unit. FIG. 13 is a side view of the first and second imaging units in the direction of arrow B in FIG. FIG. 14 is a diagram schematically showing an overall configuration of an endoscope system that is a medical observation system provided with a medical observation apparatus according to another embodiment of the present invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described with reference to the accompanying drawings.

  FIG. 1 is a diagram schematically showing an overall configuration of a surgical microscope system which is an example of a medical observation system including a medical observation apparatus according to an embodiment of the present invention. The surgical microscope system 1 shown in FIG. 1 has a function of enlarging and capturing a predetermined visual field region and displaying the captured image. The surgical microscope system 1 includes a microscope device 2 that is an image acquisition device that acquires an image for observing an object to be observed, a control device 3 that controls the operation of the surgical microscope system 1, and a microscope device. 2 includes a display device 4 that displays the captured image. The microscope apparatus 2 and the control apparatus 3 are surgical microscopes and constitute the medical observation apparatus 100 according to the present embodiment. Note that at least one of the control device 3 and the display device 4 can be configured integrally with the microscope device 2.

  The microscope apparatus 2 is connected to a microscope unit 5 that magnifies and images a minute part of the object to be observed, and a proximal end part of the microscope unit 5, and the microscope unit is movably supported with respect to the object to be observed. And a base portion 7 that rotatably supports the base end portion of the support portion 6 and is movable on the floor surface. In addition, the base part 7 is good also as a structure which supports the support part 6 by fixing to a ceiling, a wall surface, a bed, etc. instead of providing on a floor surface so that a movement is possible.

  FIG. 2 is an enlarged perspective view showing the configuration of the microscope unit 5 and its periphery. The microscope unit 5 has a cylindrical shape, and has an imaging unit therein. The detailed configuration of the imaging means will be described later. In order to image a desired imaging range, the user holds the microscope unit 5 and moves the microscope unit 5. On the side surface of the microscope unit 5, various switches constituting an input unit 24 that receives an input of an operation instruction for the microscope apparatus 2 are provided. A cover glass that protects the inside is provided on the opening surface of the lower end of the microscope unit 5 (not shown). Note that the switches shown in FIG. 2 only describe some of the various switches and buttons constituting the input unit 24.

  A user such as an operator moves the microscope unit 5 or performs a zoom operation while operating various switches while holding the microscope unit 5. In other words, the microscope unit 5 is an operation unit that receives a movement instruction from a user of the microscope unit 5 including the first imaging unit 21 described later and is provided with the input unit 24. The shape of the microscope unit 5 is preferably a shape that is elongated in the observation direction so that the user can easily grasp and change the viewing direction. For this reason, the shape of the microscope unit 5 may be a shape other than a cylindrical shape, for example, a polygonal column shape. In the present embodiment, the microscope unit 5 also serves as an operation unit. However, the present invention is not limited to this, and an operation unit that receives a movement instruction of the microscope unit 5 and is provided with various switches is used, for example, for a remote controller or a foot switch. Such a position may be provided at a position separated from the microscope unit 5.

  The support unit 6 includes a plurality of joint units and arm units that are alternately connected, and realizes translation or rotation of the microscope unit 5. In FIG. 2, the first joint portion 61, the first arm portion 71, the second joint portion 62, and the second arm portion 72 are connected in order from the distal end side.

The first joint portion 61 holds the microscope portion 5 so as to be rotatable around a first axis A ₁ that coincides with the optical axis of the microscope portion 5 on the distal end side, and also has a first arm portion 71 on the proximal end side. The first arm portion 71 is held in a state of being fixed to the distal end portion. The second joint portion 62 holds the first arm portion 71 on the distal end side so as to be rotatable around the second axis A ₂ orthogonal to the first axis A _1, and is attached to the second arm portion 72 on the proximal end side. Retained.

  FIG. 2 shows the configuration of at least some of the joints and arms of the support 6. For example, when the support part 6 has a total of 6 joint parts and 5 arm parts, the support part 6 can implement | achieve the motion of 6 degrees of freedom of 3 degrees of freedom of translation and 3 degrees of freedom of rotation.

  FIG. 3 is a block diagram showing a functional configuration of the surgical microscope system 1. The functional configuration of the surgical microscope system 1 will be described below with reference to FIG.

  First, the functional configuration of the microscope apparatus 2 will be described. The microscope apparatus 2 includes a first imaging unit 21, a second imaging unit 22, a lens driving unit 23, an input unit 24, a signal processing unit 25, a brake unit 26, and a control unit 27.

FIG. 4 is a diagram schematically illustrating the configuration of the first imaging unit 21 and the second imaging unit 22 provided in the microscope unit 5.
The first imaging unit 21 is an imaging unit for magnifying and observing an object to be observed with high image quality. The first imaging unit 21 includes a variable magnification optical system 211 having a zoom function, a first imaging optical system 212 that forms an image of light that has passed through the variable magnification optical system 211, and an image formed by the first imaging optical system 212. And a first image sensor 213 that converts an image of the observed object into an electrical image signal.

  The variable magnification optical system 211 and the first imaging optical system 212 are each configured by using one or a plurality of lenses. The lenses constituting the variable magnification optical system 211 are driven by the lens driving unit 23. The magnifications on the narrowest angle side and the widest angle side of the object image to be observed that are scaled by the zooming optical system 211 and imaged by the first image sensor 213 are set in advance.

  The first image sensor 213 is configured using, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The first image sensor 213 is used for observing the object to be observed, and has an effective number of pixels of about 8 megapixels (for example, 3840 × 2160 pixels) in order to capture an image with high image quality, in other words, a resolution higher than the so-called 4K resolution. It is preferable to have an effective number of pixels.

  The imaging signal generated by the first imaging unit 21 having the above configuration is transmitted to the control device 3 via a transmission cable. Note that the image magnification change is not limited to the variable magnification optical system 211, and may be performed by electronic zoom based on image signal processing.

  The second imaging unit 22 is an imaging unit for easily introducing a medical instrument into the visual field range of the first imaging unit 21. The second imaging unit 22 electrically collects an image of the object to be observed formed by the second image forming optical system 221 and a second image forming optical system 221 that focuses and collects light from the object to be observed. And a second image sensor 222 that outputs the image signal instead of the image signal.

In the present embodiment, the optical axis O ₂₂ of the second imaging unit 22 is inclined with respect to the optical axis O ₂₁ of the first imaging unit 21. The second imaging unit 22 is located in the vicinity of the first imaging unit 21 and includes a visual field region (first visual field region) of the first imaging unit 21 and a wider visual field region (second visual field region) than the first visual field region. Have The second imaging unit 22 captures an image (low-magnification image) having a fixed magnification lower than the zoom magnification that can be set by the first imaging unit 21. In other words, the magnification of the observed object image formed on the second image sensor 222 by the second imaging optical system 221 is lower than the predetermined magnification on the wide-angle side that can be set by the first imaging unit 21. Is set.

  Similarly to the first image sensor 213, the second image sensor 222 is configured using, for example, a CCD or a CMOS. The second image sensor 222 may have the same number of effective pixels as the first image sensor 213. For example, in order to easily introduce a medical instrument into the field of view of the first image sensor 21, The number of effective pixels may be about 2 megapixels (for example, 1920 × 1080 pixels) that is less than or equal to the number of effective pixels, in other words, the number of effective pixels may be less than the so-called HD resolution. I do not care.

  In the present embodiment, the second imaging unit 22 is provided in the microscope unit 5. However, the present invention is not limited thereto, and includes the visual field region (first visual field region) of the first imaging unit 21. The second imaging unit 22 may be provided in addition to the microscope unit 5 as long as it is a position having a visual field region (second visual field region) wider than the region. Specifically, for example, the second imaging unit 22 can be provided at the support unit 6, the base unit 7, or other installation locations dedicated to the second imaging unit 22.

  The imaging signal generated by the second imaging unit 22 having the above configuration is transmitted to the control device 3 via a transmission cable. Note that the second imaging unit 22 may have a zoom function as with the first imaging unit 21. However, the second imaging unit 22 is a configuration for facilitating detection of a medical instrument by the user, and does not generate a high-quality image for observing the affected part during surgery. In this sense, it is more preferable that the second imaging unit 22 has a simple configuration that does not have a zoom function.

  The first imaging unit 21 and the second imaging unit 22 always capture images in parallel. The surgical microscope system 1 detects a first image display mode (high magnification display mode) that continues to display the first image (high magnification image) captured by the first imaging unit 21 and the presence or absence of a medical instrument in the high magnification image. Then, it is possible to set a switching display mode in which a high-magnification image and a second image (low-magnification image) captured by the second imaging unit 22 are switched according to the detection result.

  In the high magnification display mode, for example, before the user starts an operation, the microscope unit 5 is moved to display the vicinity of a desired affected part on the display device 4, and after the rough alignment of the microscope part 5 is performed, the affected part is displayed. Is set when fine-tuning to a position that is easier to see. In the present embodiment, the setting of the high magnification display mode is performed by the user pressing the high magnification display button of the input unit 24 and inputting a setting signal.

  The switching display mode is a mode set, for example, before the user starts an operation or during an operation. In the present embodiment, the switching display mode is set when the high magnification display button of the input unit 24 is not pressed.

  The lens driving unit 23 includes an actuator that drives a lens constituting the variable magnification optical system 211 of the first imaging unit 21 under the control of the control unit 27.

  As described with reference to FIG. 2, the input unit 24 is provided on the side surface of the microscope unit 5, at least a part of which is an operation unit. The input unit 24 includes a high-magnification display button for receiving an instruction input for causing the display device 4 to display in the high-magnification display mode, a zoom switch for receiving an instruction input for a zoom operation, and the like. The surgical microscope system 1 is set to the high magnification display mode while the high magnification display button is pressed, and is set to the switching display mode when the high magnification display button is not pressed. It is desirable that the high-magnification display button and the zoom switch are provided on the side surface of the microscope unit 5 as an operation unit so that the user can reach the finger while holding the microscope unit 5. In addition to the side surface of the microscope unit 5, a foot switch that can be operated by a user's foot may be provided as the input unit 24. The foot switch may be provided with a zoom switch for changing the zoom magnification in the high-magnification image.

  The signal processing unit 25 performs predetermined signal processing on the images captured by the first imaging unit 21 and the second imaging unit 22 and outputs the images.

  The brake part 26 has a plurality of electromagnetic brakes provided for each joint part of the support part 6. The electromagnetic brake is released under the control of the control unit 27 in response to a release instruction signal that the input unit 24 accepts input. When the electromagnetic brake is released, the microscope unit 5 and each arm unit can be rotated with respect to the corresponding joint unit. A state in which the microscope unit 5 and all the arm units are rotatable with respect to the corresponding joint units is referred to as all-arm free. Note that an air brake may be applied as the brake unit 26 instead of the electromagnetic brake.

  The control unit 27 can communicate with the control unit 34 of the control device 3 and controls the operation of the microscope apparatus 2 in cooperation with the control unit 34. The control unit 27 is realized using a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated integrated circuit that performs a specific function such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The Note that the signal processing unit 25 and the control unit 27 may be provided in any place of the microscope unit 5, the support unit 6, and the base unit 7, or may be provided in the control device 3.

  Next, the functional configuration of the control device 3 will be described. The control device 3 is necessary for the operation of the image processing unit 31 that performs predetermined image processing on the imaging signal transmitted from the microscope device 2, the input unit 32 that receives input of various instruction signals, and the operation microscope system 1. It has the memory | storage part 33 which memorize | stores the various information containing information, and the control part 34 which controls the microscope system 1 for operation collectively.

  The image processing unit 31 includes a detection unit 311 that detects the presence of a medical instrument such as forceps shown in the high-magnification image sent from the microscope apparatus 2. The detection unit 311 detects a color corresponding to the medical instrument by performing image processing on the high-magnification image. Specifically, the color corresponding to the medical device here is silver. Further, for example, a specific color marker may be provided in advance at the distal end of the medical instrument, and the detection unit 311 may detect the color marker to detect the presence of the medical instrument in the visual field region. Note that the detection unit 311 may detect a medical instrument by performing pattern matching. When pattern matching is applied, the medical device template may be stored in the storage unit 33 in advance. The image processing unit 31 is realized using a general-purpose processor such as a CPU or an integrated circuit such as an ASIC or FPGA.

  The storage unit 33 is configured by using a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and various programs for operating the surgical microscope system 1 and various programs necessary for the operation of the surgical microscope system 1. Stores parameters and the like. When the detection unit 311 detects the presence / absence of a medical device in the high-magnification image by performing pattern matching, the storage unit 33 stores a template of the medical device.

  The various programs include an operation program for executing the operation method of the control device 3. Various programs can be widely distributed by being recorded on a computer-readable recording medium such as a hard disk, a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk. Various programs can also be obtained by downloading via a communication network. The communication network here is realized by, for example, an existing public line network, a LAN (Local Area Network), a WAN (Wide Area Network) or the like, and may be wired or wireless.

  The control unit 34 can communicate with the control unit 27 of the microscope apparatus 2 and controls the entire surgical microscope system 1 including the microscope apparatus 2 in cooperation with the control unit 27. The control unit 34 includes a display control unit 341 that controls display of images and the like performed by the display device 4. The display control unit 341 outputs a signal of a high-magnification image captured by the first imaging unit 21 to the display device 4 according to the detection result of the detection unit 311 and causes the display device 4 to display the high-magnification image. One of display control and second display control for outputting at least the low-magnification image signal captured by the second imaging unit 22 to the display device 4 and displaying the low-magnification image on the display device 4 is executed. In the present embodiment, the display control unit 341 is provided in the control unit 34 of the control device 3. However, the display control unit 341 is not limited thereto, and may be provided in the microscope device 2 such as the control unit 27.

  The control unit 34 is configured using a general-purpose processor such as a CPU or an integrated circuit such as an ASIC or FPGA. At least a part of the control unit 34 and at least a part of the image processing unit 31 may be configured using a common CPU, ASIC, or FPGA.

  The control device 3 having the above configuration is configured using one or a plurality of computers. When the control device 3 is configured using a plurality of computers, the plurality of computers may be communicably connected via a communication network.

  Data transmission between the microscope apparatus 2 and the control apparatus 3 may be performed by an electrical signal or an optical signal. Further, a signal having a large amount of data such as image data may be transmitted as an optical signal, and other signals may be transmitted as electric signals.

  Next, the functional configuration of the display device 4 will be described. The display device 4 receives the moving image data generated by the control device 3 from the control device 3 and displays an image corresponding to the moving image data. Such a display device 4 has a display panel made of liquid crystal or organic EL (Electro Luminescence). The display device 4 preferably has a display unit that displays an image of 55 inches or more in order to make it easy to obtain an immersive feeling during observation, but is not limited thereto.

  An outline of surgery performed using the surgical microscope system 1 having the above functional configuration will be described. First, an outline of an operation performed before the user starts an operation will be described. The user sets the operation mode to the switching display mode without pressing the high magnification display button of the input unit 24, grasps the microscope unit 5, visually observes the image displayed on the display device 4, and operates the input unit 24. Then, by inputting the release instruction signal, the operation of the brake unit 26 is released and the microscope unit 5 is moved, and the visual field range is adjusted to the target affected part. At this stage, there is no medical instrument in the vicinity of the affected part, and no medical instrument is detected within the field of view of the high-magnification image captured by the first imaging unit 21, so a low-magnification image is displayed on the display device 4.

  Thereafter, when the vicinity of the desired affected area is displayed on the display device 4, the user switches to the high magnification display mode by pressing a high magnification display button provided on the holding microscope unit 5. Thereby, in the display device 4, a high-magnification image captured by the first imaging unit 21 is displayed on the display device 4. In the state of this high-magnification image, the user finely adjusts the position of the microscope unit 5 so that the vicinity of the desired affected part is displayed on the display device 4, and operates the input unit 24 as necessary to operate the first imaging unit 21. The zoom is adjusted so that an appropriate observation image can be obtained. Thereafter, the input of the release instruction signal via the input unit 24 is terminated, the brake unit 26 is operated, and the position of the microscope unit 5 is fixed.

  As described above, in the surgical microscope system 1, a part that receives an instruction to move the microscope unit 5 from the user or an input operated by the user until the position of the microscope unit 5 is fixed at a position where an appropriate observation image is obtained. The unit 24 is collectively provided in one microscope unit 5 (operation unit). For this reason, various operations relating to the position adjustment of the microscope unit 5 can be easily performed.

  FIG. 5 is a diagram schematically showing a situation at the time of surgery using the surgical microscope system 1. Specifically, FIG. 5 is a diagram schematically illustrating a situation in which an operator 401 as a user is operating on the head of a patient 402 as an object to be observed. The operator 401 sets the switching display mode without pressing the high magnification display button of the input unit 24 in a state where the position of the microscope unit 5 is fixed, and performs an operation while viewing the display device 4. In a state where no medical instrument is detected within the field of view of the high-magnification image captured by the first imaging unit 21, a wide range of low-magnification images captured by the second imaging unit 22 are displayed on the display device 4. When the medical instrument is brought close to the affected part in this state, the medical instrument is detected within the field of view of the high-magnification image captured by the first imaging unit 21, and the high-magnification image is displayed on the display device 4.

  As described above, in the surgical microscope system 1, the operator 401 can guide the medical instrument into the field of view of the high-magnification image without moving the line of sight while keeping the line of sight toward the display device 4. In addition, it is assumed that the surgeon 401 is holding the medical instrument with both hands during the operation. However, when the medical instrument is out of the field of view of the high-magnification image during the surgery, the surgeon 401 Therefore, the operator 401 does not need to change the zoom magnification to the wide-angle side, and the operator 401 holds the medical instrument in both hands and moves the medical instrument away from the vicinity of the affected area without moving the medical instrument away from the vicinity. It can be guided within the field of view of the magnification image.

  Hereinafter, an outline of processing performed by the control device 3 will be described with reference to a flowchart shown in FIG. When the surgical microscope system 1 is set to the switching display mode (step S1: Yes), the detection unit 311 detects whether or not a medical instrument is reflected in the high-magnification image captured by the first imaging unit 21. Processing is performed (step S2).

  As a result of detection by the detection unit 311, when the medical instrument is reflected in the high-magnification image (step S <b> 3: Yes), the display control unit 341 displays the high-magnification image captured by the first imaging unit 21 on the display device 4. First display control is performed (step S4). On the other hand, as a result of detection by the detection unit 311, when the medical instrument is not shown in the high-magnification image (step S <b> 3: No), the display control unit 341 displays the low-magnification image captured by the second imaging unit 22 on the display device 4. Second display control to be displayed is performed (step S5). For example, in a state where the visual field range is adjusted to the target affected part before the user starts the operation, the medical instrument is not shown in the high-magnification image. Accordingly, in such a case, the display control unit 341 performs the second display control.

  After step S4 or S5, when the input unit 24 receives an input of an end instruction signal (step S6: Yes), the surgical microscope system 1 ends a series of operations. On the other hand, after step S4 or S5, when the input unit 24 does not accept the input of the end instruction signal (step S6: No), the surgical microscope system 1 returns to step S1.

  In step S1, when the high magnification display button of the input unit 24 is pressed and the surgical microscope system 1 is set to the high magnification display mode (step S1: No), the surgical microscope system 1 proceeds to step S4. The display control unit 341 performs the first display control. As a case where such first display control is performed, for example, a case where the user finely adjusts the position of the microscope unit 5 to a position where the affected part can be easily seen while pressing the high magnification display button before starting the operation. be able to.

  According to the embodiment of the present invention described above, the first imaging unit 21 and the second imaging unit 22 capture an image in parallel, and the presence of a medical instrument is detected by a high-magnification image by the first imaging unit 21. In this case, a high-magnification image is displayed on the display device 4, while a low-magnification image by the second imaging unit 22 is displayed on the display device 4 when the presence of a medical instrument is not detected by the high-magnification image. It is possible to switch between displaying a high-magnification image and a low-magnification image without causing a time lag by simply adding. Therefore, according to the present embodiment, the user can easily and accurately grasp the position of the medical instrument even when the medical instrument is out of the first visual field region of the medical observation apparatus 100 with a simple configuration. be able to. As a result, the user can easily introduce the medical device into the first visual field region.

  In the present embodiment, when the switching display mode is set, the display of the high-magnification image and the low-magnification image is switched according to the presence or absence of detection of the medical instrument in the high-magnification image, but the low-magnification image is displayed. At this time, the information including the information of the high-magnification image may be displayed. Hereinafter, such display examples will be described. FIG. 7 is a diagram schematically illustrating another display example (first example) when the display device 4 displays a low-magnification image. In the case shown in FIG. 7, the low-magnification image 501 is displayed as the main screen, while the high-magnification image 502 is reduced and superimposed on the lower right area of the low-magnification image 501 as a sub-screen. Note that the display device 4 may display the low-magnification image 501 and the high-magnification image 502 side by side in the left-right or top-bottom relationship as shown in FIG.

  FIG. 8 is a diagram schematically illustrating another display example (second example) when the display device 4 displays a low-magnification image. In the case illustrated in FIG. 8, the visual field center P of the first visual field region is displayed in the low-magnification image 503 as information regarding the first visual field region.

  FIG. 9 is a diagram schematically illustrating another display example (third example) when the display device 4 displays a low-magnification image. In the case shown in FIG. 9, the outline F of the first visual field region is superimposed and displayed as information about the first visual field region in the low-magnification image 504.

  According to the display examples (first to third examples) of the display device 4 described above, the information of the high magnification image can be obtained while the low magnification image is being displayed. There is no need to feel uncomfortable even when switching to a magnification image.

  In addition, instead of the first imaging unit 21 and the second imaging unit 22 capturing images in parallel, for example, the operation of the second imaging unit 22 may be turned on and off so as not to always capture images. In this case, a threshold region to be detected is set inside the outermost contour of the high-magnification image by the first imaging unit 21, and the imaging operation of the second imaging unit 22 is stopped when a medical instrument is detected within this threshold region. On the other hand, when no medical instrument is detected within this threshold region, the imaging operation of the second imaging unit 22 may be executed. Thereby, when the medical instrument is included in the high-magnification image, the second imaging unit 22 is not imaged, and only when the medical instrument is likely to be detached from the high-magnification image or only when the medical instrument is off. 22 images can be driven, and more efficient drive control can be performed.

(Other embodiments)
So far, the embodiment for carrying out the present invention has been described, but the present invention should not be limited only by the embodiment described above. For example, the configurations of the first imaging unit and the second imaging unit are not limited to those described above. FIG. 10 is a diagram illustrating another configuration example of the first imaging unit and the second imaging unit. In FIG. 10, the first imaging unit 21LR includes a stereo optical system that can generate two images that have a parallax and a part of each viewing area overlaps, and generates a three-dimensional image. In addition, the second imaging unit 22A has a wider field of view than the first imaging unit 21LR and generates an image with a lower magnification than the first imaging unit 21LR.

In the first imaging unit 21LR, the variable magnification optical system 211L and the first imaging optical system 212L, the variable magnification optical system 211R and the first imaging optical system 212R form a pair, and the optical axes O _1L are parallel to each other. _, O _1R is formed. The lenses constituting the variable magnification optical systems 211L and 211R are driven by the lens driving unit 23. The observed object images formed by the first imaging optical systems 212L and 212R are converted into electrical signals on different light receiving surfaces of the first imaging element 213LR, and output as two image signals having parallax. . An optical system 214 that includes a concave lens and a convex lens on the preceding stage of the variable magnification optical systems 211L and 211R, that is, the observation object side, and has a function of expanding the range of the distance to the observation object that can generate two images having parallax. Is provided.

FIG. 11 is a diagram schematically illustrating the configuration of the second imaging unit 22A, and is a side view of the second imaging unit 22A in the direction of arrow A in FIG. The second imaging unit 22A includes a second imaging optical system 221A, a second imaging element 222A, and a mirror 223A. The mirror 223A is on the optical axis O _{2A of} the second imaging optical system 221A and the optical system 214 of the first imaging unit 21LR, and between the second imaging optical system 221A and the optical system 214 of the first imaging unit 21LR. It is provided between the stereo optical systems that form a pair of the first imaging unit 21LR, and reflects the light that has passed through the optical system 214 and guides it to the second imaging optical system 221A.

  Also in the case of the imaging unit having the configuration shown in FIGS. 10 and 11, the same effect as that of the above-described embodiment can be obtained. It should be noted that a known configuration other than that shown in FIG. 10 can be adopted as the configuration of the stereo optical system. If the space for installing the mirror 223A cannot be secured between the stereo optical systems, the second imaging unit can be configured using an image fiber or the like.

  FIG. 12 is a diagram illustrating still another configuration example of the second imaging unit. FIG. 13 is a side view of the first and second imaging units in the direction of arrow B in FIG. 12 and 13, the configuration of the first imaging unit 21LR is the same as that in FIGS. 10 and 11.

  The second imaging unit 22LR is provided between the variable magnification optical system 211R and the optical system 214, and includes a stereo optical system that can generate two images having parallax, and generates a three-dimensional image. The second imaging unit 22LR transmits the light from the optical system 214 to the variable magnification optical system 211R and reflects the light reflected by the prism 223R in the second direction, and reflects the light reflected by the prism 223R. And an imaging optical system 221R. The second imaging unit 22LR is provided between the variable magnification optical system 211L and the optical system 214, and transmits light from the optical system 214 to the variable magnification optical system 211L and reflects it in a direction orthogonal to the transmitted light. And a second imaging optical system 221L that forms an image of the light reflected by the prism 223L. Although the second imaging optical system 221L is not shown, it is needless to say that the second imaging optical system 221L has the same configuration as the second imaging optical system 221R. The second imaging unit 22LR receives the object images formed by the second imaging optical systems 221L and 221R on different light receiving surfaces, converts them into electrical signals, and outputs them as two image signals having parallax. A second imaging element 222LR is further included.

  The prisms 223L and 223R preferably have a function of performing spectroscopy such that the amount of light transmitted to the first imaging unit 21LR is greater than the amount of light reflected to the second imaging unit 22LR. In this case, the brightness of the image captured by the second imaging unit 22LR is reduced by making the number of pixels used for image generation by the second imaging element 222LR smaller than the number of pixels used for image generation by the first imaging element 213LR. An appropriate range can be set.

  In the configuration shown in FIGS. 12 and 13, two half mirrors may be applied instead of the two prisms.

  FIG. 14 is a diagram schematically showing an overall configuration of an endoscope system that is a medical observation system provided with a medical observation apparatus according to another embodiment of the present invention. An endoscope system 81 shown in the figure includes an endoscope apparatus 82 that is an image acquisition apparatus that acquires an in-vivo image of an object to be observed by inserting a tip portion into the object to be observed. The endoscope device 82 is connected to the in-vivo image captured by the endoscope device 82 to perform predetermined image processing to generate image data for display, and supervises the entire operation of the endoscope system 81. A control device 83 for controlling, a light source device 84 for generating illumination light to be irradiated to the object to be observed from the distal end of the endoscope device 82, and a display device 85 for displaying an image or the like captured by the endoscope device 82. Prepare. The endoscope device 82, the control device 83, and the light source device 84 constitute a medical observation device 200.

  The endoscope apparatus 82 is hard and has an elongated shape, and a distal end portion is detachably connected to the base end of the insertion portion 821 and an insertion portion 821 into which the distal end portion is inserted into the object to be observed. A camera head 822 that picks up an image of the object to be observed and outputs an image pickup signal, a cable 823 that transmits an image pickup signal output from the camera head 822 to the control device 83, and a base end of the cable 823 And a light guide 825 that transmits illumination light generated by the light source device 84 to the distal end portion of the insertion portion 821. Inside the camera head 822, a first imaging unit that captures a high-magnification image and a second imaging unit that captures a low-magnification image are provided. In addition, an input unit 822a including a high-magnification display button, a zoom switch, and the like is provided on the surface of the camera head 822 that is an operation unit held by the user when in use.

  Also in the medical observation device 200 having the above configuration, when the switching display mode is set, the control device 83 detects whether or not a medical instrument is reflected in the high-magnification image, and according to the detection result. Thus, the same effect as that of the embodiment can be obtained by performing the control for switching the display of the high-magnification image and the low-magnification image.

  In addition, instead of providing the first imaging unit and the second imaging unit in one endoscope apparatus 82, the first endoscope apparatus that includes the first imaging unit and images the vicinity of the affected part at high magnification, and the affected part are included. It is good also as a structure provided with the 2nd endoscope apparatus which images a range with low magnification. In this case, the control device is connected to each of the first and second endoscope devices to control the operation of both the endoscope devices, performs image processing on the in-vivo images captured by each endoscope device, and performs display processing. Generate image data. The light source device is configured to supply illumination light to the first and second endoscope devices, respectively. In the medical observation apparatus having such a configuration, the same effect as described above can be obtained.

  Further, instead of detecting the presence / absence of a medical instrument in the high-magnification image by image processing, it may be detected based on the relative position between the first imaging unit and the medical instrument. In this case, for example, the configuration described in Patent Document 1 can be employed. Specifically, by attaching a marker to each of the microscope unit 5 and the medical instrument, and by providing a navigation device that detects position information of each marker and detects a relative position between the microscope unit 5 and the medical instrument, high magnification is achieved. Detect the presence or absence of medical equipment in the image.

  Further, for example, when the medical device is removed from the first visual field region during surgery or when the medical device starts to be introduced into the first visual field region, the timing at which the image displayed on the display device 4 is switched is changed to metadata. As an alternative, the data may be added to at least one of the high-magnification image and the low-magnification image and stored. Thereby, when editing a motion picture of surgery, it is possible to easily grasp and edit the timing at which the display is switched during actual surgery.

  As described above, the present invention can include various embodiments and the like without departing from the technical idea described in the claims.

DESCRIPTION OF SYMBOLS 1 Surgical microscope system 2 Microscope apparatus 3, 83 Control apparatus 4, 85 Display apparatus 5 Microscope part 21, 22LR 1st imaging part 22, 22A, 22LR 2nd imaging part 23 Lens drive part 24, 32, 822a Input part 25 Signal Processing unit 26 Brake unit 27, 34 Control unit 31 Image processing unit 33 Storage unit 81 Endoscope system 82 Endoscope device 84 Light source device 100, 200 Medical observation device 211, 211L, 211R Variable magnification optical system 212, 212L, 212R First imaging optical system 213, 213LR First imaging element 214 Optical system 221, 221A, 221L, 221R Second imaging optical system 222, 222A, 222LR Second imaging element 223A Mirror 223L, 223R Prism 311 Detection unit 341 Display Control unit 501, 503, 504 Low magnification image 502 High-magnification image 821 Insertion part 822 Camera head F Outline P View center

Claims (10)

A first imaging unit that images the first visual field region;
A second imaging unit that images the second visual field region that includes the first visual field region and is wider than the first visual field region at a lower magnification than the first imaging unit;
A detection unit for detecting whether or not a medical instrument is reflected in the first image captured by the first imaging unit;
First display control for outputting a signal of the first image to a display device and displaying the first image on the display device according to a detection result of the detection unit, and at least a second image captured by the second imaging unit. A display control unit that executes any one of second display control for outputting a signal of two images to the display device and displaying the second image on the display device;
A medical observation apparatus comprising: The display control unit
When the detection unit detects that the medical instrument is shown in the first image, the first display control is executed, and the detection unit does not show the medical instrument in the first image. 2. The medical observation apparatus according to claim 1, wherein the second display control is executed when the image is detected. The display control unit
As the second display control, the second image is displayed on the display device as a main screen, and the first image is displayed on the display device as a sub-screen that is displayed in an area smaller than the main screen. The medical observation apparatus according to claim 1 or 2. The display control unit
4. The display device according to claim 1, wherein when the second image is displayed as the second display control, information on the first visual field region is superimposed on the second image and displayed on the display device. The medical observation apparatus according to claim 1. A first image display mode for continuing to display the first image can be set;
The display control unit
The medical observation apparatus according to any one of claims 1 to 4, wherein the first display control is executed when the first image display mode is set.   The medical observation apparatus according to claim 5, further comprising an input unit that is provided in an operation unit that receives an instruction to move the first imaging unit by a user and that receives a setting input of the first image display mode. . The detector is
The medical device according to any one of claims 1 to 6, wherein whether or not the medical instrument is reflected in the first image is detected by performing image processing on the first image. Observation device. A first imaging unit that images the first visual field region; a second imaging unit that captures a second visual field region that includes the first visual field region and is wider than the first visual field region at a lower magnification than the first imaging unit; A control device for controlling an image acquisition device comprising:
A detection unit for detecting whether or not a medical instrument is reflected in the first image captured by the first imaging unit;
First display control for outputting a signal of the first image to a display device and displaying the first image on the display device according to a detection result of the detection unit, and at least a second image captured by the second imaging unit. A display control unit for switching and executing a second display control for outputting a signal of two images to the display device and displaying the second image on the display device;
A control device comprising: A first imaging unit that images the first visual field region; a second imaging unit that captures a second visual field region that includes the first visual field region and is wider than the first visual field region at a lower magnification than the first imaging unit; A method for operating a control device for controlling an image acquisition device comprising:
A detection step for detecting whether or not a medical instrument is reflected in the first image captured by the first imaging unit;
A display control unit that outputs a signal of the first image to a display device in accordance with a detection result of the detection unit to display the first image on the display device; and at least the second imaging A display control step of executing any one of a second display control for outputting a signal of a second image captured by the unit to the display device and displaying the second image on the display device;
A method of operating a control device comprising: A first imaging unit that images the first visual field region; a second imaging unit that captures a second visual field region that includes the first visual field region and is wider than the first visual field region at a lower magnification than the first imaging unit; In a control device for controlling an image acquisition device comprising
A detection step for detecting whether or not a medical instrument is reflected in the first image captured by the first imaging unit;
A display control unit that outputs a signal of the first image to a display device in accordance with a detection result of the detection unit to display the first image on the display device; and at least the second imaging A display control step of executing any one of a second display control for outputting a signal of a second image captured by the unit to the display device and displaying the second image on the display device;
An operation program for a control device, characterized in that

Images