JP2015119827A - Endoscope system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable an appropriate image to be easily displayed in a display device without using a plurality of endoscopes and without moving an endoscope.SOLUTION: An endoscope system includes an endoscope 1 for imaging a subject inside an observation object, a controller 2 for controlling the imaging performed by the endoscope and processing a picked-up image output from the endoscope, and a monitor 3 for displaying the image output from the control device. The endoscope includes a plurality of camera parts 6a-6e in an insertion part 5 inserted inside the observation object, and the controller performs processing to make adjustments of at least either switching or synthesis for a plurality of the picked-up images by each of the plurality of camera parts and to generate a display image, and causes the generated display image to be displayed in the monitor.

Description

  The present invention relates to an endoscope that images a subject inside an observation target, a control device that controls imaging performed by the endoscope, and that processes a captured image output from the endoscope, and the control device The present invention relates to an endoscope system that includes a display device that displays an image output from the device.

  In endoscopic surgery, an image sensor is arranged at the distal end of an insertion part that is inserted into the body of a person to be observed, and an image obtained by imaging with this image sensor is displayed on a monitor. Endoscope systems using so-called electronic endoscopes are widely used. In such an endoscope system, the affected lesion is observed from various directions, the lesion is observed in detail with a high resolution image, and a wide area around the lesion is observed with a wide-angle image. If it can be done, it will have a great effect on the surgical operation.

  As related to such a demand, conventionally, a plurality of endoscopes are used, and a plurality of captured images obtained by the plurality of endoscopes are displayed side by side on a monitor screen, or a plurality of captured images are selectively displayed. A technique for displaying is known (see Patent Documents 1 and 2). If these techniques are used, various images can be acquired simultaneously, so that appropriate images required by the operator and assistant in endoscopic surgery can be displayed on the monitor.

JP 2005-021392 A JP-A-8-018861

  However, the conventional technique has a problem in that the cost increases because a plurality of endoscopes are required. Also, in endoscopic surgery, various surgical instruments such as a scalpel and forceps are inserted in addition to the endoscope through the incision, so using a plurality of endoscopes is a great way to proceed with surgery. There was a problem of hindrance.

  In addition, if the endoscope is moved, a variety of images can be displayed even with a single endoscope, but it is difficult to move the endoscope freely due to interference with the surgical instrument. Therefore, there is a problem in that it is impossible to easily display an appropriate image required by an operator and an assistant in endoscopic surgery on a monitor.

  The present invention has been devised to solve such problems of the prior art, and its main purpose is to use a plurality of endoscopes without moving the endoscopes. An object of the present invention is to provide an endoscope system configured such that an appropriate image can be easily displayed on a display device.

  An endoscope system of the present invention includes an endoscope that images a subject inside an observation target, and a control device that controls imaging performed by the endoscope and processes a captured image output from the endoscope And a display device that displays an image output from the control device, wherein the endoscope includes a plurality of camera units in an insertion unit that is inserted into an observation target. And the control device includes an image output adjustment unit configured to generate a display image by performing at least one of switching and combining adjustment on a plurality of captured images by each of the plurality of camera units. The display image generated by the output adjustment unit is displayed on the display device.

  According to the present invention, since a plurality of camera units provided in an insertion portion of an endoscope capture an image of a subject inside an observation target, various images can be acquired simultaneously. Since a display image is generated by performing at least one of switching and synthesis on a plurality of captured images by each, without using a plurality of endoscopes and without moving the endoscope, An appropriate image can be easily displayed on the display device.

1 is an overall configuration diagram of an endoscope system according to a first embodiment. The perspective view which shows the camera parts 6a-6e provided in the insertion part 5 of the endoscope 1 Sectional drawing which shows the structure of the camera parts 6a-6e Explanatory drawing explaining the point of the imaging and image output adjustment in the endoscope system which concerns on 1st Embodiment. Functional block diagram showing a schematic configuration of the controller 2 according to the first embodiment The perspective view which shows the condition where an observer gives an instruction | indication with the instruction | indication detector 4. Explanatory drawing which shows the change state of the gain of the image signal at the time of image switching Explanatory drawing explaining the point of the imaging and image output adjustment in the endoscope system which concerns on 2nd Embodiment. Functional block diagram showing a schematic configuration of the controller 2 according to the second embodiment Explanatory drawing which shows the change state of the gain of the image signal at the time of image composition The side view which shows the insertion part 5 of the endoscope 1 in the endoscope system which concerns on 3rd Embodiment. Explanatory drawing explaining the point of the imaging and image output adjustment in the endoscope system which concerns on 3rd Embodiment. Functional block diagram showing a schematic configuration of the controller 2 according to the third embodiment Explanatory drawing explaining the point of the imaging and image output adjustment in the endoscope system which concerns on 4th Embodiment. Explanatory drawing explaining in detail the point of the image output adjustment in the endoscope system which concerns on 4th Embodiment. Functional block diagram showing a schematic configuration of a controller according to the fourth embodiment Side view showing a modification of the insertion portion 5 of the endoscope 1 Explanatory drawing which shows the modification of the endoscope system which concerns on 1st Embodiment.

  A first invention made to solve the above problems is an endoscope that images a subject inside an observation target, and an image that is controlled by the endoscope and that is output from the endoscope. An endoscope system including a control device that processes an image and a display device that displays an image output from the control device, wherein the endoscope is inserted into an observation target A plurality of camera units, and the control device performs image output adjustment for generating a display image by performing at least one of switching and combining adjustment on a plurality of captured images by each of the plurality of camera units. The display image generated by the image output adjustment unit is displayed on the display device.

  According to this, since the subject inside the observation target is imaged by the plurality of camera units provided in the insertion unit of the endoscope, various images can be acquired at the same time, and further, by each of the plurality of camera units Since a display image is generated by adjusting at least one of switching and synthesis on a plurality of captured images, it is possible to use an appropriate operation without using a plurality of endoscopes or moving an endoscope. An image can be easily displayed on a display device.

  According to a second aspect of the present invention, the plurality of camera units are configured such that each line-of-sight direction is directed to the same gaze position on the subject.

  According to this, since a plurality of captured images having different line-of-sight directions can be acquired simultaneously, it is possible to observe the portion of the subject on which the viewer pays attention from different directions without moving the endoscope.

  According to a third aspect of the present invention, the image output adjustment unit is configured to switch and output a plurality of the captured images as the display image according to an instruction from an observer.

  According to this, it is possible to observe the part of the subject on which the viewer pays attention from different directions with one display device.

  According to a fourth aspect of the present invention, the endoscope further includes a line-of-sight adjustment mechanism that individually changes a line-of-sight direction of the camera unit, and the control device is responsive to an observer's instruction to specify the gaze position. A line-of-sight direction setting unit that sets the line-of-sight direction of the camera unit so that the line-of-sight directions of the plurality of camera units are directed to the same gaze position, and the line-of-sight direction set by the line-of-sight direction setting unit And a camera control unit that controls the line-of-sight adjustment mechanism.

  According to this, the observer need only specify the gaze position, and can save the trouble of individually adjusting the line-of-sight directions of the plurality of camera units.

  According to a fifth aspect of the present invention, the plurality of camera units are configured such that the line-of-sight direction is set so as to capture areas that are shifted from each other on the subject.

  According to this, since a wide range of the subject can be imaged simultaneously, it is possible to observe the wide range of the subject without moving the endoscope.

  According to a sixth aspect of the present invention, the insertion portion has a bent portion between the plurality of camera portions, and the bending direction of the plurality of camera portions is changed by bending the bent portion. The configuration.

  According to this, since a very wide range of the subject can be imaged simultaneously, it is possible to observe an extremely wide range of the subject without moving the endoscope.

  Further, according to a seventh aspect of the invention, the image output adjustment unit generates a single continuous composite image by joining a plurality of the captured images and uses all or part of the composite image as the display image. And

  According to this, a wide range and high resolution display image can be obtained. In other words, since a plurality of captured images are joined, a wide range of images can be obtained, and the viewing angle (view angle) of each camera unit is set to be narrow in order to shoot a subject by sharing with a plurality of camera units. Therefore, a high resolution image can be obtained.

  In addition, according to an eighth aspect, the image output adjustment unit cuts out the display image by cutting out a range according to the display condition from the composite image in accordance with an instruction from an observer who specifies a display condition regarding the display image. A configuration to generate.

  According to this, the range of the subject required by the observer can be freely displayed. And since the resolution of the synthetic image which becomes the origin of a display image is high, even if it cuts out a narrow range and produces | generates a display image, an image of sufficient resolution can be displayed.

  In a ninth aspect of the invention, the display condition relates to panning and tilting for moving a display area of the display image, and the image output adjustment unit changes a position of a range to be cut out from the composite image. The display image is configured to perform pseudo pan and tilt operations.

  According to this, since the panning and tilting operations can be performed on the display image, the convenience of the observer can be improved. And since the resolution of the synthetic image which becomes the origin of a display image is high, it can pan and tilt with a high-resolution display image.

  In a tenth aspect of the invention, the display condition relates to a zoom for enlarging and reducing a display area of the display image, and the image output adjustment unit changes a size of a range cut out from the composite image. Thus, a pseudo zoom operation is performed on the display image.

  According to this, since the display image can be zoomed, the convenience of the observer can be improved. And since the resolution of the composite image which becomes the origin of a display image is high, even if it expands by zoom, an image with sufficient resolution can be displayed.

  The eleventh invention includes a plurality of display devices for each of a plurality of observers, and the control device separately assigns an arbitrary range of the composite image to the plurality of display devices in accordance with an instruction from the observer. It is set as the structure displayed on.

  According to this, since the area of the subject required by each of the plurality of observers can be displayed on the display device, the convenience of the observer can be improved.

  In a twelfth aspect of the invention, at least one of the plurality of camera units and the remaining camera units are set to have different viewing angles, and the image output adjustment unit has a narrow viewing angle. From the narrow-angle captured image obtained by the camera unit set to, a display image for gazing in which a narrow range including the gaze position on the subject is captured is generated, and from the wide-angle captured image obtained by the camera unit set to a wide viewing angle The monitoring display image is taken to capture a wide range including the gaze position on the subject.

  According to this, it is possible to simultaneously acquire a high-resolution display image for gazing and a display image for monitoring that captures a wide range. Since the portion of the subject that the viewer is interested in is displayed as a high-resolution image, the portion can be observed in detail, and at the same time, the peripheral portion image is displayed in a wide range. Widely observable.

  The thirteenth invention includes a plurality of the display devices for each of a plurality of observers, and the control device displays the gaze display image and the monitoring display image on another display device. And

  According to this, an observer who needs to observe a narrow area on the subject in detail, such as an operator of endoscopic surgery, can view a display image for gazing with a dedicated display device. An observer who needs to observe a wide area on the subject, such as a surgical assistant, can view a display image for monitoring on a dedicated display device.

  In a fourteenth aspect, the plurality of camera units are set to a wide viewing angle, and the image output adjustment unit joins the plurality of wide-angle captured images by the plurality of camera units set to a wide viewing angle. Then, one continuous composite image is generated, and all or part of the composite image is used as the monitoring display image.

  According to this, a wide range and high resolution display image for monitoring can be acquired.

  In the fifteenth aspect of the invention, the image output adjustment unit generates a composite image obtained by combining the narrow-angle captured image and the wide-angle captured image, and extracts a range including the narrow-angle captured image from the composite image. Thus, the display image for gaze is generated.

  According to this, it is possible to observe the portion of the subject on which the observer pays attention with a high-resolution image and simultaneously observe the peripheral portion.

  According to a sixteenth aspect of the present invention, the endoscope further includes a line-of-sight adjustment mechanism that individually changes a line-of-sight direction of the camera unit, and the control device moves the display area of the display image for gazing. In accordance with the instructions of the person, the line-of-sight direction of the camera unit set to a narrow viewing angle so that the imaging range of the camera unit set to a narrow viewing angle is located within the region of the display image for gaze And a camera control unit that controls the line-of-sight adjustment mechanism based on the line-of-sight direction set by the line-of-sight direction setting unit.

  According to this, since a high-resolution narrow-angle captured image by the camera unit set to a narrow viewing angle is always located in the display image for gaze, even if the display area of the display image for gaze is moved, the subject The upper important part can be observed with a high-resolution image.

  An seventeenth aspect of the invention is an endoscope that images an object inside an observation target, and a control device that controls imaging performed by the endoscope and processes a captured image output from the endoscope. An instruction detection device for detecting an instruction operation of an observer performed while viewing a display screen of the display device, the display system displaying an image output from the control device The endoscope further includes a plurality of camera units in an insertion portion that is inserted into an observation target, and the control device analyzes an instruction operation detected by the instruction detection device, and the display device An instruction content acquisition unit that acquires instruction content related to the display screen, and based on the instruction content acquired by the instruction content acquisition unit, at least switching and combining for a plurality of captured images by each of the plurality of camera units Performing adjustment of Zureka, and an image output adjusting unit that generates a display image, a configuration for displaying the display image generated by the image output adjusting section on the display device.

  According to this, since a subject inside the observation target is imaged by a plurality of camera units provided in the insertion unit of the endoscope, various images can be acquired at the same time, and further according to the instructions of the observer Since a display image is generated by performing at least one of switching and synthesis on a plurality of captured images by each of a plurality of camera units, and without using a plurality of endoscopes, An appropriate image required by the observer can be easily displayed on the display device without moving the mirror.

  In an eighteenth aspect of the invention, the endoscope further includes a line-of-sight adjustment mechanism that individually changes a line-of-sight direction of the camera unit, and the instruction content acquisition unit performs an instruction operation detected by the instruction detection device. Analyzing and acquiring instruction content regarding the line-of-sight direction of the camera unit, and the control device includes a camera control unit that controls the line-of-sight adjustment mechanism based on the instruction content acquired by the instruction content acquisition unit And

  According to this, since the line-of-sight direction of the camera unit can be adjusted by an instruction from the observer, a more appropriate image required by the observer can be easily displayed on the display device.

  According to a nineteenth aspect of the present invention, the instruction detection device includes at least one of a gesture performed by an observer, a voice uttered by the observer, a gaze of the observer, a touch operation performed by the observer, and a button operation performed by the observer. It is set as the structure which detects.

  According to this, the control device can accurately acquire the instruction content of the observer. In particular, when detecting a gesture, it is possible to give an instruction with only one hand on the spot, and when detecting a voice or line of sight, an instruction can be given without using a hand. In mirror surgery, the observer can easily give instructions even if it is troublesome to move or use hands due to the holding of surgical instruments such as endoscopes, scalpels, and forceps. This can reduce the burden on the surgeon and assistant.

  In a twentieth aspect, the plurality of camera units are arranged side by side in the longitudinal direction of the insertion unit, and the line-of-sight direction is set in a direction substantially perpendicular to the arrangement direction.

  According to this, since an insertion part can be formed in a small diameter, the operation | work which inserts an insertion part becomes easy. In particular, in endoscopic surgery, since the incision portion for inserting the insertion portion can be reduced, the burden on the patient can be reduced.

  In the twenty-first aspect, each of the plurality of camera units changes a panning operation mechanism that changes the line-of-sight direction in the longitudinal direction of the insertion unit, and changes the line-of-sight direction in a direction orthogonal to the longitudinal direction of the insertion unit. A tilt operation mechanism is provided.

  According to this, the line-of-sight direction of each of the plurality of camera units can be changed to an arbitrary direction.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an overall configuration diagram of an endoscope system according to the first embodiment. The endoscope system includes an endoscope 1 that images a subject inside an observation target, and a controller (control) that controls imaging performed by the endoscope and processes a captured image output from the endoscope. Device) 2, a monitor (display device) 3 that displays an image output from the controller 2, and an instruction detector (instruction detection device) that detects an observer's instruction operation while viewing the display screen of the monitor 3. 4 is provided.

  The endoscope 1 is provided with a plurality of camera units 6a to 6e on the distal end side of the insertion unit 5 to be inserted into the observation target, and image signals output from the camera units 6a to 6e are respectively received. Input to the controller 2.

  Next, the camera parts 6a-6e provided in the insertion part 5 of the endoscope 1 shown in FIG. 1 will be described. FIG. 2 is a perspective view showing the camera units 6a to 6e. FIG. 3 is a cross-sectional view illustrating the configuration of the camera units 6a to 6e.

  As shown in FIG. 2, the plurality of camera units 6 a to 6 e are arranged side by side in the longitudinal direction of the insertion unit 5, and the line-of-sight direction is set in a direction substantially orthogonal to the arrangement direction. That is, an imaging window 8 that guides light from the subject to an internal imaging device (not shown) is provided on the side surface of the insertion portion 5. The plurality of camera units 6a to 6e are provided with a line-of-sight adjusting mechanism that individually changes the line-of-sight directions of the camera units 6a to 6e in a state where the insertion unit 5 is inserted into the observation target.

  As shown in FIG. 3, the camera units 6 a to 6 e are housed in a cylindrical housing 9 that constitutes the insertion unit 5 of the endoscope 1, and include a camera housing 11, an imaging unit 12, and a movable body. A mirror 13, a mirror rotation actuator 14, and a camera rotation actuator 15 are provided.

  The imaging unit 12 includes optical system components such as an imaging element and a lens (not shown). The housing 9 of the insertion portion 5 is provided with an imaging window 8 made of a light transmitting material, and light from the subject is guided to the inside of the camera housing 11 via the imaging window 8 and is movable. The light is reflected by the mirror 13 and enters the imaging unit 12.

  As shown by an arrow C, the movable mirror 13 is provided so as to be rotatable about the support shaft 17.

  The mirror rotation actuator 14 is composed of an electric motor or the like, and is fixed to the camera housing 11. The mirror rotation actuator 14 is connected to the movable mirror 13, and when the mirror rotation actuator 14 is operated, the movable mirror 13 rotates about the support shaft 17 as indicated by an arrow C. Thereby, as shown by the arrow A in FIG. 2, the panning operation is performed to change the viewing direction of the camera units 6 a to 6 e in the longitudinal direction of the insertion unit 5, and the movable mirror 13, the support shaft 17, and the mirror rotate. The actuator 14 constitutes a pan operation mechanism.

  The camera housing 11 is rotatably supported by the support shafts 18 and 19. The support shafts 18 and 19 are fixed to support members 20 and 21 fixed to the housing 9 of the insertion portion 5. The support shafts 18 and 19 are arranged in the longitudinal direction of the insertion portion 5, and the camera housing 11 is centered on a central axis in a direction parallel to the longitudinal direction of the insertion portion 5 as indicated by an arrow D. It is designed to rotate.

  The camera rotation actuator 15 is composed of an electric motor or the like, and is fixed to the camera housing 11. The camera rotation actuator 15 is connected to one support shaft 18, and when the camera rotation actuator 15 is operated, the camera units 6 a to 6 e rotate about the support shafts 18 and 19 as indicated by an arrow D. To do. As a result, a tilting operation is performed to change the line-of-sight direction of the camera units 6a to 6e in a direction orthogonal to the longitudinal direction of the insertion unit 5 as indicated by an arrow B in FIG. The rotation actuator 15 constitutes a tilt operation mechanism.

  The insertion unit 5 may be provided with an illumination unit that illuminates the imaging region. In this case, in addition to arranging a light source composed of an LED or the like in the insertion unit 5, a light source is arranged in the base of the insertion unit 5 or the controller 2 in the endoscope 1, and the light of the light source is inserted using an optical fiber. You may make it guide | induced to the front end side of 5.

  Next, the point of imaging and image output adjustment in the endoscope system according to the first embodiment will be described. FIG. 4 is an explanatory diagram for explaining the points of imaging and image output adjustment in the endoscope system according to the first embodiment.

  In the first embodiment, the first to fifth camera units 6a to 6e are set so that their line-of-sight directions are directed to the same gaze object (gaze position) on the subject. Thereby, a gaze object appears in the center of the 1st-5th picked-up image by the 1st-5th camera parts 6a-6e. Moreover, since the 1st-5th camera parts 6a-6e are shifted | deviated and arrange | positioned in the longitudinal direction of the insertion part 5, the 1st-5th picked-up image by the 1st-5th camera parts 6a-6e. Then, the gaze object appears in a state of being imaged from different directions.

  The first to fifth captured images by the first to fifth camera units 6a to 6e are switched, that is, one of the first to fifth captured images is selected and displayed on the monitor 3. The Here, when switching the first to fifth captured images, it is preferable to sequentially switch to the captured images by the adjacent camera units 6a to 6e in accordance with the switching instruction. Thereby, the image is displayed so that the line-of-sight direction changes in order, so that the state of the gaze position on the subject can be grasped naturally. In this case, an image as if one camera was photographed while moving to the position of the first to fifth camera units 6a to 6e would be observed. That is, motion parallax is generated by the movement of the observer, and this allows the observer to perceive the object three-dimensionally, thereby obtaining an effect that makes it easy to understand the spatial structure of the object. In addition, you may enable it to switch to the picked-up image by arbitrary camera parts 6a-6e.

  Next, the controller 2 according to the first embodiment will be described. FIG. 5 is a functional block diagram illustrating a schematic configuration of the controller 2 according to the first embodiment.

  The controller 2 includes an image processing unit 31, an image output adjustment unit 32, a display control unit 33, a camera control unit 34, a gaze direction setting unit 35, a gaze position setting unit 36, a gaze position tracking unit 37, An instruction content acquisition unit 38.

  In the image processing unit 31, processing such as gain adjustment and dynamic range expansion is performed on a plurality of captured images respectively output from the plurality of camera units 6a to 6e.

  The image output adjustment unit 32 performs a process of generating a display image by adjusting at least one of switching and synthesis on the plurality of captured images processed by the image processing unit 31. In particular, in the present embodiment, the image output adjustment unit 32 includes an image switching unit 41. In the image switching unit 41, a plurality of captured images are switched according to an instruction from an observer, that is, a plurality of imaging images. One of the images is selected and output as a display image.

  In the display control unit 33, processing for displaying the display image output from the image output adjustment unit 32 on the monitor 3 is performed.

  In the camera control unit 34, the imaging unit 12, the mirror rotation actuator 14, and the camera rotation actuator 15 of the plurality of camera units 6a to 6e are controlled.

  In the line-of-sight direction setting unit 35, processing for setting the line-of-sight directions of the plurality of camera units 6a to 6e, that is, the imaging angles (pan angle and tilt angle) is performed in accordance with an instruction from the observer. Based on the line-of-sight direction set by the line-of-sight direction setting unit 35, the camera rotation unit 14 controls the mirror rotation actuator 14 and the camera rotation actuator 15. Thereby, the line-of-sight directions of the plurality of camera units 6a to 6e can be manually adjusted.

  The gaze position setting unit 36 performs processing for setting a gaze position on the subject in accordance with an instruction from the observer. Based on the gaze position set by the gaze position setting unit 36, in the gaze direction setting unit 35, the gaze directions of the plurality of camera units 6a to 6e face the same gaze position on the subject, that is, the gaze position is imaged. A process of setting the line-of-sight direction of the plurality of camera units 6a to 6e is performed so as to be in the center of the image. Based on the line-of-sight direction set by the line-of-sight direction setting unit 35, the camera control unit 34 rotates the mirror. The actuator 14 and the camera rotation actuator 15 are controlled.

  The gaze position tracking unit 37 performs processing for tracking a gaze position on a subject that moves relatively as the endoscope 1 moves. Based on the tracking result of the gaze position tracking unit 37, the gaze direction setting unit 35 sets the plurality of camera units 6a to 6e so that the gaze directions of the plurality of camera units 6a to 6e face the gaze position on the subject. A process for setting the line-of-sight direction is performed, and the mirror rotation actuator 14 and the camera rotation actuator 15 are controlled by the camera control unit 34 based on the line-of-sight direction set by the line-of-sight direction setting unit 35. A known object tracking technique may be used for the gaze position tracking process.

  The instruction content acquisition unit 38 performs processing for analyzing the instruction operation of the observer detected by the instruction detector 4 and acquiring instruction content corresponding to the instruction operation. In particular, in this embodiment, the instruction content regarding the display screen of the monitor 3 and the plurality of camera units 6a to 6e is acquired. The instruction content related to the display screen of the monitor 3 relates to a switching instruction when a plurality of captured images from the plurality of camera units 6 a to 6 e are switched and displayed on the monitor 3. The instruction content related to the camera units 6a to 6e relates to the gaze position when the observer designates the gaze position and the gaze direction when the gaze direction is manually adjusted.

  Next, the instruction detector 4 shown in FIG. 1 will be described. FIG. 6 is a perspective view showing a situation in which an observer gives an instruction with the instruction detector 4.

  The instruction detector 4 detects an observer's instruction operation performed while viewing the display screen of the monitor 3, and is disposed in the vicinity of the monitor 3. In particular, in this embodiment, the instruction detector 4 performs at least one of a gesture performed by the observer, a voice uttered by the observer, a gaze of the observer, a touch operation performed by the observer, and a button operation performed by the observer. Configured to detect.

  When detecting a gesture performed by an observer, the instruction detector 4 includes a camera and an image recognition unit, and a known image recognition technique is used in the image recognition unit from a captured image obtained by capturing the observer with the camera. Then, gestures, that is, movements of hands and arms are detected. Further, when detecting the voice uttered by the observer, the instruction detector 4 is constituted by a microphone and a voice recognition unit, and the voice input from the microphone is used in the voice recognition unit using a known voice recognition technique. To convert the voice into text information. In addition, when detecting the line of sight of the observer, the instruction detector 4 includes a camera and a line of sight detection unit, and a known line of sight detection is performed in the line of sight detection unit from a captured image obtained by capturing the eyes of the observer with the camera. The observer's line of sight, i.e., eye movement, is detected using technology. Further, when detecting a touch operation performed by the observer, the instruction detector 4 is configured by a touch panel and a control unit that are arranged on the monitor screen, and detects the touch operation performed on the touch panel. In addition, when detecting a button operation performed by the observer, the instruction detector 4 is configured by an operation unit and a control unit in which buttons are arranged, and detects a button operation performed by the operation unit.

  In the first embodiment, using the instruction detector 4, an instruction to switch a plurality of captured images by the plurality of camera units 6 a to 6 e and an instruction to specify a gaze position are performed. For example, when instructing with a gesture, the instruction to switch the captured image may be such that the image is switched by an action of shaking the hand or arm, and the instruction to specify the gaze position is the screen of the monitor 3 A pointer indicating the gaze position may be displayed on the top, and the gaze position may be designated by moving the pointer according to the direction of shaking the hand or arm.

  FIG. 7 is an explanatory diagram showing a change state of the gain of the image signal at the time of image switching. Here, when a plurality of captured images by the plurality of camera units 6a to 6e are switched and output, the switching source image signal and the switching destination image signal are combined and output. At this time, the switching source image signal In addition, a composite signal is generated by multiplying the image signal to be switched by a gain. In particular, here, the gain multiplied by the switching source image signal and the switching destination image signal is gradually changed, that is, the synthesis ratio of the image signals is gradually changed.

  In the example shown in FIG. 7A, the gain is changed linearly, that is, with a constant slope. In the example shown in FIG. 7B, the gain is changed in a curved line, that is, while changing the slope gradually. I am letting. As described above, when the captured image is switched and output, the image displayed on the monitor 3 is switched over time by gradually changing the composition ratio of the image signals. The observer can recognize it naturally. Note that the image signal may be simply switched from the switching source image signal to the switching destination image signal without multiplying the image signal by a gain.

  In the first embodiment, a plurality of images with different line-of-sight directions obtained by the plurality of camera units 6a to 6e are switched and displayed on one monitor 3, but a plurality of images with different line-of-sight directions are displayed. May be displayed side by side on one screen. Further, a plurality of monitors 3 may be provided, and a plurality of images having different line-of-sight directions may be displayed separately on the plurality of monitors 3.

  As described above, in the first embodiment, the subject inside the observation target is imaged by the plurality of camera units 6a to 6e, and switching adjustment is performed on the plurality of captured images by each of the plurality of camera units 6a to 6e. Since the display image generated by this is displayed on the monitor 3, an appropriate image required by the observer can be easily displayed on the monitor 3 without moving the endoscope 1.

  In particular, in the first embodiment, since the plurality of camera units 6a to 6e are set so that each line-of-sight direction is directed to the same gaze position on the subject, a plurality of captured images having different line-of-sight directions are displayed. Thus, it is possible to obtain the portion of interest on the subject from different directions without moving the endoscope 1.

  In the first embodiment, since the image output adjustment unit 32 switches a plurality of captured images and outputs them as display images in accordance with an instruction from the observer, attention is paid to the subject on one monitor 3. The part can be observed from different directions.

  Further, in the first embodiment, a line-of-sight adjustment mechanism that individually changes the line-of-sight directions of the plurality of camera units 6a to 6e is provided, and the plurality of camera units 6a to 6e are provided in accordance with an instruction from an observer who designates the gaze position. Since the gaze direction of multiple camera units is set so that each gaze direction is directed to the same gaze position, and the gaze adjustment mechanism is controlled based on the gaze direction, the observer designates the gaze position Therefore, it is possible to save the trouble of individually adjusting the line-of-sight directions of the plurality of camera units 6a to 6e. Further, in the first embodiment, since the gaze position on the subject that moves relatively as the endoscope 1 moves is tracked, the trouble of redesignating the gaze position even if the endoscope 1 moves. Can be omitted.

  Moreover, in this 1st Embodiment, since the instruction | indication detector 4 was used and the instruction | indication regarding the display screen of the monitor 3 and the instruction | indication regarding the gaze direction of several camera parts 6a-6e were performed, an observer needs. A more appropriate image can be easily displayed on the monitor 3.

  In particular, in the first embodiment, the instruction detector 4 is used to perform gestures performed by the observer, voices spoken by the observer, observer's line of sight, touch operations performed by the observer, and button operations performed by the observer. Since the observer gives an instruction by at least one of them, the controller 2 can accurately acquire the instruction contents of the observer. In particular, when detecting a gesture, it is possible to give an instruction with only one hand on the spot, and when detecting a voice or line of sight, an instruction can be given without using a hand. In mirror surgery, the observer can easily give instructions even if it is troublesome to move or use hands due to the holding of surgical instruments such as endoscopes, scalpels, and forceps. This can reduce the burden on the surgeon and assistant.

  In the first embodiment, the plurality of camera units 6a to 6e are arranged side by side in the longitudinal direction of the insertion unit 5, and the line-of-sight direction is set in a direction substantially orthogonal to the arrangement direction. Since the insertion part 5 can be formed in a small diameter, the operation of inserting the insertion part 5 becomes easy. In particular, in endoscopic surgery, since the incision portion for inserting the insertion portion 5 can be reduced, the burden on the patient can be reduced.

  In the first embodiment, each of the plurality of camera units 6a to 6e includes a pan operation mechanism that changes the line-of-sight direction in the longitudinal direction of the insertion unit, and a direction in which the line-of-sight direction is orthogonal to the longitudinal direction of the insertion unit. Since the tilt operation mechanism is provided to change the viewing direction, the line-of-sight direction of each of the plurality of camera units 6a to 6e can be changed to an arbitrary direction.

(Second Embodiment)
Next, an endoscope system according to the second embodiment will be described. The points not particularly mentioned here are the same as in the first embodiment.

  FIG. 8 is an explanatory diagram for explaining the points of imaging and image output adjustment in the endoscope system according to the second embodiment.

  In the second embodiment, the line-of-sight direction is set so that the first to fifth camera units 6a to 6e capture areas that are shifted from each other on the subject, and the first to fifth camera units 6a to 6e are configured. From 6e, the first to fifth individual captured images whose imaging regions are shifted from each other are output. Then, the first to fifth individual captured images are joined to generate one continuous composite image, and the entire composite image is used as a display image, or a part of the composite image is cut out to generate a display image. The display image is displayed on the monitor 3. At this time, if the entire composite image is used as a display image, or if a wide range of the composite image is cut out, a wide-angle image showing a wide range of the subject is displayed, and a narrow range of the composite image is cut out, converted in resolution, and enlarged to the entire screen. If displayed, an enlarged image showing a narrow range of the subject is displayed.

  In the example illustrated in FIG. 8, the first to third display ranges are cut out from the composite image to generate the first to third display images, and the first to third display images are displayed on the three monitors 3, respectively. In the monitor 3 on which the first display image obtained by cutting out the wide first display range is displayed, a wide-angle image showing a wide range of the subject is displayed. Suitable for On the monitor 3 on which the third display image obtained by cutting out the narrow third display range is displayed, an enlarged image showing a narrow range of the subject is displayed, which is suitable for a surgeon to view.

  Here, by changing the position of the range to be cut out from the composite image, it is possible to perform pseudo pan and tilt operations on the display image. In other words, the panning operation can be performed by shifting the cutout range in the horizontal direction, and the tilt operation can be performed by shifting the cutout range in the vertical direction. In addition, a pseudo zoom operation can be performed on the display image by enlarging or reducing the size of the range cut out from the composite image. That is, an enlarged display is made by reducing the size of the cutout range, and a reduced display is made by increasing the size of the cutout range.

  Next, the controller 2 according to the second embodiment will be described. FIG. 9 is a functional block diagram showing a schematic configuration of the controller 2 according to the second embodiment.

  In the second embodiment, the image output adjustment unit 32 of the controller 2 includes an image synthesis unit 42 and an image cutout unit 43.

  In the image composition unit 42, a process of generating a single continuous composite image by joining (panoramic composition) a plurality of individual captured images captured by the plurality of camera units 6a to 6e is performed. The line-of-sight directions of the camera units 6a to 6e are set so that the respective imaging ranges of the plurality of camera units 6a to 6e partially overlap each other between the two adjacent camera units 6a to 6e. A process of joining the individually captured images using the generated overlapping portion is performed.

  The image cutout unit 43 performs a process of cutting out a range according to the display condition from the composite image and generating a display image in accordance with an instruction from an observer who specifies the display condition. Display conditions relate to panning and tilting to move the display area of the display image, and zooming to enlarge and reduce the display area of the display image, and the position of the range to be cut out from the composite image is changed according to the instructions of the observer Thus, a pseudo panning and tilting operation can be performed on the display image, and a pseudo zoom operation can be performed on the display image by changing the size of the range cut out from the composite image.

  Note that the synthesized image may be displayed on the monitor 3 without being cut out. The processing performed in the image cutout unit 43 is performed independently for each display image to be displayed on the plurality of monitors 3, so that an observer who views each monitor 3 can freely perform pan, tilt, and zoom operations. Can be done.

  The controller 2 includes an image duplication detection unit 51. In the image synthesis unit 42, a synthesis process is performed in which the individual captured images are joined using the overlapping portions of the individual captured images. The overlapping state of the individual captured images depends on the distance from the camera units 6a to 6e to the subject. Change. Therefore, the image duplication detection unit 51 performs processing for detecting the duplication status of the individually captured images.

  Based on the overlapping state of the individual captured images detected by the image overlap detection unit 51, the line-of-sight directions of the plurality of camera units 6a to 6e are generated in the line-of-sight direction setting unit 35 so that appropriate overlapping portions are generated in the individual captured images. The process of setting is performed. Based on the line-of-sight direction set by the line-of-sight direction setting unit 35, the camera rotation unit 14 controls the mirror rotation actuator 14 and the camera rotation actuator 15.

  FIG. 10 is an explanatory diagram showing a change state of the gain of the image signal at the time of image composition. Here, in the synthesis process for joining a plurality of individual captured images by the plurality of camera units 6a to 6e, image signals of two individual captured images of adjacent camera units 6a to 6e, that is, the n-th camera units 6a to 6e. The image signal and the image signal of the (n + 1) th camera units 6a to 6e are multiplied by a gain and then the two image signals are combined to generate an image signal of an overlapping portion of the individual captured images. In particular, here, the gain multiplied by the image signal is gradually changed, that is, the composition ratio of the image signals is gradually changed.

  In the example shown in FIG. 10A, the gain is changed linearly, that is, with a constant slope. In the example shown in FIG. 10B, the gain is changed in a curved line, that is, while changing the slope gradually. I am letting. In this way, when generating the image signal of the overlapping portion of the individually captured images, the combined ratio of the image signals is gradually changed, so that the joint portion of the individually captured images becomes inconspicuous and a natural composite image without any sense of incongruity Can be obtained.

  In the second embodiment, a plurality of monitors 3 are provided, and display images having different cutout ranges on the composite image are separately displayed on the plurality of monitors 3. However, the number of monitors 3 is necessary. The number can be increased or decreased accordingly. For example, only one monitor 3 may be provided.

  In the second embodiment, a plurality of captured images obtained by the camera units 6a to 6e are combined. However, the plurality of captured images may be output as they are without being combined. In this case, a plurality of captured images are switched and displayed on one monitor 3, a plurality of captured images are displayed side by side on a single screen, or a plurality of captured images are displayed separately on a plurality of monitors 3. It may be.

  In the second embodiment, as in the first embodiment, the camera units 6a to 6e have a line-of-sight adjustment mechanism that mechanically adjusts the line-of-sight direction, that is, a pan operation mechanism and a tilt operation mechanism. In the second embodiment, such a line-of-sight adjustment mechanism is not necessarily required, and a configuration without the line-of-sight adjustment mechanism is also possible. However, regardless of the distance from the camera units 6a to 6e to the subject, it is desirable to have a line-of-sight adjustment mechanism in order to appropriately form the overlapping portion of the captured images.

  As described above, in the second embodiment, the subject inside the observation target is imaged by the plurality of camera units 6a to 6e, and the plurality of captured images by each of the plurality of camera units 6a to 6e are combined and cut out. Since the display image generated by the adjustment is displayed on the monitor 3, an appropriate image required by the observer can be easily displayed on the monitor 3 without moving the endoscope 1.

  In the second embodiment, since the plurality of camera units 6a to 6e set the line-of-sight direction so as to pick up areas shifted from each other on the subject, it is possible to simultaneously pick up a wide range of the subject. As a result, a wide range of the subject can be observed without moving the endoscope 1.

  In the second embodiment, a plurality of captured images are joined to generate one continuous composite image, and all or part of the composite image is used as a display image. Display images can be obtained. That is, since a plurality of captured images are joined, a wide range of images can be obtained, and since the subject is shared by the plurality of camera units 6a to 6e, the viewing angles (images) of the camera units 6a to 6e are captured. Since the (angle) can be set narrow, a high-resolution image can be obtained.

  In the second embodiment, the display image is generated by cutting out the range corresponding to the display condition from the composite image in accordance with the instruction of the observer who specifies the display condition related to the display image. The range of the subject can be displayed freely. And since the resolution of the synthetic image which becomes the origin of a display image is high, even if it cuts out a narrow range and produces | generates a display image, an image of sufficient resolution can be displayed.

  In the second embodiment, by changing the position of the range to be cut out from the composite image, the display image is subjected to pseudo pan and tilt operations, and the size of the range to be cut out from the composite image is changed. Thus, since the pseudo zoom operation is performed on the display image, the convenience of the observer can be improved. And since the resolution of the composite image that is the source of the display image is high, panning and tilting can be performed with a high-resolution display image, and even when zoomed in, an image with sufficient resolution can be displayed. it can.

  In the second embodiment, since the arbitrary ranges of the composite image are separately displayed on the plurality of monitors 3 provided for each observer, the subject area required by each of the plurality of observers is displayed. Can be displayed on the monitor 3, and the convenience of the observer can be enhanced.

(Third embodiment)
Next, an endoscope system according to the third embodiment will be described. The points not particularly mentioned here are the same as in the first embodiment.

  FIG. 11 is a side view showing the insertion portion 5 of the endoscope 1 in the endoscope system according to the third embodiment.

  In the third embodiment, first to third camera units 6 a to 6 c are arranged in the insertion unit 5 of the endoscope 1. Further, a bent portion 61 is provided between the first to third camera portions 6a to 6c. By bending the bent portion 61, the line-of-sight direction of the first to third camera portions 6a to 6c is provided. Can be changed. A bending actuator 63 is provided in the vicinity of the bending portion 61. The bending actuator 63 is composed of an electric motor or the like, and causes the bending portion 61 to perform a bending operation.

  When inserting the insertion portion 5 into the observation target, as shown in FIG. 11A, if the insertion portion 5 is in a straight state, the operation of inserting the insertion portion into the observation target is facilitated, and observation is performed. After being inserted into the target, the bent portion 61 is bent as shown in FIG.

  Next, the point of the imaging and image output adjustment in the endoscope system according to the third embodiment will be described. FIG. 12 is an explanatory diagram for explaining the points of imaging and image output adjustment in the endoscope system according to the third embodiment.

  In the third embodiment, in the first to third camera units 6a to 6c, the side on which the imaging window 8 is provided has a dominant angle (an angle greater than 180 degrees), and the side opposite to the imaging window 8 has an inferior angle (180 The bent portion 61 is bent so that the angle is smaller than the angle. Thereby, it can be set as the state which the visual line direction of the 1st-3rd camera parts 6a-6c spreads radially.

  The procedure for adjusting the image output is the same as in the second embodiment, and the first to third individual captured images whose imaging areas are shifted from each other are output from the first to third camera units 6a to 6c. The first to third individual captured images are joined to generate a composite image. Then, the entire synthesized image or a part thereof is cut out to generate a display image, and the display image is displayed on the monitor 3.

  At this time, if the entire composite image is used as a display image, or if a wide range of the composite image is cut out and used as a display image, a wide-angle image showing a wide range of the subject can be displayed on the monitor 3. Further, if a narrow wide range of the composite image is cut out, an enlarged image showing a narrow range of the subject can be displayed on the monitor 3. In addition, by changing the position of the range to be cut out from the composite image, it is possible to cause the display image to perform pseudo pan and tilt operations, and by changing the size of the range to be cut out from the composite image, A pseudo zoom operation can be performed on the image.

  Next, the controller 2 according to the third embodiment will be described. FIG. 13 is a functional block diagram illustrating a schematic configuration of the controller 2 according to the third embodiment.

  In the third embodiment, the controller 2 includes a bending angle setting unit 65 and a bending control unit 66. The bending angle setting unit 65 performs processing for setting the bending angle of the bending portion 61 in accordance with an instruction from the observer. The bending control unit 66 controls the bending actuator 63 so that the bending angle set by the bending angle setting unit 65 is obtained. Thereby, it can set to the arbitrary bending angles according to an observer's instruction | indication.

  In addition, it is good also as a structure which controls the bending actuator 63 so that it may become a predetermined bending angle according to an observer's instruction | indication, making the bending angle of the bending part 61 constant.

  In the third embodiment, the bending actuator 63 is provided in the vicinity of the bending portion 61 so that the bending portion 61 performs a bending operation. However, the bending operation mechanism can have various configurations. For example, a bending actuator may be provided at the base of the insertion portion 5 and the bending portion 61 may be bent by pushing and pulling the wire inserted through the insertion portion 5 with this bending actuator.

  As described above, in the third embodiment, the bending portion 61 is provided between the plurality of camera portions 6a to 6c, and the bending portion 61 is bent to change the line-of-sight directions of the plurality of camera portions 6a to 6c. Therefore, as a whole, a very wide viewing angle, for example, a large viewing angle exceeding 180 degrees can be realized. For this reason, an extremely wide range of the subject can be imaged at the same time, so that an extremely wide range of the subject can be observed without moving the endoscope.

(Fourth embodiment)
Next, an endoscope system according to the fourth embodiment will be described. The points not particularly mentioned here are the same as in the first embodiment.

  FIG. 14 is an explanatory diagram for explaining the points of imaging and image output adjustment in the endoscope system according to the fourth embodiment. FIG. 15 is an explanatory diagram for explaining in detail the point of image output adjustment in the endoscope system according to the fourth embodiment.

  In the fourth embodiment, as shown in FIG. 14, first to third three camera parts 6 a to 6 c are arranged in the insertion part 5 of the endoscope 1, and the first and third camera parts on both sides are arranged. 6a and 6c are set to a wide angle (wide viewing angle), the second intermediate camera unit 6b is set to a narrow angle (narrow viewing angle), and the first and third camera units 6a and 6c are the first ones. , A first display image (monitoring display image) that captures a wide range including the gaze position on the subject is generated from the third individual captured image (wide-angle captured image), and the second camera unit 6b performs the second display. From the two individual captured images (narrow-angle captured images), a second display image (a display image for gazing) that captures a narrow range including the gaze position on the subject is generated.

  In the fourth embodiment, two monitors 3 for the operator and the assistant are provided, and the first display image (monitoring display image) is displayed on the monitor 3 for the assistant, so that the operator A second display image (a display image for gazing) is displayed on the monitor 3 for use.

  In particular, in the fourth embodiment, as shown in FIG. 15, the first and third individual captured images by the first and third camera units 6a and 6c are joined to form one continuous primary composite image. The first combined image is generated as a first display image (monitoring display image), or a part thereof is cut out in a wide range to generate the first display image. The first and third camera units 6a and 6c are set so that the imaging ranges partially overlap, and the first and third individual captured images are overlapped by the first and third camera units 6a and 6c. The first and third individual captured images are joined using the portion.

  Here, when the first display image is generated by cutting out a part of the primary composite image, the position of the range to be cut out from the primary composite image is changed according to the observer's instruction, as in the above embodiment. By doing so, it is possible to perform a pseudo pan and tilt operation on the first display image, and by changing the size of the range to be cut out from the primary composite image, the first display image is simulated. Zoom operation can be performed.

  In the fourth embodiment, the second individual captured image (narrow-angle captured image) by the second camera unit 6b and the primary composite image are combined, that is, the second individual captured image on the primary composite image. A part obtained by imaging the same area as the captured image is detected, and a process of embedding the second individual captured image in the part is performed to generate one continuous secondary composite image, and the second secondary composite image is generated from the secondary composite image. A second display image (a display image for gaze) is generated by cutting out a range including the individual captured images. That is, since the second individual captured image (narrow angle captured image) has a narrow imaging range and generally has a high resolution, the consistency of the position and the enlargement ratio of the display area with the primary composite image is achieved by resolution conversion or the like. Take. At this time, the second display image is cut out so that the second individual captured image is positioned at the approximate center of the second display image.

  Here, in the same manner as in the above-described embodiment, pseudo pan and tilt operations are performed on the second display image by changing the position of the range to be cut out from the secondary composite image in accordance with an instruction from the observer. In addition, by changing the size of the range cut out from the secondary composite image, a pseudo zoom operation can be performed on the second display image. At this time, the direction of panning and tilting may be designated by looking at the screen of the assistant's monitor 3 that displays a wide range of the subject.

  In the fourth embodiment, when the pan and tilt display operations are performed on the second display image (display image for gaze), the second individual captured image (narrow angle) by the second camera unit 6b is used. The line-of-sight direction of the second camera unit 6b is controlled so that the (captured image) is positioned approximately at the center of the second display image.

  Next, the controller 2 according to the fourth embodiment will be described. FIG. 16 is a functional block diagram illustrating a schematic configuration of a controller according to the fourth embodiment.

  In the fourth embodiment, similarly to the second embodiment, the image output adjustment unit 32 of the controller 2 includes an image composition unit 42 and an image cutout unit 43.

  In the image composition unit 42, processing for generating a primary composite image by joining the first and third individual captured images (wide-angle captured images) by the first and third camera units 6a and 6c is performed. Further, a process of generating a secondary composite image by combining the second individual captured image (narrow-angle captured image) by the second camera unit 6b with the primary composite image is performed.

  The image cutout unit 43 performs a process of cutting out a required display range from the primary composite image and generating a first display image (monitoring display image). In addition, a process of generating a second display image (a display image for gazing) by cutting out a required display range from the secondary composite image is performed.

  In addition, in the line-of-sight direction setting unit 35 of the controller 2, the imaging range of the second camera unit 6 b is set for the gaze according to the instruction of the observer for moving the display area of the display image for gaze and the pan and tilt instructions. In the display image area, in particular, the process of setting the line-of-sight direction of the second camera unit 6b is performed so as to be positioned at the approximate center of the display image for gazing, and based on the line-of-sight direction set here, The camera control unit 34 controls the mirror rotation actuator 14 and the camera rotation actuator 15 that constitute the line-of-sight adjustment mechanism of the second camera unit 6b.

  In the fourth embodiment, two monitors 3 are provided, and the display image for gaze and the display image for monitoring are displayed separately on the two monitors 3, but only one monitor 3 is provided. The display image for monitoring and the display image for monitoring may be switched and displayed on the one monitor 3, or the display image for monitoring and the display image for monitoring may be displayed side by side on one display screen. .

  In the fourth embodiment, the two camera units 6a and 6c are set to a wide viewing angle, and the one camera unit 6b is set to a narrow viewing angle. However, two or more camera units are set to a narrow viewing angle. In addition, only one camera unit or three or more camera units set to a wide viewing angle may be provided.

  In the fourth embodiment, the display image for gazing is generated by cutting out the range including the narrow-angle captured image and its peripheral portion from the secondary composite image, but the narrow-angle captured image is used as it is for gazing. Alternatively, a display image for gaze may be generated by cutting out a part of the narrow-angle captured image. In this case, since the second individual captured image (narrow-angle captured image) has a narrow imaging range and generally has a high resolution, the consistency of the position and the enlargement ratio of the display area with the primary composite image by resolution conversion is high. It is not necessary to perform the process of taking

  As described above, in the fourth embodiment, a display image for gazing in which a narrow range including a gazing position on the subject is captured from a narrow-angle captured image by the second camera unit 6b set to a narrow viewing angle, Since a display image for monitoring showing a wide range including a gaze position on the subject is generated from wide-angle captured images by the first and third camera units 6a and 6c set to a wide viewing angle, high resolution is achieved. It is possible to simultaneously obtain a display image for gazing and a display image for monitoring that captures a wide range. Since the gaze position on the subject is displayed as a high-resolution image, it is possible to observe the gaze position in detail, and at the same time, since the peripheral portion of the gaze position is displayed in a wide range, the peripheral portion is widely observed. be able to.

  In the fourth embodiment, a display image for gaze and a display image for monitoring are separately displayed on a plurality of monitors 3 provided for each observer. An observer who needs to observe a narrow area on the subject in detail can view a display image for gazing on the dedicated monitor 3, and at the same time, a wide area on the object such as an assistant for endoscopic surgery can be seen. An observer who needs to observe can view a display image for monitoring on the dedicated monitor 3.

  In the fourth embodiment, a plurality of camera units 6a and 6c are set to have a wide viewing angle, and a plurality of wide-angle captured images from these camera units 6a and 6c are joined to generate one continuous composite image. Since all or part of the composite image is used as a display image for monitoring, a wide range and high resolution display image for monitoring can be acquired.

  In the fourth embodiment, a synthesized image obtained by synthesizing the narrow-angle captured image and the wide-angle captured image is generated, and a range including the narrow-angle captured image is cut out from the synthesized image to generate a display image for gazing. As a result, it is possible to observe the portion of the subject on which the observer is interested in a high-resolution image and at the same time the peripheral portion.

  Further, in the fourth embodiment, a line-of-sight adjustment mechanism that individually changes the line-of-sight directions of the plurality of camera units 6a to 6c is provided, and is narrow according to an instruction from the observer to move the display area of the display image for gaze. Based on the line-of-sight direction, the line-of-sight direction of the second camera unit 6b is set so that the imaging range of the second camera unit 6b set to the viewing angle is located in the region of the display image for gazing. Therefore, the line-of-sight adjustment mechanism of the second camera unit 6b is controlled, so that a high-resolution narrow-angle captured image by the second camera unit 6b is always located in the display image for gaze, and the display image for gaze Even if the display area is moved, an important part on the subject can be observed with a high resolution image.

  Next, modified examples of the endoscope system according to the first to fourth embodiments will be described.

  FIG. 17 is a side view showing a modification of the insertion portion 5 of the endoscope 1. Here, similarly to the above-described example, a plurality of camera units 6a to 6e are provided side by side in the insertion unit 5. In particular, here, the camera unit 6a to 6e in the insertion unit 5 is positioned at the base end side. A bent portion 71 is provided.

  If comprised in this way, when inserting the insertion part 5 in the inside of observation object, the operation | work which inserts the insertion part 5 will become easy by setting it as a straight state as shown to FIG. 17 (A). Then, when the insertion portion 5 is inserted into the observation target and the inside of the observation target is observed, a straight state as shown in FIG. ), The inside of the observation target can be freely observed.

  FIG. 18 is an explanatory diagram illustrating a modification of the endoscope system according to the first embodiment. Here, a plurality of camera units 81 a to 81 c are provided in the insertion unit 5 of the endoscope 1, and the camera units 81 a to 81 c differ from the above-described embodiment in that the line of sight mechanically adjusts the line-of-sight direction. The adjustment mechanism, that is, the pan operation mechanism and the tilt operation mechanism are not provided, and the line-of-sight directions of the plurality of camera units 81a to 81c are constant.

  With such a configuration, when a plurality of camera units 81a to 81c acquire captured images in which a gaze object is captured, and a region including the gaze object is cut out from the captured image to generate a display image, the plurality of camera units 81a to 81c are generated. As in the first embodiment, a plurality of display images having different line-of-sight directions can be acquired by adjusting the cutout in the image output adjustment unit 32 without adjusting the line-of-sight direction.

  In this case, the image output adjustment unit 32 detects a region of the gaze position from each captured image in accordance with an instruction from the observer who designates the gaze position, and cuts out a range including the region of the gaze position from the captured image. You can do it.

  As mentioned above, although this invention was demonstrated based on specific embodiment, these embodiment is an illustration to the last, Comprising: This invention is not limited by these embodiment. In addition, all the constituent elements of the endoscope system according to the present invention shown in the above embodiment are not necessarily essential, and can be appropriately selected as long as they do not depart from the scope of the present invention.

  For example, in each of the first to fourth embodiments, the characteristic configuration has been described. However, the configurations described in the first to fourth embodiments can be appropriately combined. For example, the first embodiment is a viewpoint movement mode, the second embodiment is a high resolution mode, the third embodiment is an ultra-wide angle mode, and the fourth embodiment is a combined wide-angle / narrow-angle mode. Switch accordingly. In this case, by adopting a configuration in which the insertion portion is bent, the super wide angle mode and the other operation modes of the third embodiment can be realized by the same system. Further, if a zoom function is added to the camera unit, the viewing angle (view angle) can be changed, so the wide angle / narrow angle combined mode of the fourth embodiment and the other operation modes are realized in the same system. be able to.

  In each of the first and second embodiments, five camera units are arranged, and in each of the third and fourth embodiments, three camera units are arranged. However, the number of camera units is limited to this. However, it is possible to arrange two, four, or six or more.

  In each of the first to fourth embodiments, a plurality of camera units are arranged side by side in the longitudinal direction of the insertion unit. However, the arrangement of the camera units is limited to such a one-dimensional configuration. Alternatively, a plurality of camera units can be arranged two-dimensionally.

  In each of the first to fourth embodiments, a two-dimensional image is displayed on a monitor. However, a three-dimensional image is generated from images captured by a plurality of camera units, and the three-dimensional image is converted into a three-dimensional image. You may make it display on a monitor. In this way, a subject such as an organ can be observed three-dimensionally, so that the efficiency of surgery and examination can be improved. In this case, three-dimensional display from two viewpoints can be performed using images captured by two camera units, and three-dimensional display from multiple viewpoints can be performed using images captured by three or more camera units. As the three-dimensional display method, various methods such as a glasses type, a naked eye type, and a head mounted display can be adopted.

  In each of the first to fourth embodiments, the plurality of camera units are provided with a line-of-sight adjustment mechanism (pan operation mechanism and tilt operation mechanism) that changes the line-of-sight direction. However, the camera unit includes a line-of-sight adjustment mechanism. A configuration not provided is also possible.

  That is, in the first embodiment, as shown in FIG. 18, the same function can be realized even with a configuration without a line-of-sight adjustment mechanism by adjusting the cutout. Alternatively, a point at a predetermined distance from the camera unit may be set in advance as a gaze position, and a plurality of camera units may be set so that each gaze direction is directed to the gaze position.

  In the second to fourth embodiments, when there is no line-of-sight adjustment mechanism, alignment processing for setting the line-of-sight direction so that an appropriate overlapping portion is generated in the individual captured image cannot be performed. If it is used within the range in which the above occurs, the same function can be realized even in a configuration without a line-of-sight adjustment mechanism.

  In each of the first to fourth embodiments, the line-of-sight adjustment mechanism is provided in all of the plurality of camera units. However, a configuration in which the line-of-sight adjustment mechanism is provided in a part of the plurality of camera units is also possible. For example, in the second to fourth embodiments, if a line-of-sight adjustment mechanism is provided in the remaining camera units except for one camera unit, the alignment for setting the line-of-sight direction so that an appropriate overlapping portion is generated in the individual captured image Can be processed.

  The endoscope system according to the present invention has an effect that an appropriate image can be easily displayed on a display device without using a plurality of endoscopes and without moving the endoscope. An endoscope that images a subject inside the target, a control device that controls imaging performed by the endoscope, and that processes a captured image output from the endoscope, and an image that is output from the control device It is useful as an endoscope system including a display device that displays

1 Endoscope 2 Controller (control device)
3 Monitor (display device)
4 indication detector (indication detection device)
DESCRIPTION OF SYMBOLS 5 Insertion part 6a-6e Camera part 12 Imaging part 13 Movable mirror 14 Mirror rotation actuator 15 Camera rotation actuator 31 Image processing part 32 Image output adjustment part 33 Display control part 34 Camera control part 35 Gaze direction setting part 36 Gaze direction setting part 37 Gaze position tracking unit 38 Instruction content acquisition unit 41 Image switching unit 42 Image composition unit 43 Image cutout unit 51 Image overlap detection unit 61 Bending unit 63 Bending actuator 65 Bending angle setting unit 66 Bending control unit

Claims (21)

An endoscope for imaging an object within the observation target;
A control device for controlling imaging performed by the endoscope and processing a captured image output from the endoscope;
A display device for displaying an image output from the control device;
An endoscope system comprising:
The endoscope includes a plurality of camera units in an insertion unit that is inserted into an observation target,
The controller is
An image output adjustment unit that generates a display image by performing at least one of switching and synthesis on a plurality of captured images by each of the plurality of camera units,
An endoscope system, wherein the display image generated by the image output adjustment unit is displayed on the display device.   The endoscope system according to claim 1, wherein the plurality of camera units are set such that each line-of-sight direction is directed to the same gaze position on the subject.   The endoscope system according to claim 2, wherein the image output adjustment unit switches a plurality of the captured images and outputs the images as the display image in accordance with an instruction from an observer. The endoscope further includes a line-of-sight adjustment mechanism that individually changes the line-of-sight direction of the camera unit,
The controller is
A line-of-sight direction setting unit that sets the line-of-sight direction of the camera unit so that the line-of-sight direction of each of the plurality of camera units is directed to the same gaze position in accordance with an instruction of an observer that specifies the gaze position;
A camera control unit that controls the line-of-sight adjustment mechanism based on the line-of-sight direction set by the line-of-sight direction setting unit;
The endoscope system according to claim 2, further comprising:   The endoscope system according to claim 1, wherein the plurality of camera units are set in a line-of-sight direction so as to capture regions that are shifted from each other on the subject.   The said insertion part has a bending part between the said several camera parts, The bending direction of this bending part was made to change the gaze direction of the said several camera part, It is characterized by the above-mentioned. The endoscope system described in 1.   The image output adjustment unit generates a continuous composite image by joining a plurality of the captured images, and uses all or part of the composite image as the display image. The endoscope system according to claim 6.   The image output adjustment unit generates the display image by cutting out a range according to the display condition from the composite image in accordance with an instruction from an observer who specifies a display condition regarding the display image. Item 8. The endoscope system according to Item 7. The display condition relates to pan and tilt for moving a display area of the display image,
The endoscope according to claim 8, wherein the image output adjustment unit causes the display image to perform pseudo panning and tilting operations by changing a position of a range to be cut out from the composite image. system. The display condition relates to a zoom for enlarging and reducing a display area of the display image,
The endoscope system according to claim 8, wherein the image output adjustment unit causes the display image to perform a pseudo zoom operation by changing a size of a range cut out from the composite image. A plurality of display devices for each of a plurality of observers,
The said control apparatus displays the arbitrary ranges of the said synthesized image separately on the said several display apparatus according to an observer's instruction | indication, The inside in any one of Claim 5-10 characterized by the above-mentioned. Endoscopic system. Among the plurality of camera units, at least one of the camera units and the remaining camera units are set to have different viewing angles,
The image output adjustment unit generates a display image for gaze that captures a narrow range including a gaze position on a subject from a narrow-angle captured image by the camera unit set to a narrow viewing angle, and sets the wide viewing angle The endoscope system according to claim 1, wherein a display image for monitoring in which a wide range including a gaze position on a subject is captured is generated from the wide-angle captured image obtained by the camera unit. A plurality of display devices for each of a plurality of observers,
The endoscope system according to claim 12, wherein the control device causes the display image for gaze and the display image for monitoring to be displayed on another display device. The plurality of camera units are set to a wide viewing angle,
The image output adjustment unit generates a single continuous composite image by joining a plurality of wide-angle captured images from the plurality of camera units set to a wide viewing angle, and all or a part of the composite image is generated. The endoscope system according to claim 12 or 13, wherein the display image is used for the monitoring.   The image output adjustment unit generates a composite image obtained by combining the narrow-angle captured image and the wide-angle captured image, cuts out a range including the narrow-angle captured image from the composite image, and displays the display image for gaze The endoscope system according to any one of claims 12 to 14, wherein the endoscope system is generated. The endoscope further includes a line-of-sight adjustment mechanism that individually changes the line-of-sight direction of the camera unit,
The controller is
In response to an observer's instruction to move the display area of the display image for gaze, the imaging range of the camera unit set to a narrow viewing angle is positioned within the area of the display image for gaze, A line-of-sight direction setting unit that sets a line-of-sight direction of the camera unit set to a narrow viewing angle;
A camera control unit that controls the line-of-sight adjustment mechanism based on the line-of-sight direction set by the line-of-sight direction setting unit;
The endoscope system according to any one of claims 12 to 15, further comprising: An endoscope for imaging an object within the observation target;
A control device for controlling imaging performed by the endoscope and processing a captured image output from the endoscope;
A display device for displaying an image output from the control device;
An endoscope system comprising:
Further comprising an instruction detection device for detecting an instruction operation of an observer performed while watching the display screen of the display device;
The endoscope includes a plurality of camera units in an insertion unit that is inserted into an observation target,
The controller is
An instruction content acquisition unit that analyzes an instruction operation detected by the instruction detection device and acquires an instruction content related to a display screen of the display device;
Based on the instruction content acquired by the instruction content acquisition unit, image output for generating a display image by performing at least one of switching and combining adjustment on a plurality of captured images by each of the plurality of camera units An adjustment unit;
With
An endoscope system, wherein the display image generated by the image output adjustment unit is displayed on the display device. The endoscope further includes a line-of-sight adjustment mechanism that individually changes the line-of-sight direction of the camera unit,
The instruction content acquisition unit analyzes the instruction operation detected by the instruction detection device, acquires instruction content related to the line-of-sight direction of the camera unit,
The controller is
18. The endoscope system according to claim 17, further comprising a camera control unit that controls the line-of-sight adjustment mechanism based on the instruction content acquired by the instruction content acquisition unit.   The instruction detection device detects at least one of a gesture performed by an observer, a voice uttered by the observer, a gaze of the observer, a touch operation performed by the observer, and a button operation performed by the observer. The endoscope system according to claim 17 or 18.   The plurality of camera parts are arranged side by side in the longitudinal direction of the insertion part, and the line-of-sight direction is set in a direction substantially perpendicular to the arrangement direction. The endoscope system described.   Each of the plurality of camera units includes a pan operation mechanism that changes the line-of-sight direction in the longitudinal direction of the insertion unit, and a tilt operation mechanism that changes the line-of-sight direction in a direction orthogonal to the longitudinal direction of the insertion unit. The endoscope system according to any one of claims 1 to 20, wherein

Images