JP6827516B2 - Endoscope system and how to drive the endoscope system - Google Patents

Description

The present invention relates to an endoscope system capable of observing a direct view and a side view, and a method for driving the endoscope system.

In the medical field, it is common to make a diagnosis using an endoscope system including a light source device, an endoscope, and a processor device. In an endoscopic system, the light source device produces illumination light. The endoscope has a flexible insertion portion, and by inserting the insertion portion into the subject, for example, an image sensor mounted on the tip portion of the insertion portion (hereinafter referred to as the tip portion) is used to image an observation target. To do. Then, the processor device generates an image to be observed and displays it on the monitor.

The endoscopes used in the conventional endoscope system include a direct observation type that images an observation target in the direction of the tip of the tip (that is, the direction of the front along the insertion direction of the insertion part) and a side direction of the tip. A side-view observation type that images an observation target located (that is, in the outer peripheral direction of the insertion portion) is known. Further, in recent years, an endoscope capable of observing both the tip direction and the side direction of the tip portion has been known (Patent Document 1, Patent Document 2, and Patent Document 3).

International Publication No. 2011/055641 Japanese Unexamined Patent Publication No. 2016-007745 Japanese Unexamined Patent Publication No. 2012-245157

In an endoscope capable of direct-view observation and lateral observation, a direct-view observation image obtained by imaging an observation object in the tip direction of the tip and a lateral view obtained by imaging an observation object in the lateral direction of the tip. An observation image is obtained. Therefore, when the direct-view observation image and the side-view observation image are displayed, the observable range is widened as compared with the case where only one of the direct-view observation image and the side-view observation image is displayed. However, simply displaying the direct-view observation image and the side-view observation image may not always be easy to use in observation or diagnosis.

The present invention improves the way in which the direct-view observation image and the side-view observation image are displayed when using an endoscope capable of direct-view observation and side-view observation, and as a result, it is easy to use in observation or diagnosis. It is an object of the present invention to provide a method for driving an endoscopic system and an endoscopic system.

The endoscopic system of the present invention has an insertion portion to be inserted into an observation target, a direct observation observation portion having a visual field in the direction of the tip of the insertion portion, a lateral observation observation portion having a visual field in the lateral direction of the insertion portion, and an insertion portion. An endoscope having a protruding portion that protrudes and forms a blind spot in the field of view of the lateral observation unit, a direct observation image is acquired using the direct observation unit, and a lateral observation image is obtained using the lateral observation unit. At least one of a direct observation image or a side view observation image using the image acquisition unit for acquiring the image, the distance measuring unit for measuring the distance between the tip of the endoscope and the observation target, and the distance between the tip and the observation target . One side is subjected to conversion processing to convert the aberrations of the direct-view observation image and the side-view observation image into an image that can be connected, and the direct-view observation image in which the aberrations can be connected by the conversion processing of the aberration connection portion. And a display unit for displaying a side view observation image.

The conversion process is preferably an affine transformation.

It is preferable that the aberration connecting portion converts the distortion of the direct-view observation image and the side-view observation image into a connectable image.

Yield difference connections, using the distance of the observation target and the distal end portion, it is preferable to adjust or change the parameters used in the conversion process.

The distance measuring unit preferably measures the distance between the tip portion and the observation target at the boundary between the visual field of the direct visual observation unit and the visual field of the lateral observation unit.

It is preferable that the parameters of the aberration connecting portion are adjusted or changed according to the average value of the distance between the tip portion and the observation target.

It is preferable that the aberration connecting portion adjusts or changes the parameter for each position where the direct-view observation image and the side-view observation image are connected.

It is preferable that the parameters of the aberration connecting portion are adjusted or changed by using the relative angle between the tip portion and the observation target.

For the aberration connection portion, it is preferable to obtain the relative angle between the tip portion and the observation target by using the distribution of the distance between the tip portion and the observation target.

It is preferable that the parameters of the aberration connecting portion are adjusted or changed according to the average value of the relative angles between the tip portion and the observation target.

It is preferable that the aberration connecting portion adjusts or changes the parameter for each position where the direct-view observation image or the side-view observation image is connected.

The driving method of the endoscopic system of the present invention includes an insertion portion to be inserted into an observation target, a direct observation observation portion having a visual field in the direction of the tip of the insertion portion, and a lateral observation portion having a visual field in the lateral direction of the insertion portion. An endoscope having a protruding portion that protrudes from the insertion portion and forms a blind spot in the field of view of the lateral observation portion, and a direct observation image are acquired using the direct observation portion, and the side using the lateral observation portion. An endoscope having an image acquisition unit for acquiring a visual observation image, a distance measuring unit for measuring the distance between the tip of the endoscope and an observation target, and a display unit for displaying a direct observation image and a side observation image. In the method of driving the system, the aberration connecting portion performs conversion processing on at least one of the direct-view observation image and the side-view observation image by using the distance between the tip portion and the observation target, and the direct-view observation image and the side-view observation image. The display unit includes a step of converting the aberration into an image that can be connected, and a step of displaying a direct-view observation image and a side-view observation image in which the display unit is in a state where the aberration can be connected by the conversion process of the aberration connection unit.

The present invention has improved the display method of the direct-view observation image and the side-view observation image when using an endoscope capable of direct-view observation and side-view observation, and thus is an endoscope that is easy to use in observation or diagnosis. A method of driving a system and an endoscopic system can be provided.

It is an external view of an endoscope system. It is an external perspective view of the tip part. It is a front view of the tip part. It is sectional drawing of a part of the 1st protrusion. It is sectional drawing of the 2nd protrusion. It is a block diagram of an endoscope system. It is a block diagram of a display control unit. This is a display image in standard mode. This is a display image in the direct view magnifying mode. This is a display image in the side view magnified mode. This is a display image in the direct-view magnifying mode that substantially displays only the direct-view observation image. This is a display image in the side view magnifying mode that substantially displays only the side view observation image. It is a block diagram of the endoscope system of 2nd Embodiment. It is a flowchart which shows the operation of 2nd Embodiment. It is explanatory drawing which shows the insertion direction and the removal direction. It is a graph which shows the relative display ratio of the direct view observation image and the side view observation image. It is a graph which shows the relative display ratio of the direct view observation image and the side view observation image. It is a flowchart of the modification which detects a curvature. It is explanatory drawing in the case of changing the direction of the tip portion. It is explanatory drawing in the case that the tip portion is directed in the insertion direction. It is a graph which shows the relative display ratio of the direct view observation image and the side view observation image. It is a graph which shows the relative display ratio of the direct view observation image and the side view observation image. It is a block diagram of the display control part of 3rd Embodiment. It is explanatory drawing which reduces the 1st blind spot part. It is explanatory drawing which reduces the 2nd blind spot part. This is a display image in which the second blind spot portion is reduced and hidden. It is a block diagram of the display control part of 4th Embodiment. It is explanatory drawing which adjusts the display position of the direct view observation image according to the direction of the tip part. It is explanatory drawing which adjusts the display position of the direct view observation image according to the direction of a doctor who wears a head-mounted display. It is explanatory drawing which shows the blind spot between a direct view observation part and a side view observation part. This is a display screen of a monitor that displays a blind spot between the direct observation unit and the side observation unit. It is a display image which displays a blind spot between a direct view observation part and a side view observation part. It is explanatory drawing which shows the overlap of the field of view of the direct view observation part and the side view observation part. It is explanatory drawing which shows the double display due to the overlapping visual field. It is a block diagram of the display control unit of the sixth embodiment. It is explanatory drawing which shows the operation of the overlap area detection part and overlap area removal part. It is explanatory drawing which shows the operation of the overlap area detection part and overlap area removal part. It is a block diagram of the display control part of the modification of 6th Embodiment. It is explanatory drawing which shows the distortion of the direct view observation image and the side view observation image. It is a block diagram of the display control part of 7th Embodiment. It is explanatory drawing which shows the operation of the aberration connection part. It is a block diagram of the modification of 7th Embodiment. It is explanatory drawing which shows the position to measure a distance. It is a flowchart which shows the operation of the automatic exposure control. It is a flowchart which shows the operation of the automatic exposure control of a modification. It is a block diagram of the processor apparatus 16 of 9th Embodiment. It is a block diagram of the display control unit 98 of the tenth embodiment. It is explanatory drawing which shows the mutual relationship between the display area in a monitor, a direct view observation image and a side view observation image. This is a display image in which the first blind spot portion is hidden by offsetting the direct view observation image and the side view observation image. It is explanatory drawing which shows the method of hiding at least a part of the 1st blind spot part. It is explanatory drawing which shows the method of hiding at least a part of the 2nd blind spot part. It is a block diagram of the display control part when the 1st blind spot part or the 2nd blind spot part is hidden according to the movement of the insertion part (tip part). It is a graph which shows the relationship between the insertion / removal speed of the insertion part, and the ratio of hiding the 1st blind spot part or the 2nd blind spot part. It is a graph which shows the relationship between the insertion / removal speed of the insertion part, and the ratio of hiding the 1st blind spot part or the 2nd blind spot part. It is a perspective view of the tip part which attached the tip cap. This is a display image showing the outline of the tip cap. This is a display image in which the outline of the tip cap and the portion of the through hole are masked. It is the top view of the tip part which attached the auxiliary member. It is a bottom view of the tip part which attached the auxiliary member which is locked by a claw member. It is the bottom view of the tip part which has the bottom surface observation part. It is a display image having a margin area. This is a display image in which an information display area is provided in the margin area. It is a block diagram of the processor apparatus provided with the focusing angle of view detection part. It is a display image which displays a deformed side view observation image in a margin area. It is a display image which displays a deformed side view observation image in a margin area. This is a display image in which an information display area is provided on the mask of the first blind spot portion. It is a block diagram of the processor apparatus which has a 3D model making part. It is explanatory drawing which shows the operation of the 3D model creation part. It is explanatory drawing which shows the imaging surface of an image sensor.

[First Embodiment]
As shown in FIG. 1, the endoscope system 10 includes an endoscope 12 that captures an observation target, a light source device 14 that generates illumination light, and an image obtained by imaging the observation target (hereinafter referred to as an captured image). ) Is used to generate an observation image (hereinafter referred to as an observation image), a processor device 16 is provided, a monitor 18 is a display unit for displaying the observation image, and a console 19 is one of the user interfaces. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16 by using the universal cord 11. Further, the endoscope 12 is connected to a tank 17 for storing a cleaning liquid (for example, water) or the like by using a universal cord 11. A mechanism such as a pump for sending the cleaning liquid or the like of the tank 17 is provided in, for example, the light source device 14.

The endoscope 12 has an insertion portion 12a to be inserted into the subject, an operation portion 12b at the base end portion of the insertion portion 12a, a curved portion 12c on the tip end side of the insertion portion 12a, and a tip portion 12d. Have. When the angle knob 12e on the operation unit 12b is operated, the curved portion 12c is curved. As a result of the curved portion 12c being curved, the tip portion 12d is oriented in a desired direction.

In addition to the angle knob 12e, the operation unit 12b includes, for example, a cleaning switch 13 that ejects cleaning liquid from a nozzle at the tip portion 12d. When the cleaning switch 13 is pressed when the tip portion 12d becomes dirty due to contact with an observation target or the like, the cleaning liquid is ejected from the nozzle at the tip portion 12d toward at least a part of the tip portion 12d, and as a result, the tip portion is ejected. The portion of part 12d that is exposed to the cleaning liquid can be cleaned. In the endoscope system 10, the cleaning solution is a liquid such as water or a chemical solution. Further, in the present specification, the cleaning "liquid" is used for convenience, but for cleaning, the "cleaning liquid" includes a gas such as air ejected from a nozzle, a solid, or a mixture of substances having different phases. It shall be a waste.

As shown in FIGS. 2 and 3, the tip portion 12d of the insertion portion 12a to be inserted into the observation target further protrudes from the tip surface 21 of the tip portion 12d in the Z direction, which is the tip direction of the insertion portion 12a. It has two protrusions, one protrusion 31 and a second protrusion 32. The second protruding portion 32 projects from the tip surface 21 in the tip direction (Z direction) of the insertion portion 12a adjacent to the first protruding portion 31. Hereinafter, the direction in which the first protrusion 31 is located with respect to the second protrusion 32 is referred to as the Y direction, and the directions perpendicular to the Z direction and the Y direction are referred to as the X direction. Further, the positive side in the X direction of the insertion portion 12a, the tip portion 12d, the first protruding portion 31, or the second protruding portion 32 is referred to as "left", the negative side in the X direction is referred to as "right", and the positive side in the Y direction is referred to as "right". It is called "upper" and the negative side in the Y direction is called "lower". The positive side in the Z direction of the insertion portion 12a, the tip portion 12d, the first protruding portion 31, or the second protruding portion 32 is the "front" or the "tip (tip direction)", and the negative side in the Z direction is the "base end". (Base end direction) ".

The first protruding portion 31 has a substantially cylindrical shape as a whole, has a direct-view observation window 41A which is an observation window of the direct-view observation unit 41 at its tip, and has a side surface which is an observation window of the side-view observation unit 42. A visual observation window 42A is provided. The direct observation unit 41 has a field of view in the direction of the tip of the insertion unit 12a, and images an observation target in the direction of the tip of the insertion unit 12a. The direct observation unit 41 includes, for example, an image pickup lens, an image sensor, and the like. The optical member such as an image pickup lens or the transparent protective member that protects the optical member such as an image pickup lens constituting the direct-view observation unit 41 is exposed at the tip end (plane facing the Z direction) of the first protrusion 31. The exposed portion at the tip of the first protruding portion 31 is a direct viewing observation window 41A that takes in light incident from an observation target in the tip direction with respect to the insertion portion 12a.

The side view observation unit 42 has a field of view in the side surface direction of the insertion unit 12a, and images an observation target in the side surface direction of the insertion unit 12a. The side-view observation unit 42 includes, for example, an image pickup lens, an image sensor, and the like, like the direct-view observation unit 41. However, the optical member such as an image pickup lens or the transparent protective member that protects the optical member such as an image pickup lens constituting the side view observation unit 42 forms the side surface of the first protrusion 31 (the outer periphery of the first protrusion 31). It is exposed on the surface). The exposed portion on the side surface of the first protruding portion 31 is the side view observation window 42A that takes in the light incident from the observation target in the lateral direction with respect to the insertion portion 12a. In the endoscope 12 of the present embodiment, the lateral observation unit 42 is exposed over one circumference in the circumferential direction of the first protrusion 31 except for the joint portion between the first protrusion 31 and the second protrusion 32. , One strip-shaped side view window 42A is formed.

Further, the first protruding portion 31 has a first side-viewing illumination window 43A, which is an illumination window of the side-viewing illumination unit 43, in addition to the direct-viewing observation window 41A and the side-viewing observation window 42A. The side view illumination unit 43 emits illumination light from the first side view illumination window 43A toward the field of view of the side view observation unit 42. The side view illumination unit 43, for example, directs the light guide that guides the illumination light emitted by the light source device 14 and the illumination light that guides the illumination light to the tip portion 12d using the light guide toward the field of view of the side view observation unit 42. Includes optical members such as lenses or mirrors that diffuse and emit light. An optical member such as a mirror or a transparent protective member that protects the optical member such as a mirror that constitutes the side view illumination unit 43 is exposed on the side surface of the first protrusion 31. The portion exposed on the side surface of the first protruding portion 31 is the first side viewing illumination window 43A that emits illumination light in the side surface direction of the insertion portion 12a. In the endoscope 12 of the present embodiment, a part of the outer circumference of the first protruding portion 31 excluding the joint portion excluding the joint portion between the first protruding portion 31 and the second protruding portion 32 is a side view illumination window. It has become. Further, in FIG. 2, the first side view illumination window 43A is located on the right side surface of the first protrusion 31, but the illumination light is also emitted to the field of view of the side view observation unit 42 on the left side surface of the first protrusion 31. There is a second side visual illumination window 43B (see FIG. 3). The positions and sizes of the first side viewing illumination window 43A and the second side viewing illumination window 43B are symmetrical.

In the present embodiment, as shown in FIG. 4, the direct-view observation unit 41 and the side-view observation unit 42 include a common image pickup lens 61 and an image sensor 66. The image pickup lens 61 includes a front group lens 62, a mirror lens 63 formed by joining two lenses, and a rear group lens 64. The front surface of the front lens group 62 is exposed at the tip of the first protrusion 31. That is, the front surface of the front group lens 62 constitutes the direct observation window 41A of the direct observation unit 41. Further, the side surface of the mirror lens 63 is exposed on the side surface of the first protruding portion 31. Therefore, the side surface of the mirror lens 63 constitutes the side view observation window 42A of the side view observation unit 42.

The light incident from the observation target in the tip direction of the insertion portion 12a through the front group lens 62 is guided by the mirror lens 63 to the rear group lens 64. Then, an image is formed on the imaging surface of the image sensor 66 via the cover glass 67. As a result, the image pickup lens 61 and the image sensor 66 as the direct view observation unit 41 take an image of the observation target in the tip direction of the insertion unit 12a.

On the other hand, the light incident from the observation target in the side surface direction of the insertion portion 12a through the side surface of the mirror lens 63 is the light incident on the mirror lens 63 from the junction surface of the two lenses forming the mirror lens 63 and the front surface of the mirror lens 63. In and out, the light is sequentially reflected and guided to the rear lens group 64. Then, an image is formed on the imaging surface of the image sensor 66 via the cover glass 67. As a result, the image pickup lens 61 and the image sensor 66 as the side view observation unit 42 image the observation target in the side surface direction of the insertion unit 12a.

Further, the side view illumination unit 43 includes a light guide 71, a reflection member 72, and a filling member 73. The light guide 71 is optically connected to the light source device 14, and guides the illumination light emitted by the light source device 14. Then, it is emitted from the end face of the light guide 71 to the reflecting member 72 via the filling member 73. The reflecting member 72 diffuses the illumination light incident from the light guide 71 toward the side surface of the insertion portion 12a, and emits the illumination light to at least a range including the field of view of the side view observation unit 42. The filling member 73 is a protective member that protects the exit end surface of the light guide 71 and the reflective member 72, and is transparent. Further, the filling member 73 smoothly fills the groove portion formed between the light guide 71 and the reflecting member 72 along the side surface of the first protruding portion 31. Therefore, the filling member 73 constitutes the first side viewing illumination window 43A. The same applies to the second side visual illumination window 43B.

In the endoscope 12 of the present embodiment, the side view observation window 42A is provided on the tip end side of the side surface of the first protrusion 31, and the first side view illumination window 43A and the second side view illumination window 43B are provided. 1 Although it is provided on the base end side on the side surface of the protruding portion 31, the position and order of these are arbitrary. However, since it is easy to secure the field of view of the side view illumination unit 43 by preventing eclipse due to the tip surface 21 or the like, it is preferable to provide the side view observation window 42A on the side surface of the first protrusion 31 as close to the tip as possible.

The second protruding portion 32 has a nozzle for ejecting a cleaning liquid to clean the tip portion 12d. More specifically, the second protruding portion 32 has a nozzle 51 and a nozzle 52 (see FIG. 3) for ejecting the cleaning liquid toward the side view observation window 42A. The nozzle 51 is provided on the right side surface of the second protruding portion 32, and the nozzle 52 is provided on the left side surface of the second protruding portion 32. The nozzle 51 and the nozzle 52 have the same properties in that they are provided on the second protrusion 32 and clean the side view observation window 42A by ejecting the cleaning liquid toward the side view observation window 42A.

Further, the second protruding portion 32 has a nozzle 53 at the tip of the second protruding portion 32. The nozzle 53 cleans the direct-view observation window 41A by ejecting a cleaning liquid toward the direct-view observation window 41A, which is an exposed portion of the direct-view observation unit 41.

As shown in FIG. 5, the nozzle 53 is the outlet of the air supply / liquid supply channel 76. Therefore, when the cleaning liquid or the like stored in the tank 17 is sent out through the air supply liquid supply channel 76, it is ejected from the nozzle 53. The air supply liquid supply channel 76 communicates with the second protruding portion 32, the tip portion 12d, the insertion portion 12a, the universal cord 11, and the like. In the present embodiment, the air supply liquid supply channel 76 branches at the second protruding portion 32, the tip portion 12d, the insertion portion 12a, or the universal cord 11, and is common to other nozzles such as the nozzle 51 and the nozzle 52. I understand. Therefore, for example, if the cleaning switch 13 is pressed to feed the cleaning liquid to the air supply liquid supply channel 76, the cleaning liquid is ejected not only from the nozzle 53 but also from the nozzle 51 and the nozzle 52 at the same time. Therefore, the endoscope 12 can simultaneously clean the direct observation window 41A and the side observation window 42A with a simple operation.

In addition to the nozzle 51, the nozzle 52, and the nozzle 53, the second protruding portion 32 has a direct-view illumination window 54A, which is an illumination window of the direct-view illumination unit 54 that emits illumination light toward the field of view of the direct-view observation unit 41. .. The direct-view illumination unit 54, for example, directs the light guide 77 that guides the illumination light emitted by the light source device 14 and the illumination light that is guided to the tip portion 12d by the light guide 77 toward the field of view of the direct-view observation unit 41. It includes an illumination lens 78 and the like that diffuse and emit light (see FIG. 4). The illumination lens constituting the direct-view illumination unit 54 or the transparent protective member for protecting the illumination lens is exposed at the tip of the second protrusion 32. The portion exposed at the tip of the second protruding portion 32 is the direct-view illumination window 54A. In the present embodiment, the front surface of the illumination lens 78 is exposed to the tip of the second protrusion 32. Therefore, the front surface of the illumination lens 78 forms the direct illumination window 54A.

The tip surface 21 of the tip portion 12d has the first protrusion 31 and the second protrusion 32, as well as a direct illumination window 81A which is an illumination window of the direct illumination 81 and a forceps opening 82.

The direct-view illumination unit 81 emits illumination light toward the field of view of the direct-view observation unit 41, similarly to the direct-view illumination unit 54 in the second protrusion 32. Further, the direct-view illumination unit 81 directly observes, for example, a light guide 84 (see FIG. 6) that guides the illumination light emitted by the light source device 14, and an illumination light that is guided to the tip portion 12d by using the light guide 84. Includes an illumination lens or the like (not shown) that diffuses and emits light toward the field of view of unit 41. In the present embodiment, the light guide constituting the direct-view illumination unit 81 is connected to the light guide 77 constituting the direct-view illumination unit 54. Therefore, the light guide constituting the direct-view illumination unit 81 is substantially common with the light guide 77 constituting the direct-view illumination unit 54. Therefore, the direct-view illumination unit 54 and the direct-view illumination unit 81 simultaneously emit the same illumination light from their respective illumination windows. However, the light guide 77 that constitutes the direct-view illumination unit 54 and the light guide that constitutes the direct-view illumination unit 81 may have different amounts of emitted light due to differences in thickness and the like. The illumination lens that constitutes the direct-view illumination unit 81 or the transparent protective member that protects the illumination lens is exposed on the tip surface 21. The portion exposed on the tip surface 21 is the direct-view illumination window 81A.

The forceps opening 82 is an outlet for a treatment tool such as a forceps. When a treatment tool such as a forceps is inserted from the entrance (not shown) of the proximal end portion of the endoscope 12, the treatment tool may reach the forceps opening 82 via the forceps channel and the tip thereof may protrude from the forceps opening 82. it can. The forceps channel communicates with the tip portion 12d, the insertion portion 12a, and the operation portion 12b.

The side view observation window 42A is provided on the side surface of the first protruding portion 31, and the second protruding portion 32 is adjacent to the first protruding portion 31 in the direction from the tip surface 21 to the tip of the insertion portion 12a. Since it protrudes in the Z direction), the second protruding portion 32 is a protruding portion that forms a blind spot in the visual field of the lateral observation unit 42. That is, due to the arrangement of each part, the side view observation unit 42 cannot image the observation target in a part of the angle range because the second protrusion 32 is included in a part of the field of view.

As shown in FIG. 6, the light source device 14 includes a light source 91 that generates illumination light and a light source control unit 92 that controls the light source 91. The light source 91 is, for example, a plurality of LEDs (Light Emitting Diodes) that can be independently controlled and have different wavelengths or wavelength ranges of emitted light. As the light source 91, another semiconductor light source such as LD (Laser Diode) may be used instead of the LED. A semiconductor light source and a phosphor or the like that emits light of another color using the light emitted by the semiconductor light source as excitation light may be used in combination. A lamp light source such as a xenon lamp may also be used for the light source 91. Further, the light source 91 may be formed by combining a semiconductor light source, a semiconductor light source and a phosphor, and an optical filter for adjusting a wavelength band or a spectral spectrum together with a lamp light source. For example, by using a white LED in combination with an optical filter, it is possible to emit a plurality of types of illumination light.

The light source control unit 92 controls each of the LEDs and the like constituting the light source 91 to turn on, off, and the amount of light according to the drive timing of the image sensor 66. In particular, in the case of generating one observation image using a plurality of captured images (that is, the multi-frame observation mode), the light source control unit 92 uses a plurality of observation images to be generated as a result of control of LEDs and the like. The wavelength band or spectral spectrum of the illumination light can be changed for each imaging frame for obtaining an captured image. Note that lighting means that the image sensor 66 emits light with an amount of light that allows the observation target to be imaged (that is, the image of the observation target can be visually recognized in the observation image). The extinguishing includes not only completely stopping the light emission but also dimming the light amount to such an extent that the observation target cannot be imaged by the image sensor 66. When each light source included in the light source 91 is a semiconductor light source, the light source control unit 92 controls turning on, off, and the amount of light of each light source by pulse modulation control.

The illumination light generated by the light source 91 is incident on the light guide 93. The light guide 93 is inserted from the light source device 14 into the endoscope 12 and the universal cord, and propagates the illumination light to the tip portion 12d of the endoscope 12. The light guide 93 is branched into at least a light guide 77 that constitutes the direct-view illumination unit 54, a light guide 84 that constitutes the direct-view illumination unit 81, and a light guide 71 that constitutes the side-view illumination unit 43. Propagates the illumination light. A multimode fiber can be used as each light guide after branching of the light guide 93 and the light guide 71. As an example, a fine fiber cable having a core diameter of 105 μm, a clad diameter of 125 μm, and a diameter of φ0.3 to 0.5 mm including a protective layer as an outer skin can be used.

The processor device 16 includes a control unit 96, an image acquisition unit 97, and a display control unit 98. The control unit 96 is a CPU (Central Processing Unit) or the like that collectively controls the endoscope system 10. The control unit 96 performs synchronous control for matching the imaging timing of the image sensor 66 with the light emission timing of each LED or the like constituting the light source 91, for example. The light emission timing of each LED or the like constituting the light source 91 is controlled via the light source control unit 92. Further, the control unit 96 performs automatic exposure control (AE (Auto Exposure) control) that automatically controls the exposure. In the present embodiment, the control unit 96 controls the AE by driving the image sensor 66 at a fixed timing and adjusting the amount of light emitted from each LED or the like (that is, the amount of illumination light) constituting the light source 91. do. At the time of AE control, the control unit 96 receives the direct-view observation image 111 (see FIG. 8) and the side-view observation image 112 (see FIG. 8), or the direct-view observation image 111 or the side-view observation image 112 from the image acquisition unit 97. Either one is acquired, and the amount of light emitted from each LED or the like constituting the light source 91 is determined using the acquired image.

The image acquisition unit 97 acquires the direct observation image 111 by using the direct observation unit 41, and acquires the lateral observation image 112 by using the lateral observation unit 42. In the present embodiment, since the direct view observation unit 41 and the side view observation unit 42 share the image pickup lens 61 and the image sensor 66, the image acquisition unit 97 is an image captured by the image sensor 66 (hereinafter referred to as an image captured image). ), And various image processing and the like are applied to the acquired captured image to obtain a captured image including the direct view observation image 111 and the side view observation image 112. That is, a part of the captured image obtained from the image sensor 66 is a direct view observation image 111, and another part of the captured image obtained from the image sensor 66 is a side view observation image 112.

The image acquisition unit 97 functions as, for example, a DSP (Digital Signal Processor) and a noise reduction unit.

The image acquisition unit 97 performs various processes such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing on the acquired captured image as necessary. This is a function of the image acquisition unit 97 as a DSP. The defect correction process is a process for correcting the pixel value of the pixel corresponding to the defective pixel of the image sensor 66. The offset process is a process of reducing the dark current component from the captured image subjected to the defect correction process and setting an accurate zero level. The gain correction process is a process of adjusting the signal level of each captured image by multiplying the offset processed captured image by a gain. The linear matrix processing is a process for improving the color reproducibility of the captured image after the offset processing, and the gamma conversion processing is a process for adjusting the brightness and saturation of the captured image after the linear matrix processing. The demosaic process (also referred to as isotropic processing or simultaneous processing) is a process of interpolating the pixel values of the missing pixels, and is applied to the captured image after the gamma conversion process. The missing pixel is a pixel having no pixel value because pixels of other colors are arranged in the image sensor 66 due to the arrangement of color filters. For example, since the B image is an captured image obtained by capturing an observation target in the B pixel, the pixels at the positions corresponding to the G and R pixels of the image sensor 66 have no pixel value. The demosaic process interpolates the B image to generate pixel values of pixels at the positions of the G pixel and the R pixel of the image sensor 66. The YC conversion process is a process of converting the image after the demosaic process into the luminance channel Y, the color difference channel Cb, and the color difference channel Cr.

The image acquisition unit 97 performs noise reduction processing on the luminance channel Y, the color difference channel Cb, and the color difference channel Cr by using, for example, a moving average method or a median filter method. The conversion unit 59 reconverts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into captured images of each color of BGR. This is a function of the image acquisition unit 97 as a noise reduction unit.

The display control unit 98 acquires the direct-view observation image 111 and the side-view observation image 112 from the image acquisition unit 97. Then, the display control unit 98 uses either the direct-view observation image 111 and the side-view observation image 112, or the direct-view observation image 111 or the side-view observation image 112, and displays an image for display (hereinafter, a display image). (See Fig. 8 etc.) is generated. The display control unit 98 displays the generated display image on the monitor 18. In the present embodiment, unless otherwise specified, the display control unit 98 uses both the direct-view observation image 111 and the side-view observation image 112 to generate and display a display image.

Further, the display control unit 98 masks the portion of the side view observation image 112 corresponding to the blind spot, and determines the relative display ratio between the direct view observation image 111 and the side view observation image 112 on the monitor 18 which is the display unit. Adjust. Therefore, as shown in FIG. 7, the display control unit 98 includes a display ratio setting unit 101 and a display image generation unit 102. The display ratio setting unit 101 adjusts the relative display ratio of the direct-view observation image 111 and the side-view observation image 112 on the monitor 18, which is the display unit.

The display ratio setting unit 101 has a plurality of relative display ratio settings as display modes in advance. Specifically, it has three types of display modes: a standard mode, a direct view magnifying mode, and a side view magnifying mode.

In the standard mode, the relative display ratio of the direct observation image 111 and the side observation image 112 is changed to the display ratio as captured without substantially enlarging or reducing both the direct observation image 111 and the side observation image 112. The display mode to set. That is, in the standard mode, the relative display ratio between the direct-view observation image 111 and the side-view observation image 112 is substantially the display ratio in the captured image. “Almost not enlarged or reduced” means excluding the unavoidable enlargement or reduction of the direct observation image 111 or the side observation image 112 for connecting the direct observation image 111 and the side observation image 112. Therefore, even in the standard mode, either one or both of the direct-view observation image 111 and the side-view observation image 112 may be enlarged or reduced for joining. Hereinafter, for the sake of simplicity, in the standard mode, the display area of the direct-view observation image 111 and the display area of the side-view observation image 112 on the monitor 18 which is the display unit are assumed to be substantially equal.

The direct-view enlargement mode is a display mode in which the display area of the direct-view observation image 111 on the monitor 18 which is a display unit is enlarged as compared with the case of the standard mode. That is, in the case of the direct view enlargement mode, the display ratio setting unit 101 increases the display ratio of the direct view observation image 111 relative to the side view observation image 112. Further, the direct-view magnifying mode includes a case where only the direct-view observation image 111 is substantially displayed among the direct-view observation image 111 and the side-view observation image 112.

The side view enlargement mode is a display mode in which the display area of the side view observation image 112 on the monitor 18 which is a display unit is enlarged as compared with the case of the standard mode. That is, in the side view enlargement mode, the display ratio setting unit 101 increases the display ratio of the side view observation image 112 relative to the direct view observation image 111. Further, the side view magnifying mode includes a case where only the side view observation image 112 is substantially displayed among the direct view observation image 111 and the side view observation image 112.

The specific display ratio of the direct-view observation image 111 and the side-view observation image 112 in the direct-view magnifying mode and the side-view magnifying mode is, for example, an input device such as a console 19, a switch provided on the operation unit 12b, or a foot switch (hereinafter, , Console 19 and the like) can be arbitrarily set by a doctor or the like who is a user of the endoscope system 10. The specific display ratios of the direct-view observation image 111 and the side-view observation image 112 can be arbitrarily changed even during the observation. When a doctor or the like sets a numerical value or the like that reduces the display area of the direct-view observation image 111 in the direct-view enlargement mode as compared with the case of the standard mode, the side-view enlargement mode is automatically set. The reverse is also true. Further, the standard mode, the direct view magnifying mode, and the side view magnifying mode can be arbitrarily switched by using the console 19 or the like.

The display image generation unit 102 produces either the direct-view observation image 111 and the side-view observation image 112, or the direct-view observation image 111 or the side-view observation image 112 according to the display ratio determined by the display ratio setting unit 101. If necessary, the images are enlarged or reduced and joined together, and the portion of the side view observation image 112 corresponding to the blind spot is masked to generate a display image.

“Mask” means to modulate or replace the data of the side view observation image 112 with another image or the like. The portion of the side view observation image 112 corresponding to the blind spot is a portion where the observation target cannot be imaged due to the presence of the second protrusion 32. Hereinafter, in the display image, the portion of the side view observation image 112 corresponding to the blind spot caused by the presence of the second protrusion 32 is referred to as the first blind spot portion. The display image generation unit 102 masks at least this first blind spot portion in the display image. Further, in the present embodiment, the portion of the lateral observation image 112 that is outside the field of view of the lateral observation unit 42 and cannot be imaged in the first place (hereinafter referred to as the second blind spot portion) is also displayed. Mask in the image for use. The display control unit 98 displays the display image generated by the display image generation unit 102 on the monitor 18.

In the endoscope system 10, a display image in which the relative display ratios of the direct-view observation image 111 and the side-view observation image 112 are adjusted is displayed on the monitor 18. For example, when the display mode is set to the standard mode, the display control unit 98 generates the display image 113 shown in FIG. 8 and displays it on the monitor 18. In the display image 113 in the standard mode, the display control unit 98 uses the display image generation unit 102 to connect the outer circumference of the direct-view observation image 111 and the inner circumference of the side-view observation image 112 to each other in a size almost as captured. Generate. According to the display image 113 in the standard mode, a wider range of observation objects can be observed as compared with displaying only the direct observation image 111 or only the side observation image 112.

Actually, there is no clear boundary line between the direct view observation image 111 and the side view observation image 112, but for the sake of explanation, these boundaries are shown by a two-dot chain line in FIG. 8 (the same applies hereinafter). Further, in the side view observation image 112, the portion in which the observation target is shown is generally annular, but the first blind spot portion 114 due to the second protrusion 32 is formed in a certain angle range from the center C0. Therefore, the display control unit 98 masks the first blind spot portion 114 when the display image generation unit 102 is used to generate the display image 113. Further, the outer portion of the side view observation image 112 is a second blind spot portion 115 that cannot originally image the observation target. Therefore, the display control unit 98 masks the second blind spot portion 115 when the display image generation unit 102 is used to generate the display image 113. As a result, the distinction between the first blind spot portion 114 and the second blind spot portion 115 in which the observation target is not shown and the other portion of the side view observation image 112 in which the observation target is shown is clarified.

When the display mode is set to the direct view magnifying mode, the display control unit 98 generates the display image 123 shown in FIG. 9 and displays it on the monitor 18. Specifically, when the display control unit 98 generates the display image 123 in the direct view enlargement mode, the display image generation unit 102 enlarges the direct view observation image 111 and side-views while maintaining the size of the outer circumference. The observation image 112 is compressed in the radial direction. Then, the display image 123 is generated by connecting the outer circumference of the direct-view observation image 111 and the inner circumference of the side-view observation image 112 and masking the first blind spot portion 114 and the second blind spot portion 115.

Therefore, the display image 123 in the direct view magnifying mode is the display range of the observation target (the range of the observation target that can be displayed on the monitor 18 for all the observation targets) and the display area (the area where the observation target is displayed on the monitor 18) itself. Is the same as the display image 113 in the standard mode, but the display ratio of the direct observation image 111 to the side observation image 112 is relatively large in the display range and display area of the observation target. Therefore, if the display mode is set to the direct-view magnifying mode, the doctor or the like can observe a wide range of observation targets by the direct-view observation image 111 and the side-view observation image 112, and in particular, observe the observation target reflected in the direct-view observation image 111. It will be easier to do. For example, when a notable part such as a suspected lesion is found in the tip direction of the insertion portion 12a, if the display mode is set to the direct view magnifying mode, the notable part can be observed in more detail. it can.

Further, when the display mode is set to the side view enlargement mode, the display control unit 98 generates the display image 126 shown in FIG. 10 and displays it on the monitor 18. Specifically, when the display control unit 98 generates the display image 126 in the side view enlargement mode, the display image generation unit 102 enlarges the side view observation image 112 in the radial direction. Specifically, it is stretched toward the inside (the side of the center C0) while maintaining the size of the outer circumference, while the direct-view observation image 111 is reduced to a size that matches the inner circumference of the enlarged side-view observation image 112. To do. Then, the display image 126 is obtained by connecting the inner circumference of the enlarged side view observation image 112 and the outer circumference of the reduced direct observation image 111 and masking the first blind spot portion 114 and the second blind spot portion 115. Generate.

Therefore, the display image 126 in the lateral view magnifying mode has the same display range and display area as the display image 113 in the standard mode, but is a direct observation image within the display range and display area of the observation target. The display ratio of the side view observation image 112 is larger than that of 111. Therefore, if the display mode is set to the side view magnifying mode, the doctor or the like can observe a wide range of observation targets by the direct view observation image 111 and the side view observation image 112, and in particular, the observation target reflected in the side view observation image 112. Is easier to observe. For example, when a notable part such as a suspected lesion is found in the lateral direction of the insertion portion 12a, if the display mode is set to the lateral view magnifying mode, the notable part can be observed in more detail.

As described above, the endoscope system 10 has a direct-view magnifying mode and a side-view magnifying mode for adjusting the relative display ratio of the direct-view observation image 111 and the side-view observation image 112 on the monitor 18. Then, by switching the display mode to the direct view magnifying mode or the side view magnifying mode, the observation target reflected in the direct view observation image 111 or the side view observation image 112 can be enlarged while maintaining the display range and the display area of the observation target. Therefore, the endoscope system 10 is easier to use in observation or diagnosis than before.

In the direct view magnifying mode, for example, the display ratio of the direct view observation image 111 can be set to 100%, and the display ratio of the side view observation image 112 can be set to 0%. In this case, as shown in FIG. 11, a display image 128 that substantially displays only the direct-view observation image 111 can be generated and displayed on the monitor 18. Therefore, the endoscope system 10 requires a display image 123 in the standard mode for displaying the direct observation image 111 and the side view observation image 112, and a display image 128 for displaying substantially only the direct observation image 111. It can be arbitrarily switched according to the above and displayed on the monitor 18. The display image 128 expands the direct-view observation image 111 to the full display range (the range from the center C0 to the outer circumference of the side-view observation image 112 in the display image 113 in the standard mode) and masks the second blind spot portion 115. Generate by doing.

Further, in the side view magnifying mode, for example, the display ratio of the side view observation image 112 can be set to 100%, and the display ratio of the direct view observation image 111 can be set to 0%. In this case, as shown in FIG. 12, it is possible to generate a display image 129 that substantially displays only the side view observation image 112 and display it on the monitor 18. Therefore, the endoscope system 10 includes a display image 123 in the standard mode for displaying the direct observation image 111 and the side observation image 112, and a display image 129 for displaying substantially only the side observation image 112. It can be arbitrarily switched and displayed on the monitor 18 as needed. The display image 129 extends the inner circumference in the radial direction (in the direction of the center C0) while maintaining the size of the outer circumference of the side view observation image 112, and extends the first blind spot portion 114 and the second blind spot portion 115. Generated by masking.

[Second Embodiment]
In the first embodiment, the relative display ratio of the direct-view observation image 111 and the side-view observation image 112 in the direct-view magnifying mode and the side-view magnifying mode is set by a doctor or the like who is a user of the endoscope system 10. However, the relative display ratio of the direct view observation image 111 and the side view observation image 112 in the direct view magnifying mode and the side view magnifying mode can be automatically set. In this case, as in the endoscope system 200 shown in FIG. 13, the processor device 16 is provided with a motion detection unit 201. Further, if necessary, the endoscope 12 is provided with a motion sensor 202.

The motion detection unit 201 detects the movement of the tip portion 12d of the insertion portion 12a. The movement of the tip portion 12d is the movement direction of the tip portion 12d along the observation target, the position change (speed, speed or acceleration) along the movement direction of the tip portion 12d along the observation target, and the curvature of the curved portion 12c. The direction of the tip portion 12d due to the curvature, the angle at which the tip portion 12d is deflected due to the curvature of the curved portion 12c, or the change in the direction of the tip portion 12d due to the curvature of the curved portion 12c (angular velocity or angular acceleration), etc. Is. The motion sensor 202 is a sensor for detecting the motion of the tip portion 12d, and for example, for measuring the length of insertion of the speed sensor, the acceleration sensor, the angular velocity sensor, the angular acceleration sensor, and the insertion portion 12a into the observation target. A sensor or mechanism, or a sensor or mechanism for measuring the degree of curvature of the curved portion 12c.

For example, the motion detection unit 201 acquires a plurality of either the direct-view observation image 111 and the side-view observation image 112, or the direct-view observation image 111 or the side-view observation image 112 from the image acquisition unit 97 over time. The movement of the tip portion 12d is detected by using the direct observation image 111 or the side observation image 112. When the endoscope 12 is provided with the motion sensor 202, the motion sensor is used instead of the direct observation image 111 or the side observation image 112, or in addition to the direct observation image 111 or the side observation image 112. The movement of the observation target is detected using the output signal of 202.

When the processor device 16 has the motion detection unit 201, the display control unit 98 refers to the movement of the tip portion 12d detected by the motion detection unit 201, and refers to the relative of the direct view observation image 111 and the side view observation image 112. Adjust the display ratio. That is, the display control unit 98 automatically selects the display mode due to the movement of the tip portion 12d detected by the motion detection unit 201 in the display ratio setting unit 101. Further, when the automatically selected display mode is the direct view magnifying mode or the side view magnifying mode, the relative display ratio between the direct view observation image 111 and the side view observation image is also the movement of the tip portion 12d detected by the motion detection unit 201. It is determined automatically due to.

In the endoscope system 200, for example, the display control unit 98 can adjust the relative display ratios of the direct-view observation image 111 and the side-view observation image 112 according to the insertion / removal of the insertion unit 12a. In this case, as shown in FIG. 14, the motion detection unit 201 detects whether the insertion unit 12a is inserted or removed, that is, the moving direction of the tip portion 12d is the insertion direction or the removal direction (S210). As shown in FIG. 15, the insertion direction is a moving direction of the insertion portion 12a or the like when the insertion portion 12a is inserted into the observation target 205, and a direction in which the tip portion 12d moves to the back of the observation target 205. The extraction direction is the opposite direction of the insertion direction, which is the moving direction of the insertion portion 12a or the like when the insertion portion 12a is removed from the observation target 205 along the observation target 205, and the tip portion 12d moves toward the front. is there.

When the moving direction of the tip portion 12d is the insertion direction (S211; YES), the display control unit 98 automatically sets the display mode to the direct view magnifying mode in the display ratio setting unit 101 (S212), and the monitor 18 directly sees. The display image 123 in the enlarged mode is displayed. On the other hand, when the moving direction of the tip portion 12d is the extraction direction (S211; NO), the display control unit 98 automatically sets the display mode to the side view enlargement mode in the display ratio setting unit 101 (S213), and monitors. A display image 126 in the lateral view magnifying mode is displayed on the 18.

The endoscope system 200 is set to the direct view magnifying mode when the insertion portion 12a is inserted as described above, and is set to the side view magnifying mode when the insertion portion 12a is removed. Of the side-view observation images 112, the display ratio of the image that the doctor or the like is likely to pay attention to when inserting or removing the insertion portion 12a is automatically increased on the monitor 18. Therefore, the endoscope system 200 is easier to insert and remove the insertion portion 12a than the conventional endoscope system that displays the direct observation image 111 and the side observation image 112.

In the second embodiment, the direct view enlargement mode and the side view enlargement mode are selected according to the insertion / removal of the insertion part 12a, but the motion detection unit 201 moves the moving speed of the insertion part 12a (tip part 12d) or the insertion part. When detecting the movement acceleration of the 12a (tip portion 12d), the movement speed or acceleration of the insertion portion 12a (tip portion 12d) is further taken into consideration with the direct observation image 111 in the direct view magnifying mode or the lateral view magnifying mode. The relative display ratio of the side view observation image 112 can be automatically set. For example, in FIG. 16, as shown by using the graph 111A (solid line) showing the display ratio of the direct observation image 111 and the graph 112A (single point chain line) showing the display ratio of the side view observation image 112, the moving speed of the insertion portion 12a. Is positive, and when the insertion portion 12a moves in the insertion direction, the display ratio of the direct-view observation image 111 is increased and the display ratio of the side-view observation image 112 is decreased according to the moving speed. That is, when the moving speed of the insertion unit 12a is positive, the display mode is the direct-view magnifying mode, and the display control unit 98 enlarges the direct-view observation image 111 so as to have a display ratio corresponding to the moving speed. Is generated and displayed on the monitor 18. After the moving speed V1 (V1> 0) at which the display ratio of the direct-view observation image 111 reaches 100%, the display control unit 98 generates a display image 128 that substantially displays only the direct-view observation image 111. Displayed on the monitor 18.

On the other hand, when the moving speed of the insertion portion 12a is negative and the insertion portion 12a moves in the extraction direction, the display ratio of the direct-view observation image 111 is reduced according to the movement speed, and the side-view observation image 112 Increase the display ratio. That is, when the moving speed of the insertion unit 12a is negative, the display mode is the side view magnifying mode, and the display control unit 98 enlarges the side view observation image so that the display ratio corresponds to the moving speed. 126 is generated and displayed on the monitor 18. After the moving speed V2 (V2 <0) at which the display ratio of the side view observation image 112 reaches 100%, the display control unit 98 generates a display image 129 that substantially displays only the side view observation image 112. Then, it is displayed on the monitor 18.

If there is no insertion / removal movement in the insertion portion 12a and the moving speed is "0", the display mode automatically becomes the standard display mode. Therefore, when the moving speed is "0", the display control unit 98 generates the display image 113 in the standard mode and displays it on the monitor 18.

As described above, considering not only the insertion / removal of the insertion portion 12a but also the moving speed (or acceleration) when inserting / removing the insertion portion 12a, the direct-view observation image 111 is appropriately adjusted to the finer movement of the insertion portion 12a. Alternatively, the side view observation image 112 can be enlarged and displayed. Therefore, in addition to the pre- and post-diagnosis operations of inserting or removing the insertion portion 12a, the insertion portion 12a is moved to make it easier to observe the part of interest during the observation for the diagnosis or the like. Even when the tip portion 12d is moved, the direct-view observation image 111 or the side-view observation image 112 can be appropriately enlarged and displayed according to the movement.

The tip portion 12d is moved in the insertion direction because there is a portion to be noted in the tip direction, and the tip portion 12d is moved in the removal direction because there is a portion to be noted in the side surface direction. Therefore, if the direct view magnifying mode or the side view magnifying mode is automatically set as described above, the image of the direct view observation image 111 or the side view observation image 112 including the notable portion is automatically enlarged and displayed. can do. Further, there is a notable portion at a position far from the current position of the tip portion 12d, and when the insertion portion 12a is moving fast, the direct observation image 111 or the side observation image 112 is enlarged more, so that the insertion portion 12a There is also an advantage that it is hard to lose sight of the notable part even if you move fast.

In the above modified example, the display mode is switched between the direct view magnifying mode and the side view magnifying mode due to the insertion / removal of the insertion portion 12a, and the moving speed is taken into consideration. The display mode may be switched depending on the moving speed regardless of the moving direction of the insertion portion 12a. For example, in FIG. 17, as shown by using the graph 111A showing the display ratio of the direct view observation image 111 and the graph 112A showing the display ratio of the side view observation image 112, the movement speed V3 (V3> 0) having a movement speed is shown. At the boundary, the direct view magnifying mode and the side view magnifying mode may be switched. Of course, in the same manner, the direct view magnifying mode and the side view magnifying mode may be switched with a negative moving speed as a boundary.

In the second embodiment, the direct view enlargement mode and the side view enlargement mode are selected according to the insertion / removal of the insertion portion 12a, but the motion detection unit 201 determines the direction of the tip portion 12d due to the curvature of the bending portion 12c. When detecting, the display control unit 98 can automatically select the direct view magnifying mode and the side view magnifying mode according to the direction of the tip portion 12d caused by the bending of the curved portion 12c. Specifically, for example, as shown in FIG. 18, the motion detecting unit 201 detects the presence or absence of bending of the bending portion 12c, and as a result, detects the orientation of the tip portion 12d (S250). When the curvature of the curved portion 12c is detected (S251; YES), the display control unit 98 automatically sets the display mode to the direct-view magnifying mode and magnifies and displays the direct-view observation image 111 (S252). This is because, as shown in FIG. 19, when the curved portion 12c is curved to change the direction of the tip portion 12d, there is usually a portion to be noted in the tip direction of the tip portion 12d. On the other hand, when the curved portion 12c is not curved and the tip portion 12d is oriented in the insertion direction, the display control unit 98 automatically sets the display mode to the side view magnifying mode, and the side view observation image. 112 is enlarged and displayed (S253). As shown in FIG. 20, when the curved portion 12c is not curved and the tip portion 12d is oriented in the insertion direction, the doctor or the like confirms the presence or absence of a notable portion by using the side view observation image 112 (screening). ) This is because there are many cases.

In the above, when the curved portion 12c is not curved and the tip portion 12d is oriented in the insertion direction, the display mode is automatically set to the lateral view magnifying mode, but the curved portion 12c is When the tip portion 12d is not curved and the tip portion 12d faces the insertion direction, the display mode may be automatically set to the standard mode.

Further, in the case of switching between the direct view magnifying mode and the standard mode or the side view magnifying mode according to the direction of the tip portion 12d, the motion detecting unit 201 further determines the bending angle, bending speed or acceleration of the bending portion 12c. When detecting, the relative display ratio of the direct-view observation image 111 and the side-view observation image 112 in the direct-view magnifying mode and the side-view magnifying mode is automatically calculated by using the bending angle, bending speed, or acceleration of the bending portion 12c. Can be set to. For example, in FIG. 21, as shown by using the graph 111A showing the display ratio of the direct view observation image 111 and the graph 112A showing the display ratio of the side view observation image 112, the display mode is automatically set when the bending angle is 0 degrees. The standard mode is set, and when the bending angle is larger than 0 degrees, the display mode is automatically set to the direct view magnifying mode. When the bending angle is larger than 0 degrees and 90 degrees or less, the display control unit 98 sets the relative display ratio of the direct view observation image 111 and the side view observation image 112 according to the bending angle. In this way, the direct-view observation image 111 can be enlarged and displayed to an appropriate size according to a finer change in the orientation of the tip portion 12d.

When the bending angle is larger than 90 degrees, the tip portion 12d faces rearward. In this case, since there is a high possibility that the portion to which the tip portion 12d is directed has a portion that should be noted in diagnosis or the like, the display control unit 98 generates and monitors a display image 128 that substantially displays only the direct observation image 111. It is preferable to display it on 18 (see FIG. 21). However, as shown in FIG. 22, when the bending angle is larger than 90 degrees, the display ratio of the direct view observation image 111 is reduced and the display ratio of the side view observation image 112 is increased according to the bending angle. Is also good.

Further, FIGS. 21 and 22 show an example in which the display ratio of the side-view observation image 112 does not exceed the display ratio of the direct-view observation image 111 and the direct-view magnifying mode and the standard mode are switched. By crossing the graph 112A and providing a portion where the display ratio of the side view observation image 112 exceeds the display ratio of the direct view observation image 111, the display mode can be switched between the direct view enlargement mode and the side view enlargement mode.

[Third Embodiment]
In the first embodiment and the second embodiment, when the first blind spot portion 114 is masked and the side view observation image 112 is reduced or enlarged in the direct view enlargement mode or the side view enlargement mode, this The first blind spot portion 114 is similarly reduced or enlarged. Therefore, in the first embodiment and the second embodiment, the portion of the side view observation image 112 in which the observation target is copied (the portion excluding the first blind spot portion 114 and the second blind spot portion 115) and the first 1 The blind spot portion 114 maintains a relatively constant magnitude relationship before and after reduction or expansion. However, it is not necessary to maintain a constant magnitude relationship between the first blind spot portion 114 and the portion of the side view observation image 112 captured by the observation target.

For example, the first blind spot portion 114 can be reduced as needed, and as a result, the display area of the first blind spot portion 114 can be reduced. In this case, as shown in FIG. 23, the display control unit 98 is provided with a mask display ratio setting unit 301 in addition to the display ratio setting unit 101 and the display image generation unit 102. The mask display ratio setting unit 301 covers the side view observation image 112 and the first blind spot portion 114 according to the relative display ratio of the direct view observation image 111 and the side view observation image 112 set by the display ratio setting unit 101. Change the relative display ratio of the mask.

Specifically, for example, as shown in FIG. 24, it is assumed that the first blind spot portion 114 in the display image 113 in the standard mode is in the range of the angle θ1 with the center C0 as the center. Further, it is assumed that the display ratio setting unit 101 has set the display ratio of the side view enlargement mode. Therefore, the display image generation unit 102 first reduces or enlarges the direct-view observation image 111 and the side-view observation image 112 in the radial direction centered on the center C0, and the relative of the direct-view observation image 111 and the side-view observation image 112. The display ratio is set to the display ratio determined by the display ratio setting unit 101. This is the same as the first embodiment and the like.

On the other hand, in the present embodiment, the mask display ratio setting unit 301 sets the range of the first blind spot portion 114 centered on the center C0 to an angle θ2 that is at least smaller than the angle θ1 in the lateral view enlargement mode. When the mask display ratio setting unit 301 defines the range of the first blind spot portion 114 at the angle θ2, the display image generation unit 102 reduces the direct view observation image 111 and the side view observation image 112 in the radial direction as described above. Alternatively, after enlarging, the direct-view observation image 111 and the side-view observation image 112 are further stretched or compressed in the circumferential direction. Specifically, of the direct-view observation image 111 and the side-view observation image 112, the first blind spot portion 114 is compressed in the circumferential direction at a certain angle θ1 and the other portion is stretched in the circumferential direction. As a result, the first blind spot portion 114 is accommodated in the angle θ2. As a result, a display image 302 in which the relative display ratio of the mask covering the first blind spot portion 114 is reduced with respect to the side view observation image 112 is generated and displayed on the monitor 18.

The display image 302 has a smaller display area of the mask covering the first blind spot portion 114 and the first blind spot portion 114 than, for example, the display image 126 (see FIG. 10) in the lateral view magnifying mode of the first embodiment. Moreover, the display area of the part where other observation objects are reflected is large. Therefore, according to the display image 302, the observation target can be displayed more easily.

When the display image 302 is generated, not only the side view observation image 112 but also the direct view observation image 111 compresses the portion within the angle θ1 in the circumferential direction and the portion other than the angle θ1 in the circumferential direction. The reason for stretching is to prevent the image of the observation target from being displaced at the boundary between the direct observation image 111 and the side observation image 112. Further, in the lateral view magnifying mode, the display area is reduced by compressing the first blind spot portion 114, and the display area is enlarged by enlarging the other observation target portion in the lateral view magnifying mode. The display area of the first blind spot portion 114 tends to be large on the monitor 18, and although it is masked, the first blind spot portion 114 that does not contribute to diagnosis or the like becomes conspicuous, so that it is easy to make it difficult to observe the observation target. Because.

In the third embodiment, as an example, in the lateral view magnifying mode, the first blind spot portion 114 is reduced and the portion in which other observation objects are captured is enlarged, but the direct viewing magnifying mode or the standard In the mode, the first blind spot portion 114 may be reduced and the portion in which other observation objects are captured may be enlarged. In addition, in the lateral view magnifying mode, the direct viewing magnifying mode, or the standard mode, the first blind spot portion 114 can be enlarged and the portion in which the other observation target is captured can be reduced.

In the third embodiment, the first blind spot portion 114 is reduced, but the second blind spot portion 115 can be reduced. In this case, the mask display ratio setting unit 301 sets whether or not to expand the outer circumference of the side view observation image 112 in the radial direction. For example, it is assumed that the mask display ratio setting unit 301 is set to expand the outer circumference of the side view observation image 112 in the radial direction, as shown by using an arrow extending from the outer circumference of the side view observation image 112 in FIG. 25. In this case, the display image generation unit 102 enlarges or reduces the direct view observation image 111 and the side view observation image 112 according to the display ratio set by the display ratio setting unit 101. Then, the side view observation image 112 is stretched in the radial direction to expand the display area of the side view observation image 112. As a result, the display image generation unit 102 generates, for example, a display image 304 without the second blind spot portion 115 and displays it on the monitor 18, as shown in FIG. 26. By generating and displaying the display image 304 without the second blind spot portion 115 in this way, the observation target can be displayed in the entire display range on the monitor 18, so that the observation target can be more easily observed.

The hiding of the second blind spot portion 115 in the above modification can be performed in any of the direct view magnifying mode, the side view magnifying mode, and the standard mode. Therefore, the hiding of the second blind spot portion 115 in the above modification can be performed in the direct view magnifying mode, the side view magnifying mode, or one or more selected display modes of the standard mode. Further, the hiding of the second blind spot portion 115 in the above modification is performed instead of the reduction or enlargement of the first blind spot portion 114 in the third embodiment, or the reduction or enlargement of the first blind spot portion 114 in the third embodiment. It can be done in addition to expansion.

In addition, in the above modification, the second blind spot portion 115 is completely hidden, but the mask display ratio setting unit 301 sets the display ratio of the second blind spot portion 115 to display the second blind spot portion 115. While reducing the display area, a part of the display of the second blind spot portion 115 can be left in the display image 304. In this case, similarly to the reduction of the first blind spot portion 114 of the third embodiment, the reduction ratio of the display area of the second blind spot portion 115 according to the relative display ratio of the direct observation image 111 and the side observation image 112. It is good to set. For example, in the side view enlargement mode, the display area of the second blind spot portion 115 can be reduced in inverse proportion to the display ratio of the side view observation image 112.

Although the display control unit 98 stores the angle θ1 in which the first blind spot portion 114 is located in advance, it is preferable that the data of the angle θ1 can be updated. In calibration or the like, the value of the angle θ1 is adjusted based on, for example, the visibility of the nozzle 51 and the nozzle 52 from the lateral observation unit 42, and as a result, the size of the area to be masked as the first blind spot portion 114 is adjusted. It is preferable to be able to do it. The same applies to the first embodiment and the like.

[Fourth Embodiment]
In the first embodiment, the second embodiment, and the third embodiment, the direct-view observation image 111 and the side-view observation image 112 are displayed at the center of the display image (display image 113, etc.). The display positions of the direct-view observation image 111 and the side-view observation image in the display image can be adjusted. When adjusting the display positions of the direct-view observation image 111 and the side-view observation image, as shown in FIG. 27, the display control unit 98 is provided with a display position setting unit 401. The display position setting unit 401 sets the display position of the direct view observation image 111 or the side view observation image 112 in the display image. The display image generation unit 102 adjusts to the position set by the display position setting unit 401 to generate a display image.

In particular, when the motion detection unit 201 of the second embodiment is provided in the processor device 16, the display position setting unit 401 displays the direct view observation image 111 or the side view observation image 112 according to the movement detected by the motion detection unit 201. The position can be adjusted. Specifically, as shown in FIG. 28A, when the tip portion 12d faces the insertion direction in the standard mode, the display position setting unit 401 sets the display positions of the direct view observation image 111 and the side view observation image 112. , Set to the center of the display image to be generated and displayed. Therefore, the display image generation unit 102 generates the display image 113 and displays it on the monitor 18.

On the other hand, as shown in FIG. 28B, when the tip portion 12d is turned to the right in the standard mode, the display position setting unit 401 sets the direction (angle) of the tip portion 12d or the tip portion 12d. The display position of the direct-view observation image 111 is shifted to the left according to the speed, speed, or acceleration of moving to the right. As a result, the display image generation unit 102 stretches or compresses the inner circumference of the side view observation image 112 according to the display position of the direct view observation image 111, and displays the direct view observation image 111 and the side view observation at the set position. The images 112 are joined together. As a result, the display image generation unit 102 generates a display image 410 in which the display position of the direct-view observation image 111 is biased to the left, and displays it on the monitor 18. When the tip portion 12d is directed to the right, there is usually a portion to be noted in diagnosis or the like in the right direction of the tip portion 12d, so the display position of the direct-view observation image 111 is shifted to the left as described above, and the side view is performed. Enlarging the right side portion of the observation image 112 makes it easier to observe the notable portion.

Further, as shown in FIG. 28C, when the tip portion 12d is turned to the left in the standard mode, the display position setting unit 401 sets the direction (angle) of the tip portion 12d or the tip portion 12d. The display position of the direct-view observation image 111 is shifted to the right according to the speed, speed, or acceleration of moving to the left. As a result, similarly to the above, the display image generation unit 102 generates the display image 411 in which the display position of the direct-view observation image 111 is biased to the right, and displays it on the monitor 18. When the tip portion 12d is directed to the left, there is usually a portion to be noted in diagnosis or the like in the left direction of the tip portion 12d, so the direct observation image 111 is shifted to the right as described above, and the side view observation image 112 is used. Enlarging the left part of is easier to observe the notable part. The same applies when the tip portion is directed in another direction such as upward or downward.

In the fourth embodiment, the display position of the direct-view observation image 111 is adjusted according to the movement of the tip portion 12d detected by the motion detection unit 201. As shown in FIG. 29, the monitor 18 heads the head. In the case of the mount display 421, the display position of the direct-view observation image 111 or the like can be adjusted according to the movement of the head of a doctor or the like who wears the head-mounted display 421. In this case, the motion detection unit 201 detects the motion of the head-mounted display 421.

Then, in the standard mode, when the doctor or the like wearing the head-mounted display 421 faces the front (the front of the wearer such as the doctor determined by the calibration of the head-mounted display 421), the direct-view observation image 111 is mainly displayed. The display image 113 is displayed on the head-mounted display 421 (see FIG. 29 (A)). On the other hand, when the doctor or the like wearing the head-mounted display 421 faces right (the right of the wearer such as the doctor determined by the calibration of the head-mounted display 421), the tip portion 12d is directed to the right in the fourth embodiment. A display image 410 in which the display position of the direct-view observation image 111 is biased to the left is generated and displayed on the head-mounted display 421 (see FIG. 29 (B)). Further, when the doctor or the like who wears the head-mounted display 421 faces left (the left of the wearer such as a doctor determined by the calibration of the head-mounted display 421), the tip portion 12d is directed to the left in the fourth embodiment. A display image 411 in which the display position of the direct-view observation image 111 is biased to the right is generated and displayed on the head-mounted display 421 (see FIG. 29 (C)).

In the fourth embodiment, for the sake of simplicity, the case where the display mode is the standard mode is described as an example, but the direct view observation image 111 or the side view observation image 112 is also described in the direct view magnifying mode or the side view magnifying mode. The display position of can be adjusted. That is, the fourth embodiment can be carried out in any combination with any one or more of the first embodiment, the second embodiment, and the third embodiment.

[Fifth Embodiment]
The endoscope 12 in each embodiment of the first embodiment and the like (hereinafter referred to as the first embodiment and the like) has a blind spot due to the presence of the second protrusion 32, and a field of view of the lateral observation unit 42. There may be blind spots other than the blind spots on the outside of. Specifically, as shown in FIG. 30, due to the performance of the image pickup lens 61, the field of view 501 of the direct view observation unit 41 and the field of view 502 of the side view observation unit 42 may not overlap. In this case, between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42, there is a blind spot 503 that cannot be captured by either the visual field 501 of the direct visual observation unit 41 or the visual field 502 of the lateral observation unit 42. May be done. However, in the first embodiment or the like, since the direct-view observation image 111 and the side-view observation image 112 are connected and displayed, the blind spot 503 between the field of view 501 of the direct-view observation unit 41 and the field of view 502 of the side-view observation unit 42 is displayed. If there is a part to be noted, the doctor or the like cannot visually recognize its existence on the monitor 18, and may overlook it. Therefore, it is preferable that the display control unit 98 displays the existence of the blind spot 503 on the monitor 18 and notifies the doctor or the like who is the user.

For example, as shown in FIG. 31, the display control unit 98 displays a schematic diagram 505 showing the existence of the blind spot 503 in addition to the display image of the first embodiment or the like such as the display image 113. The schematic diagram 505 displays at least the visual field 501 of the direct visual observation unit 41, the visual field 502 of the lateral observation unit 42, and the blind spot 503 between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42. .. In this way, if the display control unit 98 displays the schematic diagram 505 on the monitor 18 side by side (or superimposing) on the display image such as the display image 113, the direct view observation image 111 and the side view observation image 112 are connected. Even when a display image such as the display image 113 is displayed on the monitor 18, a doctor or the like can make a diagnosis or the like while paying attention to the existence of the blind spot 503 on the monitor 18.

The display control unit 98 may display a message indicating the presence or absence of the blind spot 503 on the monitor 18 instead of the schematic diagram 505. Further, when there is a blind spot 503 between the field of view 501 of the direct view observation unit 41 and the field of view 502 of the side view observation unit 42, as shown in FIG. 32, the display control unit 98 uses the direct view observation image 111 and the side view observation image 112. The display image 523 may be generated by masking the portion 513 corresponding to the blind spot 503 (hereinafter referred to as the third blind spot portion) 513 without connecting the two, and displayed on the monitor 18. It is preferable that the display image 523 masking the third blind spot portion 513 and the display image such as the display image 113 not masking the third blind spot portion 513 can be switched between each other at an arbitrary timing. The display of the fifth embodiment can be arbitrarily combined with the first embodiment and the like.

[Sixth Embodiment]
Contrary to the fifth embodiment, the endoscope 12 in the first embodiment and the like has a field of view 501 and a side view observation of the direct view observation unit 41 as shown in FIG. 33 due to the performance of the image pickup lens 61. The field of view 502 of the unit 42 may overlap. Hereinafter, the overlapping visual fields of the direct visual observation unit 41 and the lateral visual observation unit 42 will be referred to as an overlapping visual field 601. As described above, when the direct-view observation unit 41 and the side-view observation unit 42 have the overlapping field of view 601, as shown in FIG. 34, the direct-view observation image 111 and the side-view observation image 112 have regions corresponding to the overlap-field 601. Since the same part of the same observation target 205 is captured in 602 (hereinafter referred to as an overlapping region), when the direct-view observation image 111 and the side-view observation image 112 are connected as in the first embodiment or the like, the generated display image In 113 and the like, the same portion of the observation target appears in two places near the boundary between the direct-view observation image 111 and the side-view observation image 112 (hereinafter referred to as double display). Therefore, it is preferable that the display control unit 98 generates a display image without double display even when the direct view observation unit 41 and the side view observation unit 42 have the overlapping visual field 601.

Therefore, as shown in FIG. 35, it is preferable that the display control unit 98 is provided with, for example, an overlap region detection unit 611 and an overlap region removal unit 612. By comparing the observation target captured in the direct-view observation image 111 with the observation target captured in the side-view observation image 112, the overlapping region detection unit 611 finds the same observation target as the side-view observation image 112 in the direct-view observation image 111. The overlapping region 602 or the overlapping region 602 in which the same observation target as the direct observation image 111 is captured in the side view observation image 112 is detected. Then, as shown in FIGS. 36 and 37, the overlapping region removing unit 612 removes the overlapping region 602 from the direct view observation image 111 or the side view observation image 112. Then, the display image generation unit 102 connects the direct-view observation image 111 and the side-view observation image 112 from which the overlapping region 602 is removed to generate the display image 113 and the like. By doing so, even when the direct-view observation unit 41 and the side-view observation unit 42 have the overlapping visual field 601, the double display of the display image 113 and the like is eliminated, so that the observation target can be easily observed and displayed.

When the display control unit 98 is provided with the overlap region detection unit 611 as in the sixth embodiment, the display control unit 98 may be further provided with a signal level correction unit 613, as shown in FIG. 38. The signal level correction unit 613 corrects the signal level of at least one of the direct view observation image 111 and the side view observation image 112 by using the overlap region 602 detected by the overlap region detection unit 611. As a result, the direct observation image 111 and the side observation image 112 are the same even if there is a difference between the brightness of the observation target in the field of view 501 of the direct observation unit 41 and the brightness of the observation object in the field of view 502 of the side observation unit 42. Since the brightness can be set, it is possible to generate and display the display image 113 and the like having uniform brightness.

In the sixth embodiment and its modifications, if the direct view observation unit 41 and the side view observation unit 42 have the overlapping visual field 601, the direct view enlargement mode, the side view enlargement mode, or the standard mode of the first embodiment or the like is used. It can be carried out in any combination with.

[7th Embodiment]
In the first embodiment or the like, when the display control unit 98 generates the display image 113 or the like in the display image generation unit 102, the direct view observation image 111 or the side view observation image 112 is enlarged or reduced. Therefore, the direct-view observation image 111 and the side-view observation image 112 are connected to each other, but the degree of aberration appearing in the direct-view observation image 111 and the side-view observation image 112 may differ due to the performance of the image pickup lens 61. When the degree of the aberration appearing in the direct-view observation image 111 and the side-view observation image 112 is different, simply connecting the direct-view observation image 111 or the side-view observation image 112 by simply enlarging or reducing the image, the direct-view observation image 111 and the side The observation target may appear distorted in the display image 113 or the like due to the difference in the aberration appearing in the visual observation image 112.

For example, as shown in FIG. 39, there is a difference between the distortion 701 of the direct-view observation image 111 and the distortion 702 of the side-view observation image 112. Specifically, the distortion 701 of the direct-view observation image 111 is a so-called barrel type or pincushion type (barrel type in FIG. 39). On the other hand, the distortion 702 of the side view observation image 112 is substantially radial. Therefore, the distortion 701 of the direct observation image 111 and the distortion 702 of the side observation image 112 may not be satisfactorily connected by simply enlarging or reducing the direct observation image 111 or the side observation image 112. In this case, the display image 113 and the like are particularly easy to recognize the distortion of the observation target in the vicinity of the boundary between the direct observation image 111 and the side observation image 112. Therefore, when the direct-view observation image 111 and the side-view observation image 112 are connected to generate a display image 113 or the like and displayed on the monitor 18, the display control unit 98 causes the direct-view observation image due to the performance of the image pickup lens 61. Considering the aberration appearing in the side view observation image 112 and the side view observation image 112, it is preferable to connect the direct view observation image 111 and the side view observation image 112 to generate the display image 113 and the like.

In order for the display control unit 98 to connect the direct observation image 111 and the side observation image 112 in consideration of the aberration appearing in the direct observation image 111 and the side observation image 112 due to the performance of the image pickup lens 61, FIG. As shown in 40, the display control unit 98 is provided with an aberration connection unit 705. The aberration connecting portion 705 can perform conversion processing such as affine transformation on at least one of the direct observation image 111 and the lateral observation image 112 to satisfactorily connect the aberrations of the direct observation image 111 and the lateral observation image 112. Convert to an image. For example, as shown in FIG. 41, the aberration connecting portion 705 affine transforms the direct observation image 111 in accordance with the distortion 702 of the side observation image 112. As a result, the distortion 701 of the direct-view observation image 111 is brought into a state where it can be satisfactorily connected to the distortion 702 of the side-view observation image 112. The display image generation unit 102 uses the direct-view observation image 111 and the side-view observation image 112, which are in a state where aberrations can be satisfactorily connected by the conversion process of the aberration connection unit 705 as described above, to display the display image 113 and the like. Generate and display on monitor 18. In this way, when the direct-view observation image 111 and the side-view observation image 112 are connected in consideration of the aberration appearing in the direct-view observation image 111 and the side-view observation image 112, the direct-view observation image 111 and the side-view observation image 112 are better. Can be connected to. As a result, it becomes difficult to recognize the distortion of the observation target in the display image 113 or the like, and it becomes easy to observe the observation target.

Since the performance of the image pickup lens 61 is known when the endoscope 12 is manufactured, what kind of conversion processing is performed on the direct observation image 111 or the side observation image 112 in the seventh embodiment. The image is predetermined. Therefore, the aberration connecting portion 705 has in advance the parameters of the conversion process applied to the direct-view observation image 111 or the side-view observation image 112. However, more strictly speaking, if the distance between the tip 12d of the endoscope 12 and the observation target changes during observation, it appears in the direct observation image 111 and the side observation image 112 according to the distance between the tip 12d and the observation target. The degree of aberration such as distortion 701 and distortion 702 changes. Therefore, when the aberration connecting unit 705 is provided in the display control unit 98, it is preferable to provide the distance measuring unit 706 in the endoscope 12 or the processor device 16 as shown in FIG. 42. As shown in FIG. 43, the distance measuring unit 706 measures the distance D1 between the tip portion 12d and the observation target in the vicinity of the boundary between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42. Then, the aberration connecting portion 705 adjusts or changes the parameter for conversion processing held in advance by using the tip portion 12d measured by the ranging unit 706 and the distance D1 to be observed. As a result, the aberrations of the direct-view observation image 111 and the side-view observation image 112 can be better connected. Since the distance D1 between the tip portion 12d and the observation target differs depending on the position (see FIG. 43), the aberration connecting portion 705 connects the direct-view observation image 111 and the side-view observation image 112 according to the average value of the distance D1 or the like. Adjust or change the parameters for conversion processing for each position. Further, when the relative angle between the tip portion 12d of the endoscope 12 and the observation target changes during observation, the direct-view observation image 111 and the side-view observation image 112 are displayed according to the relative angle between the tip portion 12d and the observation target. The degree of aberrations such as the distortion 701 and the distortion 702 that appear varies. Therefore, the aberration connecting portion 705 can adjust or change the parameters for the conversion process held in advance by using the relative angle between the tip portion 12d and the observation target. The relative angle between the tip portion 12d and the observation target can be obtained, for example, by using the distribution of the distance D1. Since the relative angle between the tip portion 12d and the observation target differs depending on the position, the aberration connecting portion 705 is adjusted to the average value of the relative angles between the tip portion 12d and the observation target, or the side of the direct observation image 111. It is the same as above that the parameters for the conversion process are adjusted or changed for each position where the visual observation image 112 is connected.

The vicinity of the boundary between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42 is when there is a blind spot 503 between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42. Is near the boundary between the visual field 501 and the blind spot 503 of the direct visual observation unit 41, near the boundary between the visual field 502 and the blind spot 503 of the lateral observation unit 42, or any location within the range of the blind spot 503. When the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42 have an overlapping visual field 601, the vicinity of the boundary between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42 is the direct visual observation unit 41. Near the boundary between the visual field 501 and the overlapping visual field 601, near the boundary between the visual field 502 and the overlapping visual field 601 of the side view observation unit 42, or any location within the range of the overlapping visual field 601. When there is no blind spot 503 between the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42 and there is no overlapping visual field 601, the visual field 501 of the direct visual observation unit 41 and the visual field 502 of the lateral observation unit 42 The vicinity of the boundary is the boundary between the field of view 501 of the direct vision observation unit 41 and the field of view 502 of the side view observation unit 42 or its vicinity.

[8th Embodiment]
In the first embodiment or the like, the control unit 96 performs AE control, but depending on the distance between the tip portion 12d and the observation target or the performance of the direct-view illumination unit 54, the direct-view illumination unit 81, and the side-view illumination unit 43. The brightness of the direct-view observation image 111 and the side-view observation image 112 may be different. When the brightness of the direct-view observation image 111 and the side-view observation image 112 are different, the control unit 96 may perform AE control in which both the direct-view observation image 111 and the side-view observation image 112 always have a common constant brightness. difficult. Therefore, as shown in FIG. 44, when the control unit 96 acquires the direct observation image 111 and the side observation image 112 for AE control (S801), the control unit 96 calculates an appropriate exposure amount using the direct observation image 111. (S802). In addition, an appropriate exposure amount is calculated using the side view observation image 112 (S803). Then, the control unit 96 uses either one of the exposure amount calculated by using the direct view observation image 111 and the exposure amount calculated by using the side view observation image 112 in the AE control. (S804), and the actual AE control is performed according to the determined exposure amount. By doing so, even if the brightness of the direct observation image 111 and the side observation image 112 are different, at least one of the direct observation image 111 and the side observation image 112 can always be imaged with an appropriate brightness. .. For example, if the AE control is always performed according to the exposure amount calculated by using the direct-view observation image 111, at least the direct-view observation image 111 always has an appropriate brightness. As a result, the ease of observing the direct-view observation image 111 can be guaranteed.

In the eighth embodiment, the control unit 96 exposes either one of the exposure amount calculated by using the direct view observation image 111 and the exposure amount calculated by using the side view observation image 112. Instead, the average value of the exposure amount calculated using the direct-view observation image 111 and the exposure amount calculated using the side-view observation image 112 can be used as the exposure amount of the AE control actually performed. Further, depending on the setting, even if the exposure amount of the AE control actually performed is calculated by using the exposure amount calculated by using the direct view observation image 111 and the exposure amount calculated by using the side view observation image 112. good. For example, the exposure amount calculated by using the direct-view observation image 111 and the exposure amount calculated by using the side-view observation image 112 are weighted and averaged, and the result is adopted as the exposure amount of AE control to be actually carried out. can do.

When the direct observation image 111 and the side observation image 112 are obtained at different timings, the exposure amount for obtaining the direct observation image 111 is calculated using the direct observation image 111, and the side observation is observed. The amount of exposure for obtaining the image 112 can be calculated using the side view observation image 112.

For example, when the control unit 96 executes the AE control by adjusting the amount of illumination light, as shown in FIG. 45, the control unit 96 acquires the direct-view observation image 111 and obtains the direct-view observation image 111 of the next direct-view observation image 111. The amount of illumination light used for imaging (hereinafter referred to as the amount of illumination light for direct viewing) is determined (S811). Further, the control unit 96 acquires the side view observation image 112 and determines the amount of illumination light to be used when the next side view observation image 112 is imaged (hereinafter, referred to as the side view illumination light amount) (S812). ). Then, the control unit 96 turns on the illumination light with the amount of the direct-view illumination light, and the direct-view observation unit 41 takes an image of the observation target. As a result, the image acquisition unit 97 acquires the direct-view observation image 111 (S813). Further, the control unit 96 turns on the illumination light with the amount of the side-view illumination light, and the side-view observation unit 42 takes an image of the observation target. As a result, the image acquisition unit 97 acquires the side view observation image 112 (S814).

As described above, when the direct observation image 111 and the side observation image 112 are obtained at different timings, the exposure amount for obtaining the direct observation image 111 is calculated by using the direct observation image 111, and the direct observation image 111 is used. By calculating the amount of exposure for obtaining the side-view observation image 112 using the side-view observation image 112, both the direct-view observation image 111 and the side-view observation image 112 can be acquired with appropriate exposure.

The AE control of the eighth embodiment and the modified example can be carried out in arbitrary combination with the first embodiment and the like.

[9th Embodiment]
When the direct-view observation image 111 and the side-view observation image 112 are obtained at different timings, the direct-view observation image 111 and the side-view observation image 112 having different properties can be obtained. For example, one of the direct-view observation image 111 and the side-view observation image 112 is a normal image (hereinafter referred to as a normal image) in which the observation target is imaged by using white light as the illumination light and the observation target is represented by a natural hue. In addition, the other of the direct-view observation image 111 and the side-view observation image 112 can be an image (hereinafter referred to as a special image) processed or the like in a manner different from that of the normal image. The special image is, for example, emphasizing a specific tissue or structure such as a blood vessel by frequency enhancement processing, contour enhancement processing, marking processing (processing of adding a frame or the like to a lesion or the like to display its position, etc.). An image in which a specific tissue or structure such as a blood vessel is more easily captured than a normal image by selecting an image, an illumination light having an extremely narrow wavelength band, or a combination of the wavelength, the wavelength band, or the illumination light of the illumination light (for example,). , So-called narrow-band light observation image), color tone conversion processing (for example, color tone conversion by tone curve or color tone conversion by remapping on a specific color space), etc. An image that can be identified by the difference, or an image in which the biological information of the observation target is calculated and the normal image is colored using the calculated biological information (for example, an oxygen saturation image in which the oxygen saturation value is represented by color). ) Etc.

As described above, when one of the direct observation image 111 or the side observation image 112 is made into a normal image and the other of the direct observation image 111 or side observation image 112 is made into a special image, FIG. 46 shows. As shown, the processor device 16 is provided with an image processing unit 830. The image processing unit 830 performs frequency enhancement processing, contour enhancement processing, marking processing, color tone conversion processing, or oxygen for the direct observation image 111 or the side observation image 112 acquired from the image acquisition unit 97, if necessary. Various processes such as a saturation calculation process and a normal image coloring process using the calculated oxygen saturation are performed. Then, the display control unit 98 generates a display image 113 and the like using the direct-view observation image 111 and the side-view observation image 112 that have undergone necessary processing in the image processing unit 830, and displays them on the monitor 18.

If at least one of the direct-view observation image 111 and the side-view observation image 112 is a special image, the detection of lesions and the like can be supported as compared with the case where both the direct-view observation image 111 and the side-view observation image 112 are normal images. it can. Further, when generating an image representing biological information, there is an advantage that information called biological information can be presented.

When only one of the direct observation image 111 and the side observation image 112 is a special image, it is preferable that the direct observation image 111 is a normal image and the side observation image 112 is a special image. Since the side view observation image 112 is often used for screening to find out lesions and the like, and the lesions and the like tend to be difficult to identify as compared with the direct view observation image 111, the side view observation image 112 is used as a special image. This will support screening and improve its accuracy.

As described above, when the direct observation image 111 is a normal image and the side observation image 112 is a special image, the side observation image 112 is preferably a so-called IHb (Index of Hemoglobin) image. IHb is a mucosal hemoglobin concentration index. Then, the IHb image is an image in which the average value of the hemoglobin index of the mucous membrane is calculated, and the portion higher than the value is highlighted in red and the portion lower than the value is highlighted in white. The IHb image can be generated by image processing using an image acquired by using white light as the illumination light, similarly to the normal image. If the side view observation image 112 is made into an IHb image, inflammation of, for example, a mucous membrane can be easily observed in the side view observation image 112. In addition, when obtaining the direct-view observation image 111 and when obtaining the side-view observation image 112, white light is used as the illumination light and the content of the image processing is changed to make the direct-view observation image 111 a normal image. Moreover, the side view observation image 112 can be made into a special image.

Further, when the direct observation image 111 is a normal image and the side observation image 112 is a special image, the side observation image 112 is preferably a so-called narrow band optical image. The narrow-band optical image is, for example, a blue image obtained by imaging an observation object with blue light having a narrow wavelength band (for example, a wavelength band of about ± 10 nm) and a green image obtained by imaging an observation object with green light having a narrow wavelength band. It is a pseudo-color image in which a structure such as a blood vessel or a pit pattern is emphasized by using an image. In addition, the direct-view observation image 111 can be made into a normal image and the side by changing the illumination light and the content of the image processing between when the direct-view observation image 111 is obtained and when the side-view observation image 112 is obtained. The visual observation image 112 can be a special image.

In the above, one of the direct-view observation image 111 and the side-view observation image 112 is a special image, but naturally, both the direct-view observation image 111 and the side-view observation image 112 can be special images. .. In this case, the direct-view observation image 111 and the side-view observation image 112 can be made into different types of special images (illumination light or special images having different image processing contents). For example, the direct-view observation image 111 can be an IHb image, and the side-view observation image 112 can be a narrow-band optical image.

In the ninth embodiment, the direct-view observation image 111 and the side-view observation image 112 are acquired (imaged) at different timings, but the direct-view observation image 111 and the side-view observation image 112 are acquired at the same time for direct-view observation. At least one of the image 111 and the side view observation image 112 can be a special image. For example, white light is used as the illumination light to image the observation target, and the direct observation image 111 and the side observation image 112 are acquired at the same time. Then, the direct-view observation image 111 is subjected to normal image processing (image processing for making a normal image) to make the direct-view observation image 111 a normal image. On the other hand, the lateral observation image 112 obtained at the same time as the direct observation image 111 is subjected to a color conversion process for emphasizing the lesion to make the lateral observation image 112 a special image. In this way, when a special image is generated by image processing without the need for switching the illumination light, even if the direct-view observation image 111 and the side-view observation image 112 are acquired at the same time, at least one of them is a special image. Can be done.

Further, if the direct-viewing illumination unit 54, the direct-viewing illumination unit 81, and the side-viewing illumination unit 43 are configured to be independent of each other, different illumination lights are provided for the field of view 501 of the direct-viewing observation unit 41 and the field of view 502 of the side-viewing observation unit 42. Can be irradiated at the same time. In this case, even if it is necessary to switch the illumination light to generate the special image, the direct-view observation image 111 and the side-view observation image 112 can be acquired at the same time, and at least one of them can be a special image.

In the ninth embodiment, at least one of the direct-view observation image 111 and the side-view observation image 112 is a special image, but instead, or at least one of the direct-view observation image 111 or the side-view observation image 112 is used. After making a special image, the brightness of one of the direct-view observation image 111 or the side-view observation image 112 may be brighter or darker than the brightness of the other. For example, the side-view observation image 112 can be made relatively brighter than the direct-view observation image 111. The method of making the side view observation image 112 brighter than the direct view image 111 is a method of making the side view observation image 112 brighter than the direct view image 111 and a method of making the side view observation image 111 brighter than the side view observation image 112. Can also take a method of making it relatively dark. These can be realized by adjusting the relative balance between the amount of direct-view illumination light and the amount of side-view illumination light in the modified example of the eighth embodiment (see FIG. 45) by the control unit 96. The ninth embodiment can be implemented in any combination with the first embodiment and the like.

Further, the ratio of the amount of light for each component contained in the illumination light (hereinafter referred to as the light amount ratio) can be changed between when the direct-view observation image 111 is acquired and when the side-view observation image 112 is acquired. For example, the light source 91 is composed of a purple LED that emits purple light, a blue LED that emits blue light, a green LED that emits green light, and a red LED that emits red light, and the purple light and blue light. By adjusting the amount of green light and the amount of red light, the light source 91 can emit a plurality of types of illumination light having different light amount ratios. Therefore, while the white light is emitted when the direct observation image 111 is obtained, the light amount ratio of the purple light, the blue light, the green light, and the red light is adjusted when the side view observation image 112 is obtained. For example, illumination light for special images in which the component of purple light is relatively large compared to white light (white light with a large amount of purple component, depending on the amount of purple light, it may become almost purple light as a whole). Can emit light.

[10th Embodiment]
In the first embodiment and the like, the display area and the display position of the direct-view observation image 111 and the side-view observation image 112 in the display image 113 and the like are predetermined. In particular, in the fourth embodiment and its modifications, the display position of the direct-view observation image 111 or the side-view observation image 112 is adjusted according to the orientation of the tip portion 12d or the like, but the direct-view observation image in the display image 113 or the like is adjusted. The display areas of 111 and the side view observation image 112 are predetermined. However, the display areas of the direct-view observation image 111 and the side-view observation image 112 can be arbitrarily determined.

The display area of the direct-view observation image 111 and the side-view observation image 112 is the range of the direct-view observation image 111 and the side-view observation image 112 to be displayed on the monitor 18 by the display image 113 or the like. For example, in the display image 113, since the direct view observation image 111 and the center C0 of the side view observation image 112 are arranged at the center of the display image 113, a part of the side view observation image 112 which is originally annular is displayed. It protrudes outside the image 113 and is not displayed on the display image 113. On the other hand, the range used for the display image 113 of the side view observation image 112 is the display area of the side view observation image 112.

When adjusting the display ranges of the direct-view observation image 111 and the side-view observation image 112 with respect to the display image 113 and the like, the display control unit 98 is provided with a display area setting unit 1001 as shown in FIG. 47. As shown in FIG. 48, the display area setting unit 1001 adjusts the relative positions and sizes of the display area 1002 on the monitor 18 and the direct-view observation image 111 and the side-view observation image 112. As a result, the direct-view observation image 111 and the side-view observation image 112 are stored in the display image 113 or the like, and as a result, an area to be displayed on the monitor 18 (that is, a display area) is set. The size and shape of the display area 1002 are the size and shape of the display image 113 and the like, and in the present embodiment, it is assumed that they are predetermined by settings and the like.

For example, if the display area setting unit 1001 sets the mutual relationship between the direct view observation image 111 and the side view observation image 112 and the display area 1002 as shown in FIG. 48, the display image generation unit 102 sets the display area. The display image 1013 shown in FIG. 49 is generated and displayed on the monitor 18 according to the mutual relationship between the direct view observation image 111 and the side view observation image 112 set by the unit 1001 and the display area 1002.

In the tenth embodiment, the display control unit 98 offsets the display positions of the direct-view observation image 111 and the side-view observation image 112 with respect to the display area 1002 on the monitor 18 which is the display unit, thereby causing the second protrusion. The entire first blind spot portion 114 corresponding to the blind spot in which 32 appears is hidden. Therefore, since the first blind spot portion 114 can be removed from the display on the monitor 18, the display on the monitor 18 can concentrate on the observation of the observation target without worrying about the existence of the first blind spot portion 114 at all. Can be done.

In the tenth embodiment, the entire first blind spot portion 114 is hidden, but when the display area setting unit 1001 is provided in the display control unit 98, the display control unit 98 is a monitor which is a display unit. It is preferable to hide at least a part of the blind spot by offsetting the display positions of the direct view observation image 111 and the side view observation image 112 with respect to the display area 1002 in 18. Instead of offsetting the display positions of the direct-view observation image 111 and the side-view observation image 112 with respect to the display area 1002 as described above, or the display positions of the direct-view observation image 111 and the side-view observation image 112 with respect to the display area 1002. The first blind spot portion 114 corresponding to the blind spot in which the second protruding portion 32 appears by enlarging the direct view observation image 111 and the side view observation image 112 with respect to the display area 1002 on the monitor 18 which is the display unit. At least part of can be hidden. If even a part of the first blind spot portion 114 is hidden, it becomes possible to concentrate on the observation of the observation target rather than the case where the entire first blind spot portion 114 such as the display image 113 fits in the display area 1002. Is. For example, as shown in FIG. 50, the display control unit 98 can hide a part of the first blind spot portion 114 by enlarging the direct view observation image 111 and the side view observation image 112 with respect to the display area 1002.

Further, as shown in FIG. 51, if the display control unit 98 enlarges the direct view observation image 111 and the side view observation image 112 with respect to the display area 1002, at least a part of the second blind spot portion 115 can be hidden. .. Therefore, it becomes possible to concentrate on the observation of the observation target rather than the display image 113 or the like.

It is preferable that the first blind spot portion 114 and the second blind spot portion 115 be hidden by the offset or enlargement as the display mode. That is, the display control unit 98 is in addition to the standard mode, the direct view magnifying mode, or the side view magnifying mode in the first embodiment or the like, or the standard mode, the direct view magnifying mode, or the side view in the first embodiment or the like. It is preferable to have a blind spot hiding mode instead of the magnifying mode. When the display mode is set to the blind spot non-display mode, at least a part of the first blind spot portion 114 and the second blind spot portion 115 corresponding to the blind spot in which the second protrusion 32 is captured is hidden and directly observed. The image 111 and the side view observation image are displayed on the monitor 18 which is a display unit.

The display control unit 98 can change the ratio of the portion to be hidden from the first blind spot portion 114 corresponding to the blind spot in which the second protrusion 32 appears. For example, as shown in FIG. 52, when the processor device 16 is provided with the motion detection unit 201, the display area setting unit 1001 has the first blind spot portion 114 in accordance with the movement of the tip portion 12d (that is, the movement of the insertion portion 12a). Alternatively, the ratio of the portion to be hidden from the second blind spot portion 115 may be changed.

More specifically, when the motion detection unit 201 detects the insertion / removal of the insertion unit 12a, the display area setting unit 1001 sets the position according to the insertion / removal speed (or acceleration) of the insertion unit 12a as shown in FIG. 53. The ratio of the hidden portion of the 1 blind spot portion 114 or the second blind spot portion 115 can be set. That is, the faster the insertion portion 12a is inserted and removed, the greater the ratio of hiding the first blind spot portion 114 or the second blind spot portion 115. In this way, the display area of the first blind spot portion 114 or the second blind spot portion 115 becomes smaller as the insertion portion 12a is inserted and removed faster and the first blind spot portion 114 or the second blind spot portion 115 tends to interfere with the observation. Therefore, it is possible to support the insertion / removal of the insertion portion 12a.

In FIG. 53, the proportion of the first blind spot portion 114 or the second blind spot portion to be hidden is changed due to the speed of insertion and removal regardless of the insertion / removal direction of the insertion portion 12a. However, as shown in FIG. 54, for example, the display control unit 98 has a first blind spot portion 114 or a second blind spot portion 114 or a second in the case where the insertion portion 12a is inserted into the observation target and the case where the insertion portion 12a is removed from the observation target. You can change the percentage of the blind spot that is hidden. In addition to the above, the display control unit 98 hides the first blind spot portion 114 or the second blind spot portion according to the direction of the tip portion 12d or the speed, speed, or acceleration of changing the direction of the tip portion 12d. It is possible to change the ratio of the part to be converted.

The tenth embodiment can be implemented in any combination with the first embodiment and the like. For example, the display control unit 98 adjusts the display ratio of the side view observation image 112 and the direct view observation image 111 in the same manner as in the first embodiment and the like, and displays the display ratio of the side view observation image 112 and the direct view observation image 111. The ratio of the portion to be hidden from the first blind spot portion 114 or the second blind spot portion 115 can be changed according to the above. In the third embodiment, it is the same as the case where the first blind spot portion 114 is reduced according to the display ratio of the side view observation image 112 and the direct view observation image 111.

When the first blind spot portion 114 or the second blind spot portion 115 is hidden (particularly when the blind spot non-display mode is provided) as in the tenth embodiment, the display control unit 98 displays the side view observation image 112. Is sprayed onto the side view observation unit 42 to clean the side view observation unit 42 in the non-display area (the area outside the display area 1002) that is hidden when the display unit is displayed on the monitor 18. It is preferable to have a nozzle 51 and a nozzle 52. In this case, since the nozzle 51 and the nozzle 52 do not necessarily have to be provided in the second protruding portion 32, the cleanability of the side view observation portion 42 may be improved depending on the arrangement of the nozzle 51 and the nozzle 52.

The endoscope 12 of the first embodiment or the like is inserted into the observation target while the first protruding portion 31 and the second protruding portion 32 further protrude from the tip surface 21 of the tip portion 12d while exposing the complicated shape. In the endoscope 12 capable of direct view observation and side view observation, for example, as shown in FIG. 55, the tip portion 12d (particularly the portion of the first protrusion 31 and the second protrusion 32) is covered with the tip cap 1101. Can be used. When the tip cap 1101 is used, the shape of the exposed portion is simpler than that of the tip portion 12d having the first protruding portion 31 and the second protruding portion 32, so that dirt is less likely to remain and even if dirt adheres, it will be removed. Cheap. Further, since the tip cap 1101 is provided with a through hole 1102 at the position of the forceps opening 82, the treatment tool or the like can project from the through hole 1102 to the outside of the tip cap 1101.

Since the tip cap 1101 is transparent, basically, the direct-view illumination unit 54, the direct-view illumination unit 81, and the direct-view observation unit 41 are used for direct-view observation, and the side-view illumination unit 43 and the side-view observation unit 42 are used. Does not interfere with the lateral observation used. However, although the tip cap 1101 is transparent, the image to be observed may be distorted in the vicinity of the contour 1101A or the contour 1101A of the tip cap 1101, for example. Therefore, as shown in FIG. 56, in the display control unit 98, the contour 1101A of the tip cap 1101 is placed at a position where distortion may appear in the image to be observed in the vicinity of the contour 1101A or the contour 1101A of the tip cap 1101. It is preferable to generate a display image 1120 displaying a line 1111 (for example, a colored line or a band) indicating that the object is present and display it on the monitor 18. If the presence of the contour 1101A of the tip cap 1101 is shown in the display image 1120, the doctor or the like can recognize that the image in the vicinity of the line 1111 or the like is distorted, so that misidentification or the like can be prevented.

Instead of displaying the line 1111 or the like indicating the presence of the contour 1101A of the tip cap 1101 as described above, as shown in FIG. 57, the display control unit 98 may distort the image to be observed. A display image 1121 that masks the contour 1101A of the cap 1101 or the portion 1122 in the vicinity of the contour 1101A may be generated and displayed on the monitor 18. The image to be observed may be distorted in the portion 1123 of the through hole 1102 as well as the contour 1101A of the tip cap 1101 or the portion 1122 in the vicinity of the contour 1101A. Therefore, the display control unit 98 may also mask the portion 1123 of the through hole 1102 (see FIG. 57). Further, similarly to FIG. 56, a line or the like indicating the existence of the through hole 1102 may be displayed on the portion 1123 of the through hole 1102. Although the tip cap 1101 has a cylindrical shape, a cannonball-shaped (tapered shape) tip cap in which the tip side gradually becomes smaller can also be used. The structure of the cannonball-shaped tip cap is the same as that of the tip cap 1101 except for the outer shape.

In addition to the tip cap 1101, as shown in FIG. 58, the endoscope 12 is provided with an auxiliary member 1201 for expanding the folds of the lumen (large intestine, etc.) to be observed at the tip 12d. May be used. The auxiliary member 1201 has a substantially cylindrical shape for attachment to the tip portion 12d, and has a plurality of wing members 1202 for expanding the folds of the lumen by moving the insertion portion 12a in the removal direction. Then, in order to prevent the auxiliary member 1201 from being displaced with respect to the insertion portion 12a during observation, or to prevent the auxiliary member 1201 from coming off from the insertion portion 12a during observation, the auxiliary member 1201 is at the tip thereof. On the side, there is a locking member 1206 for locking the auxiliary member 1201 to the tip portion 12d. The locking member 1206 protrudes from the original tip portion 12d for the convenience of locking the auxiliary member 1201 to the tip portion 12d. Therefore, depending on the amount of protrusion of the locking member 1206, the locking member 1206 may enter the field of view 502 of the side view observation unit 42. Therefore, among the auxiliary members 1201, at least the locking member 1206 is preferably made of a transparent material. This is because if the locking member 1206 is transparent, its presence is inconspicuous even if it enters the field of view 502 of the side view observation unit 42.

Further, as shown in FIG. 59, the auxiliary member 1201 may have a claw-shaped member (hereinafter, referred to as a claw member) 1207 that locks the auxiliary member 1201 to the tip portion 12d instead of the locking member 1206. is there. In this case, the claw member 1207 is preferably made of a transparent material as described above. Further, unlike the locking member 1206, the claw member 1207 is partially provided. Therefore, it is preferable that the claw member 1207 is provided at the blind spot of the lateral observation unit 42 (the position where the first blind spot portion 114 is inserted). This is because if the claw member 1207 is provided only in the blind spot of the side view observation unit 42, the claw member 1207 will not be reflected in the field of view 502 of the side view observation unit 42. In this case, the claw member 1207 does not have to be made of a transparent material.

In the first embodiment and the like, the second protrusion 32 forms a blind spot in the side view observation unit 42, but as shown in FIG. 60, the side view observation unit 42 is formed on the lower surface of the second protrusion 32. An observation unit (hereinafter referred to as a bottom surface observation unit) 1301 may be provided to compensate for the blind spot. The bottom surface observation unit 1301 has the same configuration as the side view observation unit 42. The portion exposed on the lower surface of the second protruding portion 32 of the lower surface observation portion 1301 is the lower surface observation window 1301A.

In the first embodiment and the like, the observation target is displayed almost completely with respect to the display image 113 and the like, but the display screen of the monitor 18 in recent years is often horizontally or vertically long. When such a horizontally long monitor 18 (the same applies hereinafter) is used, the direct view observation image 111 and the side view observation image 112 are appropriately placed in the horizontally long display area as shown in the display image 1401 shown in FIG. Even if the image is displayed on the screen, there may be a surplus area area (hereinafter referred to as a margin area) 1402 in which only the second blind spot portion 115 is displayed and no useful information is displayed. The appropriate display is, for example, a display in which the portion outside the display area is as small as possible and the second blind spot portion 115 is made as small as possible.

As described above, when a margin area 1402 is formed on the display screen due to the aspect ratio of the display screen of the monitor 18, the display control unit 98 sets the margin as shown in the display image 1403 shown in FIG. 62. It is preferable to provide the information display area 1404 in the area 1402. Then, in the information display area 1404, information such as a range of the focusing angle of view is displayed. In order to display the range of the in-focus angle of view, as shown in FIG. 63, the processor device 16 is provided with the in-focus angle of view detection unit 1405. The in-focus angle of view detection unit 1405 acquires, for example, a direct-view observation image 111 and a side-view observation image 112 from the image acquisition unit 97, and uses the acquired direct-view observation image 111 and side-view observation image 112 to obtain a direct-view observation image. The in-focus (in-focus) region is detected in the 111 or the side-view observation image 112, and the result is converted into the angle of view. The display control unit 98 displays the in-focus angle of view calculated by the in-focus angle of view detection unit 1405 in the information display area 1404. The detection of the in-focus area using the direct-view observation image 111 and the side-view observation image 112 can be performed by a contrast method, a phase difference method, or the like, similar to the principle of detecting focus in the autofocus function of a digital camera or the like. ..

In the above, as an example, the focusing angle of view is displayed in the information display area 1404, but other information may be displayed. That is, if the information display area 1404 is provided in the margin area 1402, some information can be displayed in the information display area 1404, and at least a part of the margin area 1402 can be effectively utilized, the content of the displayed information is arbitrary. For example, as in the display image 1411 shown in FIG. 64, in the information display area 1404 of the margin area 1402, a deformed side view observation image 1412 in which the portion of the side view observation image 112 in which the observation target is captured is transformed into a rectangle. It may be displayed. The reference numerals Lα and Lβ shown in the side view observation image 112 and the modified side view observation image 1412 indicate the correspondence between the side view observation image 112 and the modified side view observation image 1412. In this way, when the modified side view observation image 1412 is displayed in the information display area 1404, only the direct view observation image 111 is displayed in the main area 1416 for displaying the observation target as in the display image 1413 shown in FIG. May be displayed.

In addition, even if there is no margin area 1402, if there is a first blind spot portion 114, an information display area 1502 is provided in the first blind spot portion 114 as shown in the display image 1501 shown in FIG. 66, and an information display area is provided. If information such as the model name of the endoscope 12 is displayed on the 1502, the first blind spot portion 114 can be effectively utilized.

In the first embodiment and the like, the direct-view observation image 111 and the side-view observation image 112 are used to generate the display image 113 and the like and display them on the monitor 18, but the direct-view observation image 111 or the side-view observation image 112 3D model (three-dimensional model) 1605 can be generated using. In this case, as shown in FIG. 67, for example, the processor device 16 may be provided with a 3D model creation unit 1601 and a storage unit 1602. As shown in FIG. 68, the 3D model creation unit 1601 generates the 3D model 1605 to be observed by using the direct observation image 111 and the side observation image 112. The 3D model 1605 created by the 3D model creation unit 1601 is generated. Then, the storage unit 1602 stores the 3D model 1605 created by the 3D model creation unit 1601. The 3D model 1605 stored in the storage unit 1602 in this way can be used for support for the next diagnosis or the like, or for re-examination by simulation or the like. In the above, the 3D model creation unit 1601 and the storage unit 1602 are provided in the processor device 16, but the 3D model creation unit 1601 and the storage unit 1602 may be provided in a computer or the like connected to the endoscope system 10.

The first embodiment and the like have been described by taking as an example an endoscope system 10 used by inserting the insertion portion 12a into the subject, but in the first embodiment and the like, the subject swallows. It can also be used for the capsule endoscope used.

In the first embodiment and the like, the direct view observation unit 41 and the side view observation unit 42 share the image sensor 61 and the image sensor 66, but the image sensor 61 and the image sensor 66 constituting the direct view observation unit 41 The image pickup lens 61 and the image sensor 66 constituting the side view observation unit 42 may be separate from each other.

In the first embodiment and the like, the direct-view observation unit 41 and the side-view observation unit 42 include a common image sensor 66. Therefore, as shown in FIG. 69, the direct-view observation image 111 is displayed on the imaging surface 1701 of the image sensor 66. It is possible to distinguish between the direct-view imaging area 1711 that captures the image, the side-view imaging area 1712 that captures the direct-view observation image 112, and the other area (the area that does not image the observation target) 1715. Therefore, in the first embodiment or the like, signal processing such as gain processing performed when the sensitivity of the image sensor 66 or the signal is read from the image sensor 66 is performed at least in the direct view imaging area 1711 and the side view imaging area 1712. (Hereinafter, simply referred to as signal processing) can be changed.

For example, the direct-view imaging area 1711 can read signals one pixel at a time, and the side-view observation area 1712 can read a plurality of pixels in combination (so-called binning). In this way, the side-view observation image 112 obtained by imaging in the side-view imaging area 1712 has higher sensitivity than the direct-view observation image 111 obtained by imaging in the direct-view imaging area 1711. As a result, it may be easier to find the lesion in the lateral observation image 112 than in the case where the binning is not read out in the lateral imaging area 1712.

Further, the gain applied when reading the signal from the pixels of the side-view imaging area 1712 can be made larger than the gain applied when reading the signal from the pixels of the direct-view imaging area 1711. In this way, the side-view observation image 112 obtained by imaging in the side-view imaging area 1712 has higher sensitivity than the direct-view observation image 111 obtained by imaging in the direct-view imaging area 1711. As a result, it becomes easier to find a lesion in the lateral observation image 112 than in the case where the gain processing using the same value of the gain is performed in the direct visual imaging area 1711 and the lateral imaging area 1712.

In the image sensor 66, all the effective pixels have the same size, but instead of the image sensor 66, an image sensor having a different pixel size between the direct-view imaging area 1711 and the side-view imaging area 1712 can be used. For example, if an image sensor in which the pixels of the side-view imaging area 1712 are larger than the pixels of the direct-view imaging area 1711 (for example, the size of the side is doubled and the area is quadrupled) is used, it does not depend on binning or gain processing. In addition, the sensitivity of the side view observation image 112 can be improved. On the contrary, if an image sensor in which the pixels of the side-view imaging area 1712 are smaller than the pixels of the direct-view imaging area 1711 is used, the resolution of the side-view observation image 112 is improved.

Further, it is possible to use an image sensor in which the color of the color filter, the number ratio of the colors, or the arrangement is different between the direct view imaging area 1711 and the side view imaging area 1712. For example, in the direct view imaging area 1711, the vertically and horizontally adjacent 4-pixel color filters are an array of BG × GR (so-called Bayer array), and in the side view imaging area 1712, the image sensor is an array of BG × RB. Can be used. This image sensor has more B pixels in the side view imaging area 1712 than in the direct view imaging area 1711. Therefore, if this image sensor is used, an image that can be easily captured by the B pixels can be observed in the side view observation image 112. It will be easier to do.

As described above, changing the pixel size between the direct-view imaging area 1711 and the side-view imaging area 1712, or changing the color of the color filter, etc., is different for each of the direct-view observation unit 41 and the side-view observation unit 42. It is particularly suitable when mounting an image sensor.

In addition, in the first embodiment and the like, the image pickup lens 61 common to the direct observation unit 41 and the side observation unit 42 has a shorter focal length in the side observation unit 42 than the focal length in the direct observation unit 41. It is preferable to keep it. This is because when observing the lumen, the lateral observation unit 42 is often closer to the cavity as the subject than the direct observation unit 41.

10,200 Endoscope system 11 Universal cord 12 Endoscope 12a Insertion part 12b Operation part 12c Curved part 12d Tip part 12e Angle knob 13 Cleaning switch 14 Light source device 16 Processor device 17 Tank 18 Monitor 19 Console 21 Tip surface 31 First Projection 32 Second projection 41 Direct observation window 41A Direct observation window 42 Side observation observation window 42A Side observation window 43 Side illumination illumination unit 43A First side illumination window 43B Second side illumination window 51, 52, 53 Nozzle 54 Direct-view illumination unit 54A Direct-view illumination window 59 Conversion unit 61 Imaging lens 62 Front group lens 63 Mirror lens 64 Rear group lens 66 Image sensor 67 Cover glass 71, 77, 84, 93 Light guide 72 Reflection member 73 Filling member 76 Air supply Liquid channel 78 Illumination lens 81 Direct illumination window 81A Direct illumination window 82 Force opening 91 Light source 92 Light source control unit 96 Control unit 97 Image acquisition unit 98 Display control unit 101 Display ratio setting unit 102 Display image generation unit 111 Direct observation image 111A, 112A Graph 112 Side view observation image 113 Display image 114 First blind spot part 115 Second blind spot part 123, 126, 128, 129, 302, 304, 410, 411, 523, 1121, 121, 1401, 1403, 1411, 1413 , 1501 Display image 201 Motion detection unit 202 Motion sensor 205 Observation target 301 Mask display ratio setting unit 401 Display position setting unit 421 Head mount display 501, 502 Field 503 Blind spot 513 Third blind spot part 601 Overlapping field 602 Overlapping area 611 Overlapping area Detection unit 612 Overlapping area removal unit 613 Signal level correction unit 701, 702 Distortion 705 Abrasion connection unit 706 Distance measurement unit 830 Image processing unit 1001 Display area setting unit 1002 Display area 1013 Display image 1101 Tip cap 1101A Contour 1102 Through hole 1111 1122 Part near the contour of the tip cap 1123 Through hole part 1201 Auxiliary member 1202 Wing member 1206 Locking member 1207 Claw member 1301 Bottom surface observation unit 1301A Bottom surface observation window 1402 Margin area 1404, 150 2 Information display area 1405 Focused angle of view detection unit 1412 Deformed side view observation image 1416 Main area 1601 3D model creation unit 1602 Storage unit 1605 3D model 1701 Imaging surface 1711 Direct vision imaging area 1712 Side view imaging area 1715 Area where the observation target is not imaged C0 Center D1 Distance Lα, Lβ Code V1, V2, V3 Movement speed

Claims (12)

An insertion part to be inserted into an observation target, a direct-view observation part having a field of view in the direction of the tip of the insertion part, a side-view observation part having a field of view in the side surface direction of the insertion part, and a side-view observation part protruding from the insertion part. An endoscope having a protruding part that forms a blind spot in the field of view of the part,
An image acquisition unit that acquires a direct observation image using the direct observation unit and acquires a lateral observation image using the lateral observation unit.
A distance measuring unit that measures the distance between the tip of the endoscope and the observation target,
Using the distance between the tip and the observation target, at least one of the direct observation image and the lateral observation image is subjected to a conversion process to connect the aberrations of the direct observation image and the lateral observation image. The aberration connection part that converts to the obtained image,
A display unit for displaying the direct-view observation image and the side-view observation image in which aberrations can be connected by the conversion process of the aberration connection unit.
Endoscopic system with. The endoscope system according to claim 1, wherein the conversion process is an affine transformation. The endoscope system according to claim 1 or 2, wherein the aberration connecting portion converts the distortion of the direct-view observation image and the side-view observation image into a connectable image. Before SL aberration connections, using the distance of the observation target and the distal end portion, the endoscope system according to any one of claims 1 to 3 to adjust or change the parameters used in the conversion process. The endoscope system according to claim 4, wherein the distance measuring unit measures the distance between the tip portion and the observation target at the boundary between the visual field of the direct visual observation unit and the visual field of the lateral visual observation unit. The endoscope system according to claim 4 or 5, wherein the aberration connecting portion adjusts or changes the parameter according to an average value of a distance between the tip portion and the observation target. The endoscope system according to claim 4 or 5, wherein the aberration connecting portion adjusts or changes the parameter for each position where the direct observation image and the side observation image are connected. The endoscope system according to any one of claims 4 to 7, wherein the aberration connecting portion adjusts or changes the parameter by using the relative angle between the tip portion and the observation target. The endoscope system according to claim 8, wherein the aberration connecting portion obtains a relative angle between the tip portion and the observation target by using the distribution of the distance between the tip portion and the observation target. The endoscope system according to claim 8 or 9, wherein the aberration connecting portion adjusts or changes the parameter according to an average value of the relative angles between the tip portion and the observation target. The endoscope system according to claim 8 or 9, wherein the aberration connecting portion adjusts or changes the parameter for each position where the direct observation image or the side observation image is connected. An insertion part to be inserted into an observation target, a direct observation part having a field of view in the direction of the tip of the insertion part, a side view observation part having a field of view in the side surface direction of the insertion part, and a side view observation part protruding from the insertion part. An endoscope having a protruding portion forming a blind spot in the field of view of the portion and the direct-view observation unit are used to acquire a direct-view observation image, and the side-view observation unit is used to acquire a side-view observation image. An endoscopic system having an image acquisition unit, a distance measuring unit that measures the distance between the tip of the endoscope and the observation target, and a display unit that displays the direct observation image and the side observation image. In the driving method
The aberration connecting portion performs conversion processing on at least one of the direct-view observation image and the side-view observation image by using the distance between the tip portion and the observation target, and the direct-view observation image and the side-view observation image. Steps to convert the aberrations into a connectable image,
A step of displaying the direct-view observation image and the side-view observation image in which the display unit is in a state where aberrations can be connected by the conversion process of the aberration connection unit.
How to drive an endoscopic system.

Images