JP2011062328A - Endoscope apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an endoscope apparatus for enabling endoscopic observations of a plurality of visual fields at the same time. <P>SOLUTION: The endoscope apparatus has: a visual field combining means for combining first light flux for forming an image of a first visual field on the imaging surface of an imaging means with second light flux for forming an image of a second visual field while changing strength ratio; and a visual field separating means for generating image data of the first visual field and image data of the second visual field on the basis of at least two image data of images formed on the imaging surface by respective combined light fluxes different in strength ratio. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an endoscope apparatus, and more particularly to an endoscope apparatus that can simultaneously observe a plurality of fields of view.

  Conventionally, endoscopes are widely used in medical and industrial fields in order to perform observation and treatment in a narrow space such as in a body cavity. In recent years, electronic endoscopes in which a solid-state image sensor is mounted at the distal end of an insertion portion are particularly widespread with the downsizing and higher performance of solid-state image sensors such as CCD image sensors and CMOS image sensors. A general electronic endoscope has a glass plate (optical aperture), an objective lens, and a solid-state image sensor in the longitudinal direction along the optical axis at the distal end of a flexible and slender cable-shaped insertion portion. It is arranged so that the area in front of the distal end of the insertion portion can be observed. However, when observing organs with a large number of folds such as the small intestine, for example, a blind spot is formed on the back side of the fold, so that there are cases where sufficient observation cannot be performed only by forward observation. Therefore, an endoscope attachment has been proposed that enables simultaneous observation of the front visual field and the side visual field by attaching to the distal end of the endoscope as described in Patent Document 1. The endoscope attachment described in Patent Literature 1 has a hyperboloid convex mirror with an opening formed in the center, and an image in front of the endoscope tip passes through the opening of the convex mirror to the center of the imaging surface. The side image is reflected by the convex mirror and formed on the imaging surface around the front image. Therefore, the front view and the side view are simultaneously displayed on one screen, and the front view and the side view can be performed simultaneously.

  By the way, in the technical field of window materials used for buildings and vehicles, various so-called “light control glasses” that control sunlight transmitted through the window glass have been proposed. The light control glass includes a light control mirror that performs light control by absorbing light and a light control mirror that performs light control by reflecting light. In the dimming mirror, a film having a property that the reflectance continuously and reversibly changes between a mirror state having a high reflectance and a transparent state having a low reflectance and a high transmittance is formed on a transparent substrate. (Patent Document 2). Such a light control mirror is mainly used for controlling the solar energy flowing into the room from the window glass to obtain an energy saving effect.

WO2006-004083 Japanese Patent Application No. 2005-274630

  In the medical field, it is important to make the insertion portion of the electronic endoscope thin so that the subject can receive an endoscopic examination without pain. One of the main factors that determine the outer diameter of the electronic endoscope is the external dimensions of the solid-state image sensor, particularly the size of the light receiving surface. For this reason, the size of the imaging surface of the solid-state imaging device used in the electronic endoscope is strictly limited, and therefore the number of effective pixels of the solid-state imaging device is also limited. In the endoscope apparatus described in Patent Document 1, since the front observation image and the rear observation image are divided into regions on the imaging surface of the solid-state imaging device having a limited number of effective pixels, There is a problem that the resolution or viewing angle is lowered.

  The present invention has been made in view of the above circumstances. That is, an object of the present invention is to provide an endoscope apparatus that enables simultaneous observation of a plurality of fields of view without reducing the resolution and the angle of view.

  According to the present invention, an endoscope apparatus capable of simultaneously observing a plurality of visual fields is provided. The endoscope apparatus according to the embodiment of the present invention has an intensity ratio between a first light flux connecting an image of a first field of view and a second light flux connecting an image of a second field of view on an imaging surface of an imaging unit. Based on at least two image data of the image formed on the imaging surface by the visual field synthesis means for synthesizing while changing and each of the combined luminous fluxes having different intensity ratios, the image data of the first visual field and the second visual field Visual field separation means for generating image data.

  According to the endoscope apparatus having such a configuration, it is possible to simultaneously observe a plurality of fields of view without reducing the resolution and the viewing angle.

  The visual field synthesis unit may be configured to periodically change the intensity ratio at a period longer than the frame period of the imaging unit. In this case, the visual field separation unit can generate image data of the first and second visual fields based on at least two image data captured in phases having different intensity ratios.

  According to the above configuration, since the intensity ratio at the time is easily known from the time when each image is captured (phase of intensity ratio modulation), the images can be separated by simple processing.

  The visual field synthesizing means may include an optical element having a variable reflectance and transmittance. In this case, it is desirable that the first light beam is transmitted through the optical element and the second light beam is reflected by the optical element. Further, it is desirable that the visual field synthesis unit is configured to change the intensity ratio by changing the reflectance and transmittance of the optical element. With such a configuration, simultaneous observation of a plurality of fields of view is possible with an extremely simple structure.

の解として
フレーム時刻ｔにおける第１の視野の画像データｉｍｇ_ｐ（ｘ，ｙ，ｔ）、及び
フレーム時刻ｔにおける第２の視野の画像データｉｍｇ_ｒ（ｘ，ｙ，ｔ）
を生成するように構成されてもよい。 The visual field separation means is
Image data img (x, y, t) captured at frame time t,
Image data img (x, y, t−1) captured at frame time t−1,
Reflectance of the optical element in the frame time t alpha _t, and based on the reflectivity alpha _t-1 of the optical elements in the frame time t-1,
Simultaneous equations consisting of the following equations (1) and (2) (however, approximation of equations (3) and (4) is performed)

As the solution of the first visual field image data img _p (x, y, t) at the frame time t and the second visual field image data img _r (x, y, t) at the frame time t.
May be configured to generate.

  According to the above configuration, the image data of the first and second visual fields can be separated by simple calculation, and processing can be performed using an inexpensive arithmetic device with low processing performance.

  As for an optical element, it is desirable for the reflective surface to be formed in the shape of a two-leaf hyperboloid. In this case, the reflected image can be easily converted into a panoramic image or a projected image.

  The optical element may include a metal thin film having such a property that the reflectance is reduced by hydrogenation and the reflectance is increased by dehydrogenation. The first field of view may include a front field of view, and the second field of view may include a rear field of view.

  The endoscope apparatus according to the embodiment of the present invention enables simultaneous observation of a plurality of fields of view without reducing resolution or viewing angle.

It is a disassembled side view which shows the external appearance of the principal part of the endoscope apparatus which concerns on embodiment of this invention. 1 is a block diagram illustrating a schematic configuration of an endoscope apparatus according to an embodiment of the present invention. It is the figure which expanded the vicinity of the front-end | tip of the insertion part of the endoscope which concerns on embodiment of this invention. It is the figure which expanded the vicinity of the front-end | tip of the insertion part of the endoscope which concerns on embodiment of this invention. It is a figure explaining the outline of the film | membrane structure of the variable optical element which concerns on embodiment of this invention. It is a timing chart explaining operation | movement of the simultaneous observation mode which concerns on embodiment of this invention. It is a flowchart explaining the process which the endoscope apparatus which concerns on embodiment of this invention performs. It is a flowchart explaining the detail of the image separation process which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention. It is a flowchart explaining the detail of the image compositing process which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention. It is a figure explaining the image processing which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is an exploded side view showing an external appearance of a main part of an endoscope apparatus 1 according to an embodiment of the present invention. FIG. 2 is a block diagram showing a schematic configuration of the endoscope apparatus 1. The endoscope apparatus 1 includes an endoscope (electronic scope) 10, a processor 20, and a monitor 30.

  The electronic scope 10 inserts a flexible cable-shaped insertion portion 14 into a body cavity of a subject, and images an observation site in the body cavity with an imaging device provided at the distal end of the insertion portion 14. Is a unit. The electronic scope 10 is divided into a connection portion 11, a universal cable portion 12, an operation portion 13, and an insertion portion 14 in order from the proximal end side to the longitudinal direction, and is configured to be detachable from the processor 20 via the connection portion 11. . A visual field synthesis mechanism 16 according to the embodiment of the present invention is incorporated at the distal end of the insertion portion 14.

  In the electronic scope 10, a light guide (for example, an optical fiber bundle) 15 for propagating illumination light from the light source unit 22 of the processor 20 to the tip of the insertion unit 14 is disposed over the entire length along the longitudinal direction. . The light guide 15 has one line on the proximal end side where the illumination light enters, but is branched into two lines (15a, 15b) from the middle. Illumination light emitted from the exit end of the light guide illuminates the observation region (field of view) substantially uniformly via the field of view synthesis optical system 16 described later. A light distribution lens for diffusing illumination light may be disposed immediately after the exit ends of the light guides 15a and 15b. In the present embodiment, the visual field synthesis optical system 16 also serves as a light distribution lens to diffuse illumination light. The visual field synthesis mechanism 16 has an optical path for forming an image of the visual field in the front (extension direction of the distal end of the insertion portion 14) on the imaging surface and an optical path for forming an image of the visual field from the side to the rear on the imaging surface. Is a mechanism for switching continuously and periodically. Details of the visual field synthesis mechanism 16 will be described later.

  A subject (for example, an inner wall of a body cavity) located in the observation region is illuminated by the illumination light emitted from the visual field synthesis mechanism 16. A part of the illumination light irradiated on the inner wall of the body cavity is scattered (or reflected) on the surface layer of the inner wall of the body cavity and returns to the tip of the electronic scope 10. An objective lens (objective optical system) 144 is disposed at the tip of the electronic scope 10 so as to face the light receiving surface of the CCD image sensor 146. The objective lens 144 is configured to collect return light from a subject incident through the visual field synthesis mechanism 16 and form a subject image on the light receiving surface of the CCD image sensor 146. The objective lens 144 has a focal point of the other curved surface (curved surface with the opposite sign) that forms a pair with one curved surface of a bilobal hyperboloid that defines the shape of a variable optical element 161 (a component of the visual field synthesis mechanism 16) described later. It is arranged so as to coincide with the lens center (first principal point). The CCD image sensor 146 generates an image signal based on the light intensity distribution of the subject image formed on the imaging surface. The CCD image sensor 146 constitutes a single-plate color image sensor having a Bayer color filter.

  Although the objective optical system in the present embodiment is composed of a single optical lens, it is composed of an optical system including a plurality of optical lens groups and optical elements other than lenses such as optical filters and deflection elements. Also good.

  The operation unit 13 is a part that the surgeon holds to operate the electronic scope 10. The operation unit 13 is provided with various operation means such as an operation button 132 and an operation knob 134 for the operator to operate the endoscope system 1 during the operation. Further, a forceps channel 148 for inserting a treatment tool such as forceps penetrates from the forceps insertion opening 148a provided in the operation section 13 to the distal end of the insertion section 14, and the operator inserts the treatment tool from the insertion opening 148a. It is inserted into a forceps channel and sent to an observation site, and tissue collection and various treatments can be performed using a treatment tool.

  As shown in FIG. 1, the connection portion 11 is provided with a light guide connection plug 19 and a signal line connection plug 18. By inserting the light guide connection plug 19 into a light guide connection socket 29 provided in the processor 20, the electronic scope 10 and the processor 20 are optically connected. Further, by inserting the signal line connection plug 18 into the signal line connection socket 28 provided in the processor 20, various signal lines and power lines of the electronic scope 10 are electrically connected to corresponding wirings in the processor 20. The connection unit 11 is provided with well-known engaging means (not shown) (for example, a screw mechanism or a hook mechanism) for detachably attaching the electronic scope 10 to the processor 20.

  The connection unit 11 includes a CCD drive circuit 112 that supplies a CCD drive signal to the CCD image sensor 146, a signal processing circuit 114 that converts an analog image signal output from the CCD image sensor 146 into a digital video signal, and A visual field synthesis control circuit 116 that supplies a control signal to the visual field synthesis mechanism 16 is disposed. The CCD drive circuit 112 is connected to the system control circuit 21 of the processor 20 via the signal line connection plug 18, generates a CCD drive signal based on the control signal from the system control circuit 21, and passes through the wire 146a. To the CCD image sensor 146. The CCD drive signal is also connected to a visual field synthesis control circuit 116 and a video signal processing circuit 23 of the processor 20 described later (not shown), and a CCD drive signal is supplied to these circuits as a synchronization signal. The signal processing circuit 114 is connected to the video signal processing circuit 23 of the processor 20 via the signal line connection plug 18, and transmits the generated digital video signal to the video signal processing circuit 23. The visual field synthesis control circuit 116 is connected to the system control circuit 21 of the processor 20 via the signal line connection plug 18, generates a drive voltage for the visual field synthesis mechanism 16 based on the control of the system control circuit 21, and It supplies to the visual field synthesis mechanism 16 via 164a, 164b.

  The processor 20 processes the digital video signal from the system control circuit 21 that comprehensively controls each part of the processor 20 and the electronic scope 10, the light source unit 22 that generates illumination light and supplies it to the light guide 15, and the electronic scope 10. A video signal processing circuit 23 that generates and outputs a video signal, a storage unit 24 that is a large-capacity storage device such as a hard disk drive, and a touch panel that displays information necessary for operation of the endoscope apparatus 1 and receives user operations. It has a display 25, a video output interface 26 for outputting various video signals to the external monitor 30, a user input device interface 27 for connecting user input devices such as a keyboard 274 and a mouse 272, a network interface, and the like. Yes. The video signal processing circuit 23 includes an input signal determination unit 23b and an image synthesis unit 23b. Details of the video signal processing circuit 23 will be described later. The light source unit 22 condenses the illumination light emitted from the light source body 222, the automatic diaphragm mechanism 224 that adjusts the amount of illumination light coupled to the light guide 15, and the light source body 222 to the light guide 15 with high efficiency. A condensing lens 226 for coupling is provided. The light source main body 222 is composed of a control unit that controls the entire light source unit 22, a high-intensity white lamp (xenon lamp) that emits illumination light, a lamp power supply circuit that supplies driving power to the lamp, a lamp holder, and other elements (not shown). Has been.

  Next, the configuration of the visual field synthesis mechanism 16 provided at the distal end of the insertion portion 14 of the electronic scope 10 will be described in detail. The electronic scope 10 according to the embodiment of the present invention includes a front observation for observing the front from the distal end of the insertion portion 14 (a range of approximately 0 to ± 60 ° from the optical axis direction of the objective lens), and all lateral and rear azimuth directions. Side / backward observation for observing (in the range of approximately ± 60 to 160 ° from the optical axis direction of the objective lens) is possible simultaneously or individually, and the visual field synthesis mechanism 16 switches between the front visual field and the side / back visual field. It is done. Specifically, the visual field synthesis mechanism 16 switches the optical path of the illumination light and the return light from the observation site between a front optical path FP and a side / rear optical path BP, which will be described later.

  3 and 4 are enlarged views of the vicinity of the distal end of the insertion portion 14 in which the visual field synthesis mechanism 16 is incorporated. The left direction in the figure is the distal direction of the insertion portion 14. 3 shows the front optical path FP when observing the front visual field, and FIG. 4 shows the side and rear optical paths BP when observing the side and rear visual fields by arrows. . The field synthesis mechanism 16 according to the embodiment of the present invention includes a field synthesis optical system 160 and a side window 165. The field-of-view synthesizing optical system 160 includes a first transparent member 162, a second transparent member 163, and a variable optical element 161 sandwiched between both transparent members. The variable optical element 161 is a film-like optical element whose reflectivity changes depending on voltage, and is optical between a reflective state in which light is reflected with a high reflectivity and a transmissive state in which light is transmitted with a low reflectivity. It has the property that the state changes continuously and reversibly. When the variable optical element 161 is in the transparent state, a front visual field image is formed on the CCD image sensor 146 by the light bundle of the front optical path FP shown in FIG. 3, and when the variable optical element 161 is in the reflective state, FIG. A side / rear visual field image is formed on the CCD image sensor 146 by the light flux of the side / rear optical path BP shown. In addition, the variable optical element 161 can take an intermediate state between a transparent state and a reflective state. At this time, the light bundle of the front optical path FP and the light bundle of the side / rear optical path BP are applied to the variable optical element 161. Are mixed at a ratio corresponding to the applied voltage, and an image in which the front visual field image and the side / rear visual field image are superimposed is formed. Also, when the variable optical element 161 is switched between the transparent state and the reflective state, the reflectance continuously changes through an intermediate state. Details of the variable optical element 161 will be described later.

  The first transparent member 162 and the second transparent member 163 are a pair of axisymmetric members formed from a transparent optical material, and are arranged with the symmetry axis serving as the optical axis directed in the longitudinal axis direction of the insertion portion 14. Yes. The first transparent member 162 is a bowl-shaped member, and a concave surface defined by one curved surface of a two-leaf hyperboloid is formed inside. The outer surface of the first transparent member 162 is composed of a plane perpendicular to the symmetry axis and an annular curved surface surrounding the symmetry axis. The second transparent member 163 is a substantially conical member, and has a plane perpendicular to the axis of symmetry and a convex curved surface having the same bilobal hyperboloid as the inner surface of the first transparent member 162. The concave curved surface of the first transparent member 162 and the convex curved surface of the second transparent member 163 are in close contact with each other through the film-like variable optical element 161 and the adhesive layer 161f to constitute the variable optical element 161. Optical materials for forming the first transparent member 162 and the second transparent member 163 include optical glass such as crown glass, flint glass, and quartz glass, or polycarbonate (PC), polymethyl methacrylate (PMMA), and cyclic olefin resin. An optical resin such as is used.

  Next, a detailed configuration of the variable optical element 161 will be described. FIG. 5 is a diagram for explaining the outline of the film configuration of the variable optical element 161 in the present embodiment. The variable optical element 161 of the present embodiment utilizes an optical phenomenon seen in some metal thin films such as magnesium, that is, a phenomenon that reversibly changes from a state showing metal reflection to a transparent state by hydrogenation. is there. In addition, the variable optical element applicable to the embodiment of the present invention is not limited to the one using the change in optical characteristics due to hydrogenation of the metal as described above, and other physical, chemical, or mechanical. A mechanism may be used.

  The variable optical element 161 is a film-like optical element composed of a transparent electrode layer 161a, an ion storage layer 161b, a solid electrolyte layer 161c, a proton injection layer 161d, and a reflectivity variable layer 161e. It is integrally formed on the concave curved surface. The transparent electrode layer 161a is a thin film of ITO (Indium-Tin Oxide), which is a transparent conductive material, and a voltage for moving hydrogen ions between the ion storage layer 161b and the reflectivity variable layer 161e is applied. . The transparent electrode layer 161a is formed on the concave curved surface of the first transparent member 162 by a dip coating method or the like. An ion storage layer 161b, which is a tungsten oxide thin film, is formed on the transparent electrode layer 161a. The ion storage layer 161b is formed by a reactive sputtering method using metallic tungsten as a target in a mixed atmosphere of oxygen and argon. A solid electrolyte layer 161c, which is a tantalum oxide thin film, is formed on the ion storage layer 161b. The solid electrolyte layer 161c is formed by RF sputtering using tantalum oxide as a target in a mixed atmosphere of oxygen and argon. A proton injection layer that is a platinum thin film is formed on the solid electrolyte layer 161c. Also, a reflectivity variable layer 161e, which is a magnesium-nickel alloy thin film, is formed on the proton injection layer 161d. The proton injection layer 161d and the reflectivity variable layer 161e are formed by a magnetron sputtering method in an argon atmosphere. After deposition of the reflectivity variable layer 161e, the variable optical element 161 is exposed to a hydrogen atmosphere, and protons are taken into the variable optical element 161. Thereafter, the concave curved surface of the first transparent member 162 on which the variable optical element 161 is formed and the convex curved surface of the second transparent member 163 are bonded together by an adhesive material (adhesive layer 161f) such as silicone to form the visual field synthesis mechanism 16. Is done.

The magnesium / nickel alloy thin film constituting the reflectivity variable layer 161e has a property of being in a transparent state that transmits visible light when combined with hydrogen and in a reflective state exhibiting a metallic luster when hydrogen is released. When the transparent electrode layer 161a is grounded and a positive voltage is applied to the reflectivity variable layer 161e, protons (H ⁺ ) in the reflectivity variable layer 161e pass through the reflectivity variable layer 161e and the solid electrolyte layer 161c to store ions. Move to layer 161b and accumulate. As a result, the reflectivity variable layer 161e is dehydrogenated and becomes a reflection state. When a negative voltage is applied to the reflectivity variable layer 161e, protons accumulated in the ion storage layer 161b pass through the solid electrolyte layer 161c and the reflectivity variable layer 161e and move to the reflectivity variable layer 161e. Thereby, the reflectivity variable layer 161e is hydrogenated and becomes transparent. Thus, by applying a voltage between the transparent electrode layer 161a and the reflectivity variable layer 161e, the transparent state and the reflective state are reversibly switched according to the polarity of the voltage.

  It should be noted that conductive layers 162a and 162b of ITO film are formed at two places on the end face (left end face in FIG. 3) of the edge of the first transparent member 162 of the present embodiment. The conductive layers 162a and 162b are electrically connected to the transparent electrode layer 161a and the reflectivity variable layer 161e, respectively. The conductive layers 162a and 162b extend to the outer peripheral surface of the first transparent member 162, and are connected to the visual field synthesis control circuit 116 through wires 164a and 164b, respectively. The field-of-view synthesizing optical system 160 is disposed so as to close the distal end side opening of the cylindrical side window 165 and sealed with a sealing material. Further, the distal end portion 14 is inserted into the proximal end side opening of the side window 165, and is similarly sealed with a sealing material. Thereby, the space S enclosed by the front-end | tip part 14, the visual field synthetic | combination optical system 160, and the side surface window 165 is sealed.

  Next, operations of the visual field synthesis mechanism 16 and the visual field synthesis control circuit 116 will be described. As described above, the visual field synthesis mechanism 16 is connected to the visual field synthesis control circuit 116 via the wires 164a and 164b, and operates according to the drive signal supplied from the visual field synthesis control circuit 116 based on the control of the system control circuit 21. To do. The visual field synthesizing mechanism 16 performs simultaneous observation mode for simultaneously observing the front visual field and the lateral / rear visual field, forward observation mode for observing only the front visual field with high image quality, and observing only the lateral / rear visual field with high image quality. It operates in three types of observation modes: side and rear observation modes. In the forward observation mode, the visual field synthesis control circuit 116 applies a negative voltage to the reflectivity variable layer 161e of the variable optical element so that the visual field synthesis optical system 160 is in a transparent state so that the front visual field is always observed. In the side / rear observation mode, the field synthesis control circuit 116 applies a positive voltage to the reflectivity variable layer 161e so that the field synthesis optical system 160 is in a reflective state so that the side / rear field is always observed. To do. In the simultaneous observation mode, the visual field synthesis control circuit 116 applies a positive voltage and a negative voltage alternately and periodically to the reflectivity variable layer 161e, and modulates the reflectivity of the variable optical element 161. Imaging by the image sensor 146 is performed.

  FIG. 6 is a timing chart for explaining the operation in the simultaneous observation mode according to the embodiment of the present invention. FIG. 6B is a graph showing the change over time of the drive voltage applied by the visual field synthesis control circuit 116 to the reflectivity variable layer 161e of the variable optical element. FIG. 6A is a graph showing the change over time in the reflectance of the variable optical element 161 when the drive voltage shown in FIG. 6B is applied to the reflectance variable layer 161e. FIG. 6C is a chart showing the read transfer timing of the CCD image sensor 146. In the CCD image sensor 146, reading and transfer are performed every 1/30 seconds to generate 30 frames per second. As shown in FIG. 6, the visual field synthesis control circuit 116 starts applying a positive voltage (+ 6V) to the reflectivity variable layer 161e in the transparent state (reflectance α = 0%) at time T = 0 seconds. As the positive voltage application time elapses, dehydrogenation of the reflectivity variable layer 161e proceeds gradually, and the reflectivity increases continuously. At time T1 (T1 = 2/3 seconds) when the reflectivity of the reflectivity variable layer 161e is substantially saturated (α≈98%), the visual field synthesis control circuit 116 sets the voltage applied to the reflectivity variable layer 161e to a negative voltage (− 6V). As the negative voltage application time elapses, the hydrogenation of the reflectivity variable layer 161e gradually proceeds, and the reflectivity continuously decreases. At time T2 (T2 = 1 second) when the reflectivity variable layer 161e reaches a transparent state where the reflectivity is substantially 0% (T2 = 1 second), the visual field synthesis control circuit 116 again applies the voltage applied to the reflectivity variable layer 161e to a positive voltage (+ 6V). Switch to. That is, the visual field synthesis control circuit 116 alternately repeats the application of a positive voltage for 2/3 seconds and the application of a negative voltage for 1/3 seconds to the reflectivity variable layer 161e. Thereby, the reflectance variable layer 161e changes continuously and periodically between the transparent state and the reflective state.

Further, as shown in FIG. 6A, the reflectivity of the reflectivity variable layer 161e changes with the same waveform in each period. That is, since the reflectance in the same phase of each cycle is a constant value, the reflectance of the reflectance variable layer 161e can be known from the phase of the applied voltage. Further, since the transmittance of the reflectivity variable layer 161e is uniquely determined by the reflectivity, the transmittance of the reflectivity variable layer 161e can also be known from the phase of the applied voltage. The storage unit 24 of the processor 20 stores waveform data of reflectance and transmittance for one cycle in the simultaneous observation mode in advance. The waveform data stored in the storage unit 24 is obtained by associating the frame time t with the reflectance or transmittance. Here, the frame time t is an integer value normalized by dividing the time T (unit: second) by the frame period (read transfer period) T _F (unit: second) of the CCD drive signal.

  Next, the process which the endoscope apparatus 1 which concerns on embodiment of this invention performs is demonstrated. FIG. 7 is a flowchart for explaining processing performed by the endoscope apparatus 1. When the endoscope apparatus 1 is activated, first, the system control circuit 21 recognizes the electronic scope 10 connected to the processor 20 (S1), and determines whether or not the visual field synthesis mechanism 16 is mounted on the electronic scope 10 ( S2). When the visual field synthesizing mechanism 16 is not mounted on the electronic scope 10 (S2: NO, S9: NO), image data is acquired from the CCD image sensor 146 (S8), and then the process proceeds to the color adjustment process (S11). If the visual field synthesis mechanism 16 is mounted on the electronic scope 10 (S2: YES), the system control circuit 21 notifies the visual field synthesis control circuit 116 of the observation mode to be executed next (S3). The observation mode can be selected by user operation. Simultaneous observation mode for simultaneously observing the front visual field and the lateral / rear visual field, front observation mode for observing only the front visual field with high image quality, and lateral / It is selected from three types of observation modes, that is, a lateral / rear observation mode in which only the rear visual field is observed. If the observation mode is not designated by the user or the simultaneous observation mode is selected (S4: YES), an image separation process (S5) described later is subsequently performed. When the forward observation mode is selected (S4: NO, S9: NO), after acquiring the image data (S8), the process proceeds to the color adjustment process (S11). When the side / rear observation mode is selected (S4: NO, S9: YES), after acquiring the image data (S8), the process proceeds to the reflection image conversion process (S10). The reflection image conversion process will be described later.

FIG. 8 is a flowchart for explaining the details of the image separation process (S5). In the image separation process, first, the video signal processing circuit 23 obtains a frame synchronization signal from the CCD drive signal supplied from the CCD drive circuit 112 (S52), and obtains the time T (unit: second) at this time (S53). ). Then, time T is converted to frame time t. Next, with reference to the waveform data of the reflectance and transmittance of the variable optical element 161 in the simultaneous observation mode stored in advance in the storage unit 24, the reflectance α _t and the transmission of the variable optical element 161 at the frame time t are transmitted. The rate β _t is acquired (S53). Next, the video signal processing circuit 23 acquires image data img (x, y, t) captured by the CCD image sensor 146 at the frame time t from the digital video signal received from the signal processing circuit 114, and at the frame time t. The reflectance α _t , the transmittance β _t , and the image data img (x, y, t) are stored in the storage unit 24. Next, the reflectance α _t−1 , the transmittance β _t−1 , and the image data img (x, y, t−1) at the previous frame time t−1 are read from the storage unit 24.

同様に、フレーム時刻ｔ−１において取得される画素値ｉｍｇ（ｘ，ｙ，ｔ−１）は次の式（２）により記述される。

上記の式１及び式２からなる連立方程式を解くことにより、ｉｍｇ_ｐ（ｘ，ｙ，ｔ）、及びｉｍｇ_ｒ（ｘ，ｙ，ｔ）の値が得られる（Ｓ５７）。なお、ここで次の式（３）及び式（４）の近似が用いられる。

すなわち、同時観察モードにおいて異なるフレーム時刻に取得された２つの画像データから、前方視野像と後方視野像のデータを分離することができる。なお、必ずしも連続するフレームの画像データから連立方程式を立てる必要はなく、例えばフレーム時刻ｔ−１における画素値の代わりにフレーム時刻ｔ−ｉ（ｉ＝２，３，４・・・）における画素値を使用して計算してもよい。そして、このようにして計算された前方視野の画像データｉｍｇ_ｐ（ｘ，ｙ，ｔ）及び後方視野の画像データｉｍｇ_ｒ（ｘ，ｙ，ｔ）は、記憶部２４に記憶される（Ｓ５８）。 Here, the image acquired in the simultaneous observation mode is an image obtained by superimposing the image of the front visual field and the image of the rear visual field. At frame time t, the pixel value in the pixel (x, y) of the front visual field image (transmission image) is img _p (x, y, t), and the pixel value of the rear visual field image (reflection image) is img _r (x, y). , T), where the reflectance of the variable optical element 161 is α _t and the transmittance is β _t , the pixel value img (x, y, t) in the pixel (x, y) of the image acquired in the simultaneous observation mode is It is described by the following equation (1).

Similarly, the pixel value img (x, y, t−1) acquired at the frame time t−1 is described by the following equation (2).

The values of img _p (x, y, t) and img _r (x, y, t) are obtained by solving the simultaneous equations consisting of the above formulas 1 and 2 (S57). Here, approximations of the following equations (3) and (4) are used.

That is, the data of the front visual field image and the rear visual field image can be separated from the two image data acquired at different frame times in the simultaneous observation mode. It is not always necessary to establish simultaneous equations from image data of successive frames. For example, instead of pixel values at frame time t−1, pixel values at frame time ti (i = 2, 3, 4...) May be used to calculate. The front view image data img _p (x, y, t) and the rear view image data img _r (x, y, t) calculated in this way are stored in the storage unit 24 (S58). .

  In subsequent processes S6 to S7, a process of seamlessly joining the front view image and the side / rear view image to generate one composite image is performed. This series of processing is performed by the image composition unit 23b of the video signal processing circuit 23. Here, a front visual field image and a side / rear visual field image acquired using the visual field synthesis mechanism 16 will be described. FIG. 9 is a longitudinal sectional view of the lumen into which the distal end portion of the insertion portion 14 of the electronic scope 10 is inserted. FIG. 10 is an assumption diagram when the arrow direction (forward direction) is viewed from the viewpoint V in FIG. 9. It should be noted that the image shown in FIG. 10 is not a forward visual field image observed from the visual field synthesis mechanism 16 but an image having a wider angle than the forward visual field image. For the convenience of explanation, eight radial auxiliary lines AL and four triangular marks M are marked in FIG. The field synthesis mechanism 16 provided at the tip of the insertion unit 14 emits a front beam FL for forming a front field image and a side / back beam BL for forming a side / back field image. . A region D in FIGS. 9 and 10 is a region where the front light beam FL having a sufficient amount of light does not reach and an image is not formed. In FIG. 10, a region F located on the outer periphery of the region D is a region of the inner wall of the lumen that is illuminated by the front light flux FL, that is, a front visual field. In FIG. 10, a region B positioned on the outer periphery of the region F is a region of the inner wall of the lumen illuminated by the side / rear light beam BL, that is, a side / rear visual field. In FIG. 10, a region H located further on the outer periphery of the region B is a blind spot of the visual field synthesis mechanism 16.

A front view image obtained by the view synthesis mechanism 16 at this time is shown in FIG. 11, and a side / rear view image is shown in FIG. That is, the images shown in FIGS. 11 and 12 are respectively represented by image data img _p (x, y, t) and img _r (x, y, t) obtained by the image separation process S5. 11 is an image (that is, an image observed from the same direction as that in FIG. 10) formed by the light flux that passes through the visual field synthesizing mechanism 16, and therefore a part (regions D to F) and the visual field in FIG. Only the corners are different images. On the other hand, the side / rear view image in FIG. 12 is an image formed by the light beam reflected from the variable optical element 161 (that is, an image observed from the opposite direction to FIG. 10). In the image of the rear visual field B, the inner side and the outer side are reversed. Further, since the reflecting surface of the variable optical element 161 is a convex curved surface, the scale of the radiation direction (the direction away from the optical axis) is also different from the image of FIG. 12 corresponds to the areas D to F in FIG. 10, but is not irradiated with the side / rear light beam BL, and thus is an area where an image is not formed. Yes. In FIG. 12, the blind spot H is located inside the lateral / rear visual field B. As described above, the blind spot H is also an area where no image is formed.

  The purpose of the processes S6 to S7 is to generate a wide-angle image corresponding to the image of FIG. 10 using the front view image (FIG. 11) and the side / rear view image (FIG. 12) obtained by the image separation process S5. It is to be. For this purpose, first, in the reflection image conversion processing S6, the side / rear view image shown in FIG. 12 is converted into an image observed from the direction indicated by the arrow V by known coordinate conversion. The converted image is shown in FIG. In the present embodiment, the same image conversion as that in step S6 is performed in the reflection image conversion processing S10.

  In the next image composition processing S7, the side / rear view image of FIG. 13 converted in S8 and the front view image of FIG. 11 are joined to generate one wide-angle image. Specifically, the front view image of FIG. 11 is embedded in the region N in the side / rear view image of FIG. FIG. 15 is a flowchart for explaining the details of the image composition processing S7. In the image composition processing S7, it is first determined whether or not there is a gap region between the front visual field image and the side / rear visual field image (S71). The gap region here refers to a region that is not included in any field of view in the boundary region between the front field of view and the side / back field of view. In other words, the blind spot surrounded by the front view and the side / rear view is the gap region. The presence / absence of the gap region is known information determined by the optical design of the visual field synthesis optical system 160, and is associated with the type information of the electronic scope 10 recognized in S1, and is stored in the database (not stored). Recorded).

  When there is a gap region (S71: YES), if the front view image and the side / rear view image are directly connected, a discontinuous and unnatural composite image is obtained. Therefore, as shown in FIG. 15, a composite image in which a boundary line G is inserted between the front visual field image and the side / rear visual field image is generated (S72).

  When there is no gap region (S71: NO), the front field image and the side / rear field image are joined together using the mosaicing technique so that the corresponding feature points overlap each other. Specifically, first, feature points are extracted in a region where the front view and the side / rear view overlap (S73). A general-purpose technique can be used for extracting feature points, but feature points unique to an endoscopic image such as a branch point of a blood vessel can be used. In this case, for example, an image is converted into a color tone that makes it easy to identify a blood vessel by emphasizing red, for example, and then a blood vessel outline is extracted to identify a branch point of the blood vessel. Next, the feature points of the front visual field image and the side / rear visual field image are associated (S74), and one or both of the images are deformed so that the distribution of the feature points coincides (S75). The image and the side / rear view image are joined (S76). In this way, the composite image shown in FIG. 16 is generated. Since the generated synthesized image (FIG. 16) has a different viewpoint from the image observed in the forward direction from the viewpoint V in FIG. 9 (FIG. 10), from the normal endoscopic observation image (forward observation image), It contains a lot of information that cannot be obtained.

  In the present embodiment, the feature point extraction process S73 is performed even when the presence or absence of the gap region is unknown. When no feature point is extracted in the feature point extraction process S73, image composition (S72) for inserting a boundary line is performed.

  The synthesized image data generated by the image synthesis process S7 is then adjusted in color tone according to conditions set in advance by the user (S12), then converted into a video signal conforming to the NTSC standard and output to the monitor 30. (S12).

  The above is description of an example of embodiment of this invention. The present invention is not limited to the above embodiment, and various modifications are possible within the scope of the technical idea of the present invention. For example, although the above-described embodiment has a configuration in which the front visual field image and the side / rear visual field image are combined into one image, the present invention is not limited to this configuration. For example, only one image of the front visual field image and the side / rear visual field image may be displayed on the monitor, and the image displayed on the monitor may be switched to the other by a user operation. Further, in the above embodiment, the side / rear view image is configured to be converted into an image similar to the front observation image, but a projection view viewed from the focal point of the variable optical element 161 (hyperboloid mirror). Or a panoramic image. Since the variable optical element 161 has a mirror surface with a hyperboloid, the reflected image (side / rear view image) can be easily converted into a projection view or a panoramic image. In this case, the front visual field image and the side / rear visual field image are not synthesized, for example, a configuration in which they are displayed side by side on one screen, or only one of the images is displayed and the image to be displayed is switched by a user operation. You may make it the structure which displays a structure or each image on a separate monitor. In the above embodiment, the visual field synthesizing mechanism 16 is built in the electronic scope 10, but the visual field synthesizing mechanism may be an attachment structure that is detachable from the electronic scope. In the above embodiment, the visual field synthesis control circuit 116 is provided in the connection part 11 of the electronic scope 10, and the video signal processing circuit 23 is provided in the processor 20. However, both of these circuits are provided in either the electronic scope or the processor. May be provided. For example, by providing the visual field synthesis control circuit 116 and the video signal processing circuit 23 in the electronic scope, it is possible to simultaneously observe the front visual field and the side / rear visual field using an existing processor only by changing the program. In addition, by providing the visual field synthesis control circuit 116 and the video signal processing circuit 23 in the processor, the configuration of the video scope is simplified, and a video scope compatible with simultaneous observation can be manufactured at low cost.

DESCRIPTION OF SYMBOLS 1 Endoscope apparatus 10 Electronic scope 11 Connection part 112 CCD drive circuit 114 Signal processing circuit 116 Field composition control circuit 12 Universal cable part 13 Operation part 14 Insertion part 144 Objective lens 146 CCD image sensor 15 Light guide 16 Field composition mechanism 160 Field-of-view synthesis optical system 161 Variable optical element 161a Transparent electrode layer 161b Ion storage layer 161c Solid electrolyte layer 161d Proton injection layer 161e Reflectivity variable layer 162 First transparent member 163 Second transparent member 165 Side window S Space 20 Processor 21 System control circuit 22 Light source unit 23 Video signal processing circuit 30 External monitor

Claims (7)

Visual field synthesis means for synthesizing the first light flux connecting the image of the first visual field on the imaging surface of the imaging means and the second light flux connecting the image of the second visual field while changing the intensity ratio;
A field of view that generates image data of the first field of view and image data of the second field of view based on at least two image data of images formed on the imaging surface by the combined light beams having different intensity ratios. An endoscope apparatus having a separating means and capable of simultaneously observing a plurality of fields of view. The visual field synthesizing means periodically changes the intensity ratio in a period longer than the frame period of the imaging means,
The said visual field separation means produces | generates the image data of the said 1st and 2nd visual field based on the at least 2 image data imaged in the phase from which the said intensity ratio differs. Endoscopic device. The visual field synthesizing means has an optical element whose reflectance and transmittance are variable,
The first light flux passes through the optical element;
The second light flux is reflected by the optical element;
The endoscope apparatus according to claim 1, wherein the visual field synthesis unit changes the intensity ratio by changing a reflectance and a transmittance of the optical element. The visual field separating means includes
Image data img (x, y, t) captured at frame time t,
Image data img (x, y, t−1) captured at frame time t−1,
Reflectance of the optical element in the frame time t alpha _t, and on the basis of the reflectance alpha _t-1 of the optical elements in the frame time t-1, the simultaneous equations consisting of the following formulas (1) and (2)

As the solution of the first visual field image data img _p (x, y, t) at the frame time t and the second visual field image data img _r (x, y, t) at the frame time t.
The endoscope apparatus according to claim 3, wherein:   The endoscope apparatus according to any one of claims 1 to 4, wherein the optical element has a reflecting surface formed in a hyperboloid shape.   The endoscopic device according to any one of claims 3 to 5, wherein the optical element includes a metal thin film having a property that the reflectance is reduced by hydrogenation and the reflectance is increased by dehydrogenation. .   The endoscope apparatus according to any one of claims 1 to 6, wherein the first visual field includes a front visual field, and the second visual field includes a rear visual field.

Images