JP2017225755A - Endoscope apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an endoscope apparatus which uses an imaging unit with two imaging elements laminatingly mounted thereon to enable simultaneous observation of reflection from a subject caused by white light and reflection from the subject caused by infrared light with a desired performance attained.SOLUTION: Used is an imaging unit 21 on which are mounted laminatingly, from the side receiving light from a subject, a color filter 93 allowing infrared light to transmit therethrough, a first imaging element 83 photoelectrically converting white light, and a second imaging element 87 photoelectrically converting infrared light which has transmitted through the color filter 93 and the first imaging element 83, such that the reflection of white light from the subject and the reflection of infrared light from the subject are observed simultaneously. A timing control unit 69 causes a first infrared light source K2 and a second infrared light source K3 to illuminate at least during an illuminating period of a white light source K1 to allow that an image signal from the second imaging element 87 is read simultaneously with the reading of an imaging signal from the first imaging element 83 during at least a part of a period during which the image signal from the first imaging element 83 is read.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an endoscope apparatus.

  2. Description of the Related Art Endoscopic devices that observe tissue in a body cavity are widely known. In general, an endoscope apparatus irradiates an observation part in a body cavity with white light, and an imaging element receives reflected light from the observation part to generate an observation image.

  The electronic endoscope apparatus described in Patent Document 1 irradiates the observation site with excitation light by a blue filter in addition to the normal observation mode in which the observation site is irradiated with visible light, that is, white light, and a full-color normal observation image is displayed. And a special observation mode for displaying a narrowband image (NBI image) and an autofluorescence observation image. According to the special observation mode, it is possible to observe things that do not appear in the normal observation image such as blood vessel running.

JP 2007-50106 A Japanese Patent Laying-Open No. 2015-192015

  In the electronic endoscope apparatus described in Patent Document 1, two CCDs (solid-state imaging devices) are provided for observation in the normal observation mode and the special observation mode, and the two CCDs are arranged in parallel. Yes. However, when a plurality of CCDs are arranged in parallel, there is a problem that the diameter of the distal end portion of the endoscope inserted into the body cavity is increased. The solid-state imaging device described in Patent Document 2 has a configuration in which semiconductor substrates on which photoelectric conversion units are formed are stacked in a plurality of stages. Therefore, the above-described problem of the electronic endoscope device of Patent Document 1 can be solved. There is sex. However, when using the solid-state imaging device of Patent Document 2 for the imaging unit of the endoscope, it is necessary to devise for satisfying a desired level of performance such as sensitivity or resolution.

  The present invention has been made in view of the above-described circumstances, and reflected light from a subject by white light and reflected light from the subject by infrared light using an imaging unit in which two imaging elements are stacked and arranged. It is an object of the present invention to provide an endoscope apparatus that can perform simultaneous observation of the above while satisfying desired performance.

An endoscope apparatus according to an aspect of the present invention includes:
An illumination unit that emits illumination light from the distal end portion of the endoscope, an imaging unit that images an observation region of the subject irradiated with the illumination light, a control unit that controls the illumination unit and the imaging unit, An endoscopic device comprising:
The illumination unit includes a white light source for generating white light, and an infrared light source for generating infrared light,
The imaging unit includes a plurality of pixels having photoelectric conversion units arranged in a two-dimensional matrix, a first imaging element that generates a visible light image, and a photoelectric conversion unit that photoelectrically converts light transmitted through the first imaging element. A plurality of pixels having a two-dimensional matrix, and a first light transmission film that transmits light having a wavelength component of the infrared light. The first light transmission film, the first image sensor, and the second image sensor are stacked in this order from the incident side,
The control unit causes the infrared light source to emit light only during a light emission period of the white light source, reads an image signal from the second image sensor, and reads an image signal from the first image sensor. The reading is performed simultaneously with the reading of the image signal from the first image sensor in all reading periods of the period.

  According to the present invention, the two image pickup devices are stacked by controlling the light emission periods of the white light source and the infrared light source and the readout timing of the image signals from the two image pickup devices of the image pickup unit. It is possible to provide an endoscope apparatus that can perform simultaneous observation of reflected light from a subject using white light and reflected light from the subject using infrared light while satisfying desired performance.

1 is an external view of an endoscope apparatus for explaining an embodiment of the present invention. It is a figure which shows the internal structure of the endoscope apparatus shown by FIG. It is a figure which shows the relationship between the wavelength of the light radiate | emitted from the light source for white light K1, the light source for 1st infrared light K2, and the light source for 2nd infrared light, and relative intensity. It is sectional drawing which shows the structure of a solid-state imaging device. It is a figure which shows the spectral transmission characteristic of the color filter of 1st Embodiment. (A) is a figure which shows an example of the Bayer arrangement in a color filter, (B) is a figure which shows the other example of the Bayer arrangement in a color filter. It is a figure which shows an example of the light emission timing of each light source at the time of setting of 2 types simultaneous observation mode in 1st Embodiment, and the read-out timing of the image signal from each image pick-up element. It is a figure which shows the spectral transmission characteristic of the color filter of 2nd Embodiment. It is a figure which shows the pixel integrated by the binning process of the image signal obtained from a 2nd image sensor. It is a figure which shows an example of the light emission timing of each light source at the time of setting of 2 types simultaneous observation mode in 2nd Embodiment, and the read-out timing of the image signal from each image pick-up element. It is a figure which shows an example of the light emission timing of each light source at the time of setting of 3 types simultaneous observation mode in 2nd Embodiment, and the read-out timing of the image signal from each image pick-up element.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is an external view of an endoscope apparatus 100 for explaining an embodiment of the present invention.

  The endoscope apparatus 100 includes an endoscope 11, a control device 13, a display unit 15 such as a liquid crystal display device, and an input unit 17 such as a keyboard and a mouse for inputting information to the control device 13.

  The control device 13 includes an illumination unit 45 and a control unit 47 that performs signal processing of an image signal output from the endoscope 11. A display unit 15 and an input unit 17 are connected to the control unit 47.

  The endoscope 11 includes an endoscope insertion portion 19 that is inserted into a subject, an operation portion 23 that performs an operation for bending and observing the distal end of the endoscope insertion portion 19, and the endoscope 11. Connector portions 25 and 27 are detachably connected to the control device 13.

  The endoscope insertion portion 19 includes a flexible soft portion 29, a bending portion 31, and a distal end portion (hereinafter also referred to as an endoscope distal end portion) 33.

  Although not shown, various channels such as a forceps channel for inserting a tissue collection treatment instrument and the like, a channel for air supply / water supply, and the like are provided inside the operation unit 23 and the endoscope insertion unit 19. .

  FIG. 2 is a diagram showing an internal configuration of the endoscope apparatus 100 shown in FIG.

  The endoscope distal end portion 33 includes an illumination window 35 for irradiating light to an observation region, a diffusion plate 58 disposed opposite to the illumination window 35, and an illumination disposed between the diffusion plate 58 and the illumination window 35. Lens 59, imaging unit 21 that receives reflected light from the observation region, observation window 40 for allowing reflected light from the observation region to enter the light receiving surface of imaging unit 21, observation window 40 and imaging And an objective lens 39 provided between the unit 21 and the unit 21.

  The bending portion 31 is provided between the flexible portion 29 and the endoscope distal end portion 33, and can be bent by a turning operation of an angle knob 43 (see FIG. 1) disposed in the operation portion 23.

  The bending portion 31 can be bent in an arbitrary direction and an arbitrary angle according to a part of the subject in which the endoscope 11 is used, and the illumination window 35 and the observation window of the endoscope distal end portion 33. 40 can be directed to the desired viewing site.

  The control device 13 includes an illumination unit 45 that emits illumination light supplied to the observation region from the illumination window 35 of the endoscope distal end 33, and a control unit 47 that controls the illumination unit 45 and the imaging unit 21. The illumination unit 45 and the control unit 47 are connected to the endoscope 11 via connector units 25 and 27, respectively.

  Hereinafter, the illumination unit 45 of the control device 13 will be described in detail. As shown in FIG. 2, the illumination unit 45 includes a light source driving unit 49, a white light source K1, a first infrared light source K2, a second infrared light source K3, and an optical fiber bundle 51. And optical fibers 53 and 54. The wavelength of the infrared light emitted from the first infrared light source K2 is different from the wavelength of the infrared light emitted from the second infrared light source K3.

  The white light source K1 is a semiconductor element that emits light for normal observation (white light observation) and narrowband light observation. The white light source K1 includes an R-LED (Red-Light Emitting Diode), a G-LED (Green-Light Emitting Diode), a B-LED (Blue-Light Emitting Diode), and a V-LED (Violet-Light Emitting Diode). ) Four-color LEDs (Light Emitting Diodes).

  As shown in FIG. 3, the R-LED is a light emitting diode that emits red light having a wavelength band of 600 to 650 nm. The G-LED is a light emitting diode that emits green light having a wavelength band of 480 to 600 nm. The B-LED is a light emitting diode that emits blue light having a wavelength band of 420 to 500 nm. The V-LED is a light emitting diode that emits purple light having a wavelength band of 380 to 420 nm. By combining the light generated by each LED, white light is emitted from the white light source K1. The white light source K1 may be a xenon lamp or a white LED.

  The first infrared light source K2 is a laser diode that emits laser light having a wavelength band of 790 to 820 nm and a center wavelength of 810 nm, which belongs to the near infrared band, for infrared light observation. The second infrared light source K3 is a laser diode that emits laser light having a wavelength band of 905 to 970 nm belonging to the near infrared band and a center wavelength of 940 nm for infrared light observation. The first infrared light source K2 and the second infrared light source K3 may be a combination of a xenon lamp and a filter or an LED.

  As a light source combining the white light source K1, the first infrared light source K2, and the second infrared light source K3, each of the R component, G component, and B component filter elements, and the transmission wavelength are as follows. A surface sequential exposure type illumination unit combining a rotary filter in which filter elements of a first infrared component of 810 nm and a second infrared component having a transmission wavelength of 940 nm are arranged in the rotation direction and a xenon lamp may be used. .

  The white light source K1, the first infrared light source K2, and the second infrared light source K3 are individually driven by the light source driving unit 49 according to the observation mode of the subject. In the observation mode, “normal observation mode” in which reflected light is observed by irradiating the observation area of the subject with white light, and the observation area of the subject is irradiated with light in a specific wavelength region of visible light. "Narrowband light observation mode" for observing reflected light and "Infrared light observation mode" for observing reflected light by irradiating the observation area of the subject with infrared light, normal observation and infrared light There are “two types simultaneous observation mode” in which observation is performed simultaneously and “three types simultaneous observation mode” in which normal observation, infrared light observation, and narrowband light observation are performed simultaneously. The light source driver 49 emits light only from the white light source K1 in the normal observation mode and the narrow-band light observation mode, and for the first infrared light source K2 and the second infrared light in the infrared light observation mode. Light is emitted only from the light source K3, and light is emitted from all the light sources K1, K2, and K3 in the two-kind simultaneous observation mode and the three-kind simultaneous observation mode. In addition, when white LED is used as the light source K1 for white light, the light irradiated to the observation region when performing narrow-band light observation is light in a specific wavelength region among visible light emitted from the white LED. Light that has passed through a filter (not shown) that passes through.

  White light emitted from the white light source K1 is input to the optical fiber bundle 51 via a condenser lens (not shown). Infrared light emitted from the first infrared light source K2 is input to the optical fiber 53 via a condenser lens (not shown). Infrared light emitted from the second infrared light source K3 is input to the optical fiber 54 via a condenser lens (not shown). The optical fiber bundle 51 and the optical fibers 53 and 54 are bundled together by a light guide 55. The light guide 55 extends to the endoscope distal end portion 33 via the connector portion 25. The light emitted from the end face of the light guide 55 on the endoscope distal end 33 side is diffused by the diffusion plate 58 and then emitted from the illumination window 35 through the lens 59.

  The light source driving unit 49 drives the light sources K1, K2, and K3 independently by pulse driving. That is, the light source drive unit 49 supplies different drive signals (drive pulse patterns) to the respective light sources. The light source emits light during the period when the drive pulse is at a high level. The light source drive unit 49 drives the R-LED, G-LED, B-LED, and V-LED independently during the drive control of the white light source K1.

  Next, the imaging unit 21 provided at the endoscope distal end portion 33 will be described in detail. The imaging unit 21 images the observation region of the subject irradiated with the light from the illumination unit 45. Light from the subject passes through the objective lens 39 and enters the imaging unit 21. Note that ICG (Indocyanine Green) is administered to the subject in advance.

  As shown in FIG. 4, the imaging unit 21 includes a color filter 93 as a first light transmission film, a first imaging element 83, a light transmission film 89 as a second light transmission film, and a second imaging element 87. And a layered arrangement in this order from the light incident side from the subject.

  The color filter 93 separates the RGB components of the light incident on the imaging unit 21 and transmits infrared light having a wavelength of about 790 nm or more. The arrangement pattern of the color filter 93 is a Bayer arrangement in which the RGB color components are arranged in a zigzag pattern, but even if the general Bayer arrangement shown in FIG. A Bayer array with many B components shown in FIG. The color filter 93 is arranged so that each color component corresponds to a pixel described later of the first image sensor 83.

  FIG. 5 is a diagram showing the spectral transmission characteristics of the color filter 93. The R component of the color filter 93 transmits red light R and infrared light having a wavelength band of about 550 nm or more, as indicated by a dotted line in FIG. The G component transmits green light G having a wavelength band of 460 to 630 nm and near-infrared light having a wavelength band of 790 to 850 nm, as indicated by a one-dot chain line in FIG. Further, the B component transmits blue light B having a wavelength band of 380 to 550 nm and infrared light having a wavelength of about 900 nm or more, as indicated by a two-dot chain line in FIG. As described above, among the infrared light, the R component is 810 nm near-infrared light emitted from the first infrared light source K2, and the 940 nm near-red light emitted from the second infrared light source K3. Although external light is also transmitted, the G component transmits only 810 nm near-infrared light emitted from the first infrared light source K2, and the B component is 940 nm emitted from the second infrared light source K3. Transmits only near-infrared light.

  The first image sensor 83 is configured by arranging a plurality of pixels each having the photoelectric conversion unit 81 in a two-dimensional matrix, and generates a visible light image.

  The light transmission film 89 is a white light cut filter having a characteristic of blocking the wavelength component of white light, that is, light having a wavelength band of about 750 nm or less. By providing the light transmission film 89 between the first image sensor 83 and the second image sensor 87, the noise component of light for the second image sensor 87 can be removed.

  The second image sensor 87 includes a pixel having a photoelectric conversion unit 85 that photoelectrically converts light of a near-infrared wavelength component (790 to 1000 nm) from a subject that has passed through the first image sensor 83 and the light transmission film 89. A plurality of elements are arranged in a two-dimensional matrix. The position of the pixel of the first image sensor 83 and the position of the pixel of the second image sensor 87 viewed from the light incident side overlap each other. Accordingly, the color components of the color filter 93, the pixels of the first image sensor 83, and the pixels of the second image sensor 87 overlap each other when viewed from the light incident side.

  Among the pixels constituting the second image sensor 87, the pixel corresponding to the position of the G component of the color filter 93 when viewed from the light incident side is emitted from the first infrared light source K2 in the observation region. The reflected near-infrared light of 810 nm is photoelectrically converted. Further, the pixel corresponding to the position of the B component of the color filter 93 when viewed from the light incident side photoelectrically converts 940 nm near-infrared light emitted from the second infrared light source K3 and reflected by the observation region. . Further, the pixel corresponding to the position of the R component of the color filter 93 as viewed from the light incident side photoelectrically converts near-infrared light. The image signal obtained by this photoelectric conversion is converted into a 810 nm component and a 940 nm component. Since it cannot be decomposed, it is not used for images for infrared light observation.

  The light receiving area of the photoelectric conversion unit 85 in each pixel of the second image sensor 87 is larger than the light receiving area of the photoelectric conversion unit 81 in each pixel of the first image sensor 83. If the light receiving area of the photoelectric conversion unit 85 is large, the second imaging element 87 can achieve a desired sensitivity even if the reflected light in the near infrared band from the subject is weak.

  Both the first image sensor 83 and the second image sensor 87 are CMOS (Complementary Metal Oxide Semiconductor) image sensors, and are driven by a rolling shutter system. For this reason, the first image sensor 83 and the second image sensor 87 have a plurality of columns of pixel rows (n rows of Line 1 to Line n) as drive units, and are exposed for each pixel row under the control of the control unit 47. It is driven by shifting the time and the readout period of the image signal (charge). That is, reading of the image signal from the image sensor is sequentially performed while shifting for each pixel row, and exposure of the image sensor is performed for each pixel row by the electronic shutter, and exposure is performed except when the image signal is read.

  Next, the control unit 47 of the control device 13 will be described in detail. As shown in FIG. 2, the control unit 47 includes a storage unit 71, a timing control unit 69, and a signal processing unit 66.

  The storage unit 71 includes a duty ratio of each drive pulse constituting a drive signal supplied to each LED of the white light source K1 included in the illumination unit 45, a timing at which the on / off level of each drive pulse is switched, and the imaging unit 21. A first table in which the timing for reading an image signal (charge) from the first image sensor 83 is stored is stored. In addition, the storage unit 71 includes a duty ratio of a driving pulse that constitutes a driving signal supplied to the first infrared light source K2, a timing at which the on / off level of the driving pulse is switched, and a second imaging element of the imaging unit 21. The second table in which the timing for reading the image signal (charge) from 87 is defined. In addition, the storage unit 71 includes a duty ratio of a driving pulse constituting a driving signal supplied to the second infrared light source K3, a timing at which the on / off level of the driving pulse is switched, and a second imaging element of the imaging unit 21. A third table in which the timing for reading the image signal (charge) from 87 is defined is stored.

  The timing control unit 69 determines a light source to be driven according to an observation mode set according to an instruction from the operation unit 23 or the input unit 17 of the endoscope 11. The timing controller 69 drives the light source based on at least one of the information specified in the first table, the information specified in the second table, and the information specified in the third table stored in the storage unit 71. The timing of driving the light sources K1, K2, and K3 performed by the unit 49 and the timing of reading the image signal by the signal processing unit 66 from the first image sensor 83 and the second image sensor 87 are controlled.

  The signal processing unit 66 performs signal processing on the image signal from the endoscope 11 read according to the control from the timing control unit 69 to generate image data. An image based on the image data generated by the signal processing unit 66 is displayed on the display unit 15. Note that the normal observation image, the narrow-band light observation image, and the infrared light observation image generated by the signal processing unit 66 may be displayed side by side or displayed in a superimposed manner.

  Hereinafter, the operation of the endoscope apparatus 100 according to the first embodiment will be described in detail with reference to FIG. When the operator presses an observation mode switching button 73 (see FIG. 2) provided in the endoscope 11, the timing control unit 69 causes the normal observation mode, the narrowband light observation mode, the infrared light observation mode, It switches to various observation modes of two kinds simultaneous observation mode which performs observation and infrared light observation simultaneously, and three kinds simultaneous observation mode which performs normal observation, infrared light observation and narrowband light observation simultaneously.

  FIG. 7 shows an example of the operation of the endoscope apparatus 100 when the two-type simultaneous observation mode is set. In the example shown in FIG. 7, the timing control unit 69 causes all LEDs of the white light source K1 to emit light at a predetermined duty ratio, and the first infrared light source K2 and the second infrared light source K3 are white. The light source driving unit 49 is controlled so as to emit light only during the light emission period of the light source K1. In addition, the timing control unit 69 reads out each image signal from the first imaging element 83 and the second imaging element 87 included in the imaging unit 21 during the non-emission period of the light sources K1, K2, and K3. To control. That is, the reading of the image signal from the second image sensor 87 is performed simultaneously with the reading of the image signal from the first image sensor 83 in all the reading periods of the image signal reading period from the first image sensor 83. Note that the readout of each image signal from the first image sensor 83 and the second image sensor 87 is shifted for each pixel row in each element because the first image sensor 83 and the second image sensor 87 are both CMOS. It is done sequentially.

  According to this operation example, the image signal of the reflected light from the observed region irradiated with white light (VGRB) and the reflected light from the observed region irradiated with near-infrared light (IR1) with a center wavelength of 810 nm are irradiated. And the image signal of the reflected light from the observation region irradiated with the near-infrared light (IR2) having the center wavelength of 940 nm are read out simultaneously, so that the image signal by two near-infrared lights having different wavelengths can be obtained. The synthesized infrared light observation and normal observation can be performed simultaneously at the same frame rate. Moreover, since the image signal for normal observation and the image signal for infrared light observation are obtained from one solid-state imaging device 61, the diameter of the endoscope insertion portion 19 inserted into the body cavity is increased. In addition, the endoscope apparatus 100 capable of the two-type simultaneous observation mode at the same frame rate can be realized.

  As described above, according to the present embodiment, each light emission of the white light source K1, the first infrared light source K2, and the second infrared light source K3 included in the illumination unit 45, and insertion of an endoscope. The readout operations of the first imaging element 83 and the second imaging element 87 included in one solid-state imaging device 61 capable of realizing a reduction in the diameter of the unit 19 are controlled so as to cooperate at the same timing. For this reason, normal observation by reflected light from the subject with white light and infrared light observation by reflected light from the subject with near infrared light can be simultaneously performed at the same frame rate.

(Second Embodiment)
A second embodiment will be described with reference to FIGS. The second embodiment differs from the first embodiment in the spectral transmission characteristics of the color filter 93 and the light emission periods of the first infrared light source K2 and the second infrared light source K3. . Except for this point, the second embodiment is the same as the first embodiment, and therefore, the description of the same or equivalent matters as the first embodiment is simplified or omitted.

  The spectral transmission characteristics of the color filter 93 of the second embodiment are shown in FIG. The R component of the color filter 93 of the second embodiment transmits red light R and infrared light having a wavelength band of about 550 nm or more, as indicated by a dotted line in FIG. Further, the G component transmits green light G having a wavelength band of 460 to 630 nm and infrared light having a wavelength of about 790 nm or more, as indicated by a one-dot chain line in FIG. Further, as shown by a two-dot chain line in FIG. 8, the B component transmits blue light B having a wavelength band of 380 to 550 nm and infrared light having a wavelength of about 790 nm or more. As described above, any color component of the color filter 93 sufficiently transmits infrared light of 790 nm or more. However, among the three components, only the G component or B component having a high composition ratio in the Bayer array may transmit infrared light having a wavelength of about 790 nm or more.

  In addition, the signal processing unit 66 according to the second embodiment bins and reads out the image signal from the second image sensor 87. That is, as illustrated in FIG. 9, the signal processing unit 66 regards a plurality of adjacent (for example, 2 × 2 = 4) pixels of the second image sensor 87 as one pixel, and images of the plurality of adjacent pixels. The signal is integrated into one image signal. This apparently improves the light receiving sensitivity of the second image sensor 87 that receives infrared light having a wavelength of about 790 nm or more. Further, by causing the pixels of the second image sensor 87 read out by binning processing to correspond to the repeating unit of the Bayer array of the color filter 93, the influence due to the difference in the transmittance of infrared light due to the RGB components of the color filter 93 is eliminated. it can. Note that the number of pixels integrated by the binning process may be a multiple of 2 × 2. Instead of performing the binning process, the size of the pixel of the second image sensor 87 may be set to a size corresponding to the repeating unit of the Bayer array of the color filter 93.

  Hereinafter, the operation of the endoscope apparatus 100 according to the second embodiment will be described in detail with reference to FIG.

  FIG. 10 shows an example of the operation of the endoscope apparatus 100 when the two-type simultaneous observation mode is set. In the example illustrated in FIG. 10, the timing control unit 69 controls the light source driving unit 49 so that all the LEDs of the white light source K1 emit light with a predetermined duty ratio, and from the first imaging element 83 included in the imaging unit 21. The signal processing unit 66 is controlled so that the image signal is read during the non-emission period of the white light source K1. In addition, the timing control unit 69 causes the light source driving unit 49 to alternately perform the light emission of the first infrared light source K2 and the light emission of the second infrared light source K3 for each light emission period of the white light source K1. The signal processing unit 66 is controlled to read the image signal from the second image sensor 87 of the imaging unit 21 simultaneously with the readout of the image signal from the first image sensor 83. Note that the readout of each image signal from the first image sensor 83 and the second image sensor 87 is shifted for each pixel row in each element because the first image sensor 83 and the second image sensor 87 are both CMOS. It is done sequentially.

  According to this operation example, readout of an image signal of reflected light from an observation region irradiated with near-infrared light (IR1) having a center wavelength of 810 nm and irradiation with near-infrared light (IR2) having a center wavelength of 940 nm are performed. The readout of the image signal of the reflected light from the observed region is performed at a frame rate that is half that of the readout of the image signal of the reflected light from the observed region irradiated with white light (VGRB). Infrared light observation in which image signals of two near infrared lights are combined and normal observation can be performed simultaneously. Moreover, since the image signal for normal observation and the image signal for infrared light observation are obtained from one solid-state imaging device 61, the diameter of the endoscope insertion portion 19 inserted into the body cavity is increased. In addition, the endoscope apparatus 100 capable of the two-type simultaneous observation mode can be realized.

  If the second imaging element 87 cannot obtain sufficient sensitivity because the reflected light of near-infrared light from the observation region is weak, the signal processing unit 66 performs the binning process described above. Therefore, although the resolution of infrared light observation is lowered, infrared light observation with a desired sensitivity is possible.

  FIG. 11 shows an example of the operation of the endoscope apparatus 100 when the three-type simultaneous observation mode is set. In the example shown in FIG. 11, the light emission periods of the first infrared light source K2 and the second infrared light source K3 and the readout timing of the image signal from the second image sensor 87 are the same as in the example shown in FIG. However, the timing control unit 69 emits light from all LEDs for normal observation and G- for narrowband light observation during the light emission period of the white light source K1 driven at a predetermined duty ratio. The light source driving unit 49 is controlled so that only the LED and the B-LED emit light alternately. Note that the reading of the image signal from the first image sensor 83 is performed during a non-light emission period in which all the LEDs of the white light source K1 are turned off.

  According to this operation example, readout of an image signal of reflected light from the observation region irradiated with white light and reflection of light reflected from the observation region irradiated with light of a specific wavelength band out of visible light are performed. Reading of the image signal is performed at a half frame rate as compared with the example shown in FIG. 10, but normal observation, infrared light observation, and narrowband light observation can be performed simultaneously. In addition, since an image signal for normal observation, an image signal for infrared light observation, and an image signal for narrowband light observation are obtained from one solid-state imaging device 61, an endoscope inserted into a body cavity The endoscope apparatus 100 capable of the three-type simultaneous observation mode can be realized without increasing the diameter of the insertion portion 19.

  As described above, according to the present embodiment, each light emission of the white light source K1, the first infrared light source K2, and the second infrared light source K3 included in the illumination unit 45, and insertion of an endoscope. The readout operations of the first imaging element 83 and the second imaging element 87 included in one solid-state imaging device 61 capable of realizing a reduction in the diameter of the unit 19 are controlled so as to cooperate at the same timing. For this reason, even when the color filter 93 having the spectral transmission characteristics shown in FIG. 8 is used, normal observation using reflected light from the subject with white light and reflected light from the subject with near-infrared light are used. Infrared light observation can be performed simultaneously. In addition, when the second imaging element 87 cannot obtain sufficient sensitivity because the reflected light from the subject by the near infrared light is weak, infrared with a desired sensitivity is performed by performing a binning process. Light observation is possible.

  In the above description, 810 nm is described as an example of the center wavelength of the laser light emitted from the first infrared light source K2, but is not limited to 810 nm as long as it is in the range of 790 to 820 nm. Similarly, the center wavelength of the laser light emitted from the second infrared light source K3 has been described by way of example of 940 nm, but is not limited to 940 nm as long as it is in the range of 905 to 970 nm.

  In the above description, the case where the endoscope insertion portion 19 inserted into the body cavity is a so-called flexible endoscope having flexibility has been described. However, the endoscope apparatus 100 according to the present invention includes an endoscope insertion portion. It is also possible to apply to a so-called rigid mirror camera head formed of a hard material.

As described above, the endoscope apparatus disclosed in this specification is
An illumination unit that emits illumination light from the distal end portion of the endoscope, an imaging unit that images an observation region of the subject irradiated with the illumination light, a control unit that controls the illumination unit and the imaging unit, An endoscopic device comprising:
The illumination unit includes a white light source for generating white light, and an infrared light source for generating infrared light,
The imaging unit includes a plurality of pixels having photoelectric conversion units arranged in a two-dimensional matrix, a first imaging element that generates a visible light image, and a photoelectric conversion unit that photoelectrically converts light transmitted through the first imaging element. A plurality of pixels having a two-dimensional matrix, and a first light transmission film that transmits light having a wavelength component of the infrared light. The first light transmission film, the first image sensor, and the second image sensor are stacked in this order from the incident side,
The control unit causes the infrared light source to emit light only during a light emission period of the white light source, reads an image signal from the second image sensor, and reads an image signal from the first image sensor. The reading is performed simultaneously with the reading of the image signal from the first image sensor in all reading periods of the period.

The infrared light source includes a first infrared light source and a second infrared light source having different emission wavelengths,
The first light transmission film is configured by RGB color filters arranged in a Bayer array,
The first color component constituting the color filter transmits light of a wavelength component emitted by the first infrared light source,
The second color component constituting the color filter transmits light of a wavelength component emitted from the second infrared light source.

Of the pixels constituting the second image sensor, the first pixel corresponding to the position of the first color component photoelectrically converts light of the wavelength component emitted by the first infrared light source, The second pixel corresponding to the position of the second color component photoelectrically converts light of the wavelength component emitted by the second infrared light source,
The control unit reads an image signal obtained by photoelectric conversion of at least one of the first pixel and the second pixel when reading the image signal from the second image sensor.

  The light receiving area of the photoelectric conversion unit of the second image sensor is larger than the light receiving area of the photoelectric conversion unit of the first image sensor.

  Further, the control unit reads out an image signal from the second image sensor by performing a binning process.

Further, the first light transmission film is composed of RGB color filters arranged in a Bayer array,
The pixel of the image signal of the second image sensor read out by the binning process is a pixel corresponding to the repeating unit of the Bayer array.

  The imaging unit includes a second light transmission film that does not transmit light having the wavelength component of the white light between the first imaging element and the second imaging element.

  The white light source is composed of a plurality of light emitting diodes having different emission wavelengths, and the infrared light source is composed of a laser diode.

  The first image sensor and the second image sensor are sensors driven by a rolling shutter system.

21 Imaging unit 45 Illumination unit 69 Timing control unit 81 Photoelectric conversion unit 83 First imaging element 85 Photoelectric conversion unit 87 Second imaging element 93 Color filter (first light transmission film)
100 Endoscopic device 89 Light transmission film (second light transmission film)
K1 Light source for white light K2 Light source for first infrared light K3 Light source for second infrared light

Claims (9)

An illumination unit that emits illumination light from the distal end portion of the endoscope, an imaging unit that images an observation region of the subject irradiated with the illumination light, a control unit that controls the illumination unit and the imaging unit, An endoscopic device comprising:
The illumination unit includes a white light source for generating white light, and an infrared light source for generating infrared light,
The imaging unit includes a plurality of pixels having photoelectric conversion units arranged in a two-dimensional matrix, a first imaging element that generates a visible light image, and a photoelectric conversion unit that photoelectrically converts light transmitted through the first imaging element. A second imaging element in which a plurality of pixels having a two-dimensional matrix are arranged, and a first light transmission film that transmits light of a wavelength component of the infrared light, and the light from the subject The first light transmission film, the first image sensor, and the second image sensor are stacked in that order from the incident side,
The control unit causes the infrared light source to emit light only during a light emission period of the white light source, reads an image signal from the second image sensor, and reads an image signal from the first image sensor. An endoscope apparatus that performs simultaneously with reading of an image signal from the first image sensor during all reading periods of the period. The endoscope apparatus according to claim 1,
The infrared light source includes a first infrared light source and a second infrared light source having different emission wavelengths,
The first light transmission film is configured by RGB color filters arranged in a Bayer array,
The first color component constituting the color filter transmits light of a wavelength component emitted by the first infrared light source,
An endoscope apparatus in which the second color component constituting the color filter transmits light of a wavelength component emitted from the second infrared light source. The endoscope apparatus according to claim 2,
Of the pixels constituting the second image sensor, the first pixel corresponding to the position of the first color component photoelectrically converts the light of the wavelength component emitted by the first infrared light source, and the second The second pixel corresponding to the position of the color component photoelectrically converts the light of the wavelength component emitted from the second infrared light source,
The endoscope device, wherein the control unit reads an image signal obtained by photoelectric conversion of at least one of the first pixel and the second pixel when reading the image signal from the second imaging element. The endoscope apparatus according to any one of claims 1 to 3,
An endoscope apparatus in which a light receiving area of a photoelectric conversion unit of the second image sensor is larger than a light receiving area of a photoelectric conversion unit of the first image sensor. The endoscope apparatus according to claim 1 or 4,
The said control part is an endoscope apparatus which reads the image signal from a said 2nd image sensor by binning processing. The endoscope apparatus according to claim 5,
The first light transmission film is configured by RGB color filters arranged in a Bayer array,
An endoscope apparatus, wherein a pixel of an image signal of the second image sensor read out by the binning process is a pixel corresponding to a repeating unit of the Bayer array. The endoscope apparatus according to any one of claims 1 to 6,
The endoscope apparatus, wherein the imaging unit includes a second light transmission film that does not transmit light having a wavelength component of the white light between the first imaging element and the second imaging element. The endoscope apparatus according to any one of claims 1 to 7,
The endoscope apparatus, wherein the white light source is configured by a plurality of light emitting diodes having different emission wavelengths, and the infrared light source is configured by a laser diode. The endoscope apparatus according to any one of claims 1 to 8,
The endoscope apparatus, wherein the first image sensor and the second image sensor are sensors driven by a rolling shutter system.

Images