JP2018192043A - Endoscope and endoscope system - Google Patents

Abstract

To acquire both characteristics of simultaneous-type real-time properties and a frame sequential-type high resolution/color reproducibility.SOLUTION: An endoscope includes: a solid-state image sensor including a plurality of pixels disposed into a two-dimensional matrix shape, and a color filter having color selection filters disposed corresponding to the pixels; and an insertion section on which the solid-state imaging sensor is installed, and that causes return light from a subject to enter the pixels. The color filters cause the return light to enter 50% or more first pixels out of all the pixels of the solid-state imaging sensor without restricting the wavelength band of the return light, cause the return light to enter second pixels other than the first pixels out of all the pixels by restricting the wavelength band of the return light by first spectral characteristics, and causes the return light to enter the remaining third pixels other than the first and second pixels by restricting the wavelength band of the return light by second spectral characteristics different from the first spectral characteristics.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an endoscope and an endoscope system for obtaining a color endoscope image.

  In recent years, electronic endoscopes that can observe various organs such as organs in a body cavity by inserting an elongated insertion portion into a body cavity or perform various therapeutic treatments using a treatment instrument inserted into a treatment instrument channel as necessary. Widely used.

  A color image capturing method of an electronic endoscope is obtained by irradiating visible light corresponding to R, G, and B for each readout frame of a monochrome imager, thereby obtaining each color image and synthesizing the image by image processing. There are a frame sequential type that obtains a color image and a simultaneous type that irradiates white light for each readout frame of a color imager and obtains a full color image for each readout.

  As described above, in an electronic endoscope, a color imager in which an on-chip color filter (hereinafter referred to as OCF) is mounted on a photodiode (PD) unit or a monochrome imager in which no OCF is mounted is employed.

  As an OCF of a color imager, for example, Patent Document 1 discloses a solid-state imaging device having a color filter layer including a visible light transmission filter and an infrared transmission filter and an infrared cut filter layer.

  Patent Document 2 discloses a solid-state imaging device in which a luminance filter is arranged in addition to R, G, and B3 primary color filters as an OCF of a color imager.

  Patent Document 3 discloses an example in which the OCF of the color imager is composed of complementary color filters having spectral characteristics of two kinds of colors, one of combinations of Mg and Cy, Y and Mg, and Y and Cy. ing.

  Patent Document 4 includes a filter unit in which the number of filters that transmit green light is more than half of the total number of filters, and the number of filters that transmit blue light is equal to or greater than the number of filters that transmit green light. Thus, a technique for obtaining high resolution in any of the observation methods of normal light observation (white light observation (WLI)) and narrow-band light observation (NBI) is disclosed.

  An endoscope equipped with a color imager is driven in a simultaneous manner that irradiates a subject with white light, photoelectrically converts return light, and generates a color image of the endoscope by one reading. However, the color imager has a drawback that it is difficult to obtain color reproducibility when the color image is generated by the color arrangement or arrangement of the OCF, and the resolution is deteriorated.

  On the other hand, an endoscope equipped with a monochrome imager irradiates a subject with illumination light of R (red) light, G (green) light, and B (blue) light for each frame, and photoelectrically converts return light for each frame. Thus, it is driven in a frame sequential manner that generates an RGB color image of the endoscope by multiple readings. A monochrome imager is easier to obtain color reproducibility than a color imager, and an image with the same number of effective pixels can be obtained, so the resolution does not deteriorate. However, the monochrome imager has a frame rate of each color that is 1/3 that of the color imager, and has a difficulty in operability in terms of moving image response and image blurring.

International Publication No. 2014-041742 JP 2003-318375 A JP 2008-79296 A Japanese Patent Laying-Open No. 2015-116328

  In an electronic endoscope equipped with a monochrome imager, when white light is used as illumination light, a color image cannot be obtained. On the other hand, when a frame sequential illumination is performed in an electronic endoscope equipped with a color imager, the resolution of each color image is extremely low, and an image with sufficient resolution cannot be obtained. Therefore, when operating the endoscope system by both simultaneous and frame sequential methods, an endoscope equipped with a simultaneous color imager and an endoscope equipped with a monochrome imager for frame sequential. There was a problem that it was necessary to prepare and separately.

  The present invention employs an imager that can obtain an endoscopic image with a sufficient resolution regardless of whether the endoscope is used for both simultaneous and frame sequential endoscopes. It is an object of the present invention to provide an endoscope and an endoscope system that can obtain characteristics of both high resolution and color reproducibility.

  An endoscope according to one aspect of the present invention includes a solid-state imaging device having a plurality of pixels arranged in a two-dimensional matrix, and a color filter in which a color selection filter is arranged corresponding to the pixels, and the solid-state imaging device , And an insertion section for allowing the return light from the subject to enter the pixel, and the color filter applies the return light to the first pixel of 50% or more of all the pixels of the solid-state imaging device. The wavelength band of the return light is limited to the second pixel other than the first pixel of all the pixels by the first spectral characteristics, and the wavelength band of the return light is incident. The wavelength band of the return light is made incident on the remaining third pixels other than the first and second pixels, out of all the pixels, with the second spectral characteristic different from the first spectral characteristic.

  An endoscope system according to an aspect of the present invention controls the endoscope, a light source device that generates illumination light through the insertion unit, the light source device, and the solid-state imaging element. And a processor for driving the endoscope in a simultaneous manner or a frame sequential manner.

  According to the present invention, by adopting an imager that can obtain an endoscopic image with a sufficient resolution even when it is adopted for both a simultaneous type and a plane sequential type endoscope, There is an effect that it is possible to obtain characteristics of both high resolution and color reproducibility in a sequential manner.

The block diagram which shows the structure of the endoscope system containing the endoscope which concerns on the 1st Embodiment of this invention. 4 is an explanatory diagram showing a color arrangement pattern of a color selection filter of the color filter 22. FIG. 4 is an explanatory diagram showing a color arrangement pattern of a color selection filter of the color filter 22. FIG. 4 is an explanatory diagram showing a color arrangement pattern of a color selection filter of the color filter 22. FIG. 4 is an explanatory diagram showing a color arrangement pattern of a color selection filter of the color filter 22. FIG. 4 is an explanatory diagram showing a color arrangement pattern of a color selection filter of the color filter 22. FIG. FIG. 2B is an explanatory diagram illustrating a signal obtained by irradiation with R light when the color filter 22 illustrated in FIG. 2A is employed. FIG. 2B is an explanatory diagram illustrating a signal obtained by irradiation with G light when the color filter 22 illustrated in FIG. 2A is employed. FIG. 2B is an explanatory diagram illustrating a signal obtained by irradiation with B light when the color filter 22 illustrated in FIG. 2A is employed. FIG. 3 is an explanatory diagram showing a color arrangement pattern when a complementary color selection filter is used as the color filter 22. FIG. 3 is an explanatory diagram showing a color arrangement pattern when a complementary color selection filter is used as the color filter 22. FIG. 3 is an explanatory diagram showing a color arrangement pattern when a complementary color selection filter is used as the color filter 22. FIG. 3 is an explanatory diagram showing a color arrangement pattern when a complementary color selection filter is used as the color filter 22. FIG. 3 is an explanatory diagram showing a color arrangement pattern when a complementary color selection filter is used as the color filter 22. Explanatory drawing which shows the signal obtained by irradiation of R light in the case of employ | adopting the color filter 22 shown to FIG. 4A. Explanatory drawing which shows the signal obtained by irradiation of G light at the time of employ | adopting the color filter 22 shown to FIG. 4A. Explanatory drawing which shows the signal obtained by irradiation of B light in the case of employ | adopting the color filter 22 shown to FIG. 4A. The block diagram which shows the specific structure of the image process part 27 which performs a demosaicing process. The timing chart which shows the operation | movement employ | adopted in the 2nd Embodiment of this invention. The timing chart which shows the operation | movement employ | adopted in the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of an endoscope system including an endoscope according to a first embodiment of the present invention. The present embodiment makes it possible to obtain an endoscopic image having a sufficient resolution regardless of whether it is driven by a simultaneous or frame-sequential method with an endoscope equipped with one imager. As will be described later, the imager employed in the present embodiment employs a color filter configured by combining two types of filters having different spectral characteristics and a transparent filter, so that color reproducibility, resolution, and moving image are obtained. It has a solid-state imaging device with excellent responsiveness.

  As shown in FIG. 1, an endoscope system 1 is mounted on an endoscope 2 that is inserted into a body or a body cavity, a light source device 3 that supplies illumination light to the endoscope 2, and the endoscope 2. The image signal for display is input by driving the image capturing unit 21 and inputting the processor 4 as an image processing apparatus that performs image processing on the image capturing signal acquired by the image capturing, and the display image signal generated by the processor 4. And a color monitor 5 that constitutes a display unit that displays the image as an endoscopic image.

  The endoscope 2 includes an elongated insertion portion 7 that is inserted into the subject 6, an operation portion 8 provided at the rear end of the insertion portion 7, and a cable 9 that extends from the operation portion 8. The connector 10 provided at the end of the cable 9 is detachably connected to the light source device 3.

  A light guide 11 that transmits illumination light is inserted through the insertion unit 7, the operation unit 8, the cable 9, and the connector 10, and illumination light from the light source device 3 is inserted into a base end side end surface (incident end surface) of the light guide 11. It is designed to be incident.

  The light source device 3 includes a first light emitting diode (first LED) 12a, a second LED 12b, and a third LED 12c that generate illumination light. The LED light emission control circuit 13 in the light source device 3 supplies power for causing the LEDs 12a, 12b, and 12c to emit light, and controls the amount of light emission by varying the current supplied to the LEDs 12a, 12b, and 12c. Can be done.

  The light emitted from the LED 12 a is selectively transmitted through the dichroic mirror 14 a and is collected by the condenser lens 15. The light emitted from the LED 12b is selectively reflected by the dichroic mirror 14b, selectively reflected by the dichroic mirror 14a, and collected by the condenser lens 15. The light emitted from the LED 12c selectively passes through the dichroic mirror 14b, is selectively reflected by the dichroic mirror 14a, and is collected by the condenser lens 15. The condenser lens 15 collects the incident light and makes it incident on the incident end face of the light guide 11. The LEDs 12a, 12b, and 12c may emit R (red) band light (R light), G (green) band light (G light), and B (band light) (B light), respectively.

  The light source device 3 can sequentially emit R light, G light, and B light from the condenser lens 15 under the control of the LED light emission control circuit 13 when driven in a surface sequential manner. The light source device 3 can also emit white light having a wavelength band of, for example, 385 to 750 nm from the condenser lens 15 under the control of the LED light emission control circuit 13 during WLI observation. The light source device 3 can also emit, for example, G light and B light from the condenser lens 15 simultaneously or alternately under the control of the LED light emission control circuit 13 during NBI observation. Furthermore, the light source device 3 can emit light in a band necessary for the observation target from the condenser lens 15 under the control of the LED light emission control circuit 13 during special light observation such as infrared light (IR) observation. It is like that.

  The light guide 11 transmits the illumination light incident from the incident end surface, and emits the illumination light to the subject 6 from the distal end surface of the light guide 11 disposed at the distal end portion of the insertion portion 7 via the illumination lens 16. The LEDs 12a, 12b, and 12c, the dichroic mirrors 14a and 14b, the LED emission control circuit 13, the condenser lens 15, the light guide 11, and the illumination lens 16 constitute an illumination unit 17 that illuminates the subject 6.

  An observation window is provided at the distal end of the insertion portion 7 adjacent to the illumination lens 16 disposed in the illumination window, and an objective lens 18 is attached to the observation window. An imaging surface of an imaging device 19 such as a CMOS sensor or a charge coupled device (CCD) is disposed at the imaging position of the objective lens 18. The objective lens 18 and the imaging device 19 constitute an imaging unit 21 that receives the return light from the subject 6 illuminated by the illumination unit 17 and captures the subject optical image formed on the imaging surface by photoelectric conversion.

  The imaging device 19 has a plurality of pixels arranged in a two-dimensional matrix, and photoelectric conversion is performed in each pixel so that an imaging signal corresponding to the received light is output. . A color filter 22 having a color selection filter (also simply referred to as a filter) that restricts the wavelength band of light incident on each pixel for each pixel is disposed opposite to the imaging surface of the imaging element 19.

  One end of a signal line 24 inserted through the endoscope 2 is connected to the image sensor 19, and the other end of the signal line 24 is connected to a contact of the connector 10. The contact of the connector 10 is further detachably connected to the processor 4 via a signal line 25 in the connection cable. The processor 4 generates a display image signal to be displayed in color on the color monitor 5 by performing image processing on the image pickup signal output from the drive unit 26 that generates the image pickup element drive signal for driving the image pickup element 19 and the image pickup element 19. And an image processing unit 27.

  The imaging device 19 outputs an imaging signal obtained by photoelectric conversion to the image processing unit 27 when an imaging device driving signal is applied from the driving unit 26. The image processing unit 27 generates an image signal by demosaicing processing or the like for the image pickup signal and outputs the image signal to the color monitor 5. Thus, the image captured by the image sensor 19 on the color monitor 5 is displayed as an endoscopic image.

  In the present embodiment, the color filter 22 provided in the image sensor 19 includes three types of color selection filters, which are a primary color filter having two types of spectral characteristics as a color selection filter and a filter that does not limit the wavelength band. Yes. Further, the color filter 22 may be constituted by three types of color selection filters, which are a complementary color filter having two types of spectral characteristics as a color selection filter and a filter that does not limit the wavelength band. In addition, as a color selection filter (hereinafter referred to as a Y filter) that enters a pixel without limiting the wavelength band of the return light, a transparent filter, a neutral density filter, a gray filter, and the like are conceivable. Even when no color selection filter is arranged, the wavelength band of the return light is not limited. In the following description, the state in which the Y filter is disposed includes not only the state in which a filter such as a transparent filter is disposed but also the state in which no color selection filter is disposed.

(Example of primary color filter)
2A to 2E are explanatory diagrams showing color arrangement patterns of the color selection filter of the color filter 22. FIG. Each color selection filter is provided corresponding to each pixel, and the image sensor 19 outputs an image signal corresponding to the color arrangement of the color filter 22.

  2A to 2C show three examples of the color arrangement pattern when the primary color filter is employed. 2A to 2C, G represents a color selection filter (hereinafter referred to as G filter) that transmits the wavelength band of G light, and B represents a color selection filter (hereinafter referred to as B filter) that transmits the wavelength band of B light. Y indicates a state in which the Y filter is disposed, that is, a transparent color selection filter is disposed or a color selection filter is not disposed.

  2A to 2C show that the color filters 22 having the same color arrangement pattern are disposed horizontally and vertically every 2 × 2 for each pixel constituting the light receiving element that performs photoelectric conversion. Yes. FIG. 2A shows an example in which the Y filter is arranged in two diagonal pixels of 2 × 2 pixels, and the G filter and the B filter are arranged in the remaining two pixels, one pixel each. FIG. 2B shows an example in which Y filters are arranged side by side in two vertical pixels of 2 × 2 pixels, and one G filter and one B filter are arranged for each of the remaining two pixels. FIG. 2C shows an example in which Y filters are arranged in two horizontal pixels of 2 × 2 pixels, and a G filter and a B filter are arranged for each of the remaining two pixels.

  That is, in all of the examples of FIGS. 2A to 2C, Y filters are arranged in 50% of all pixels, and primary colors are used for half of the remaining 50% pixels (25% of all pixels). An example is shown in which a G filter that is a filter is arranged, and a B filter that is a primary color filter is arranged on the remaining pixels (25% of the pixels). In the present embodiment, the Y filter may be arranged in 50% or more of all pixels.

  As described above, in the present embodiment, as the color filter 22, a color filter composed of a primary color filter having two types of spectral characteristics and a Y filter can be adopted, and the color filters shown in FIGS. Instead of the G filter, a color selection filter (hereinafter referred to as an R filter) that transmits the wavelength band of R light may be used, and the color filter 22 may be configured by R, B, and Y filters. For example, FIG. 2D shows an example in which an R filter is employed instead of the G filter in FIG. 2A. Moreover, it replaces with B filter of FIG. 2A-FIG. 2C, R filter may be employ | adopted and the color filter 22 may be comprised by R, G, and Y filter. For example, FIG. 2E shows an example in which an R filter is employed instead of the B filter in FIG. 2A.

(Example of simultaneous demosaicing (primary color filter))
Light having a wavelength band limited according to the spectral characteristics of each color selection filter of the color filter 22 is incident on each pixel of the image sensor 19. Each pixel in which the R filter, G filter, B filter, or Y filter is disposed is referred to as an R pixel, a G pixel, a B pixel, and a Y pixel, respectively, and photoelectric conversion of these R pixel, G pixel, B pixel, and Y pixel is performed. The obtained pixel signals are referred to as R signal, G signal, B signal, and Y signal, respectively. For the Y pixel, when the subject 6 is irradiated with R light, G light, and B light, respectively, an R signal, a G signal, and a B signal are obtained as the Y signal.

  For example, in the example of FIG. 2A, a Y signal, a G signal, a B signal, and a Y signal are obtained from the pixels P1 to P4, respectively. The Y signal of the pixel P1 when the subject 6 is irradiated with white light is the sum of the R signal, G signal, and B signal of the pixel P1 when the subject 6 is irradiated with R light, G light, and B light, respectively. expressed. That is, Y = R + G + B, and R = Y−G−B. Since the Y filter is disposed in the pixel P1, the R signal in the pixel P1 is R = Y− using the estimated value G ′ of the G signal in the pixel P1 and the estimated value B ′ of the B signal in the pixel P1. It is represented by G′-B ′.

  The estimated value G ′ can be obtained by interpolation processing using a plurality of G signals from a plurality of G pixels appearing for each pixel in the horizontal and vertical directions. Similarly, the estimated value B ′ can be obtained by interpolation processing using a plurality of B signals from a plurality of B pixels appearing for each pixel in the horizontal and vertical directions.

  In this way, the R signal, the estimated value G ′, and the estimated value B ′ can be obtained for the pixel P1. By the same method, the R signal, the estimated value G ′ and the estimated value B ′ in the pixel P4 can be obtained for the pixel P4. Further, the estimated value R ′, the G signal, and the estimated value B ′ can be obtained for the pixel P2, and the estimated value R ′, the estimated value G ′, and the B signal can be obtained for the pixel P3.

If a transparent filter is used as the Y filter or the filter is not arranged, the amount of incident light on the Y pixel is larger than the amount of incident light on the other R, G, and B pixels, It is conceivable that the calculation accuracy in the demosaicing process decreases. Therefore, in such a case, the sensitivity of each pixel may be matched by adopting a neutral density filter as the Y filter or by separately arranging a neutral density filter in a pixel where no filter is arranged. For example, a neutral density filter may be attached to the Y pixel so that the integral value of the spectral sensitivity with other pixels is within a predetermined constant ratio.
In addition, the R, G, B filters are not necessarily completely separated from each other in the pass wavelength band, and are adjacent to the pass wavelength band of other filters in the vicinity of the pass wavelength band boundary (hereinafter referred to as color mixture). May be referred to as an area). When the R, G, and B filters have a mixed color gamut, it can be considered that the calculation accuracy in the demosaicing process is lowered. Therefore, in order to suppress such a decrease in calculation accuracy, the emitted light of the light source device 3 may be controlled. For example, you may comprise so that the wavelength band (mixed color area) where the wavelength band of the emitted light of each LED12a-12c of the light source device 3 may mutually overlap is not included.

  Note that the demosaicing method described above is merely an example, and various demosaicing methods can be employed. That is, in the present embodiment, the color filter 22 is composed of two kinds of primary color filters having different spectral characteristics and a Y filter that does not limit the wavelength band, whereby an RGB signal can be obtained by a predetermined demosaicing process. .

  Further, during NBI observation, the subject 6 may be irradiated with the combined light of G and B light. In this case, for example, G + B which is the sum of the G signal and the B signal is obtained from the pixel P1 in FIG. 2A. Therefore, the B signal at the pixel P1 can be obtained by B = Y−G ′ using the estimated value G ′ of the G signal at the pixel P1. Rather than obtaining the estimated value B ′ by interpolation processing using a plurality of B signals, the B signal is obtained from the Y signal based on the incident light of the corresponding pixel, and from the simultaneous NBI observation using the Bayer array High resolution can be obtained.

  In the present embodiment, the color components that are not separated by the color selection filter are calculated by calculation as described above. In order to be able to calculate the color component with high accuracy in this calculation, a filter having a spectral characteristic having a pass band of a wavelength band of 570 to 750 nm that can be said to be a wavelength band between yellow (Ye) and R is adopted as the R filter. It is preferable to do. Further, as the B filter, it is preferable to employ a filter having a spectral characteristic having a pass band in the wavelength band of 385 to 495 nm, which can be said to be a wavelength band between purple (V) and B. Further, as the G filter, it is preferable to employ a spectral characteristic filter having a wavelength band of 450 to 650 nm, which can be said to be a wavelength band between red (R) and blue (B).

  As described above, the endoscope according to the present embodiment can be driven simultaneously with excellent real-time characteristics. Further, in the present embodiment, it is possible to drive in a frame sequential manner.

(Example of demosaicing in surface sequential type (primary color filter))
In the surface sequential method, the subject 6 is sequentially irradiated with R light, G light, and B light, and these return lights are sequentially incident on the imaging surface of the image sensor 19. The Y filter transmits these R light, G light, and B light as they are. Therefore, for the Y pixel, the R signal is output in the same manner as the R pixel when the R light is irradiated, the G signal is output as in the G pixel when the G light is irradiated, and the B signal is output in the same manner as the B pixel when the B light is irradiated. To do.

3A to 3C are explanatory diagrams illustrating signals obtained by irradiation with R light, G light, and B light when the color filter 22 illustrated in FIG. 2A is employed. 3A to 3C, R, G, and B indicate an R signal, a G signal, and a B signal, respectively.
As shown in FIG. 3A, in the pattern shown in FIG. 2A, when R light is irradiated, two pixels out of 2 × 2 pixels output an R signal in the same manner as the R pixel. Note that the remaining two pixels can be obtained by interpolation processing using a plurality of R signals obtained from pixels around each pixel.

  As shown in FIG. 3B, in the pattern shown in FIG. 2A, when G light is irradiated, 3 pixels out of 2 × 2 pixels output a G signal in the same manner as the G pixel. Note that the remaining one pixel can be obtained by interpolation processing using a plurality of G signals obtained from pixels around the pixel.

  As shown in FIG. 3C, in the pattern shown in FIG. 2A, when B light is irradiated, three pixels out of 2 × 2 pixels output a B signal in the same manner as the B pixel. The remaining one pixel can be obtained by interpolation processing using a plurality of B signals obtained from pixels around the pixel.

  When the endoscope system is driven in a frame sequential manner when a Bayer color filter is employed, an R signal is obtained from one pixel out of 2 × 2 pixels when irradiated with R light, and 2 × 2 when irradiated with B light. A B signal is obtained from one pixel of the pixels, and a G signal is obtained only from two of the 2 × 2 pixels at the time of G light irradiation. Therefore, when an imager employing a Bayer arrangement is used, sufficient resolution cannot be obtained by the frame sequential method.

On the other hand, in the present embodiment, the color filter 22 is composed of two kinds of primary color filters having different spectral characteristics and a Y filter that does not limit the wavelength band. A resolution RGB signal can be obtained. In particular, during NBI observation, the subject 6 is irradiated with G light or B light in a surface sequential manner. In this case, 3/4 of all pixels operate in the same manner as the G pixel or B pixel. In other words, it is possible to obtain an extremely higher resolution than when the Bayer array is used.
Although only the example of the color filter of FIG. 2A has been described in FIGS. 3A to 3C, sufficient resolution can be obtained in the case of the color filter of FIGS. 2B to 2E.

(Example of complementary color filter)
4A to 4E are explanatory views showing color arrangement patterns when a complementary color selection filter is used as the color filter 22. 4A to 4C, Mg indicates a color selection filter (hereinafter referred to as Mg filter) that transmits the wavelength band of magenta (Mg) light, and Cy indicates a color selection filter that transmits the wavelength band of cyan (Cy) light ( Hereinafter, it is referred to as a Cy filter), and Y is a Y filter.

  4A to 4C also show that the color filters 22 having the same color arrangement pattern are arranged horizontally and vertically every 2 × 2 for each pixel constituting the light receiving element that performs photoelectric conversion. ing. FIG. 4A shows an example in which the Y filter is arranged in two diagonal pixels of 2 × 2 pixels, and the Mg filter and the Cy filter are arranged in the remaining two pixels, one pixel each. FIG. 4B shows an example in which Y filters are arranged in two vertical pixels of 2 × 2 pixels, and an Mg filter and a Cy filter are arranged for each of the remaining two pixels. FIG. 4C shows an example in which Y filters are arranged in two horizontal pixels of 2 × 2 pixels, and an Mg filter and a Cy filter are arranged for each of the remaining two pixels.

  That is, in all of the examples of FIGS. 4A to 4C, Y filters are arranged in 50% of all pixels, and complementary colors are used for half of the remaining 50% pixels (25% of all pixels). An example is shown in which an Mg filter that is a filter is disposed, and a Cy filter that is a complementary color filter is disposed on the remaining pixels (25% of the pixels). In the present embodiment, even when a complementary color filter is used, the Y filter may be arranged in 50% or more of all pixels.

  As described above, in the present embodiment, the color filter 22 may be a filter composed of a complementary color filter having two types of spectral characteristics and a Y filter, and the Mg of FIGS. 4A to 4C. Instead of the filter, a color selection filter (hereinafter referred to as a Ye filter) that transmits the wavelength band of yellow (Ye) light may be used, and the color filter 22 may be configured by Ye, Cy, and Y filters. For example, FIG. 4D shows an example in which a Ye filter is adopted instead of the Mg filter of FIG. 4A. Further, instead of the Cy filter of FIGS. 4A to 4C, a Ye filter may be adopted, and the color filter 22 may be configured by Ye, Mg, and Y filters. For example, FIG. 4E shows an example in which a Ye filter is adopted instead of the Cy filter in FIG. 4A.

(Simultaneous demosaicing example (complementary color filter))
Each pixel in which the Mg filter, Cy filter, Ye filter, or Y filter is arranged is referred to as an Mg pixel, Cy pixel, Ye pixel, and Y pixel, respectively, and photoelectric conversion of these Mg pixel, Cy pixel, Ye pixel, and Y pixel is performed. The obtained pixel signals are referred to as Mg signal, Cy signal, Ye signal, and Y signal, respectively.

  The Cy filter transmits G light and B light, the Ye filter transmits R light and B light, and the Mg filter transmits R light and G light. That is, the Cy filter absorbs R light from white light. For example, when the subject 6 is irradiated with white light, R = Y−Cy ′ is established for the pixel P1 in FIG. 4A. Cy ′ represents an estimated value Cy ′ of the Cy signal in the pixel P1. Similarly, G = Y−Mg ′ is established for the pixel P1 using the estimated value Mg ′ of the Mg signal in the pixel P1. Further, the B signal in the pixel P1 is obtained by B = YRG.

  Thus, the R signal, the G signal, and the B signal can be obtained for the pixel P1. By the same method, R, G, B signals can be obtained for the other pixels P2 to P4.

  Note that the demosaicing method described above is merely an example, and various demosaicing methods can be employed. That is, in the present embodiment, the color filter 22 is composed of two types of complementary color filters having different spectral characteristics and a Y filter that does not limit the wavelength band, whereby an RGB signal can be obtained by a predetermined demosaicing process. .

  In addition, during the NBI observation, the subject 6 may be irradiated with the combined light of G and B light. Since the Mg pixel transmits B light, in this case, it can be considered that the Mg pixel outputs a B signal. From the Y pixel, B + G which is the sum of the B signal and the G signal is output. Further, G + B, which is the sum of the B signal and the G signal, is also obtained from the Cy pixel. Therefore, for the Y pixel and the Cy pixel, the G signal can be obtained by subtracting the estimated value B ′ of the B signal. Rather than obtaining the estimated value G ′ by interpolation processing using a plurality of G signals, the G signal is obtained from the signal based on the incident light of the corresponding pixel, and compared with the simultaneous NBI observation using the Bayer array. High resolution can be obtained.

  Thus, even when the complementary color filter is used, the color components that are not spectrally separated by the color selection filter are calculated by calculation as described above. In order to calculate the color component with high accuracy in this calculation, it is preferable to employ a filter having spectral characteristics having a pass band in the wavelength band of 385-480 nm and 610-700 nm, which can be said to be the wavelength band of Mg, as the Mg filter. . Further, as the Cy filter, it is preferable to employ a spectral characteristic filter having a 385-610 nm wavelength band as a pass band, which can be said to be a Cy wavelength band. Further, as the Ye filter, it is preferable to employ a filter having spectral characteristics having a pass band of a wavelength band of 470 to 750 nm that can be said to be a wavelength band between the wavelength bands of the B light and the R light.

  As described above, the endoscope according to the present embodiment can be driven simultaneously with excellent real-time characteristics even when the complementary color filter is used. Further, even when a complementary color filter is employed, it is possible to drive in a frame sequential manner.

(Example of demosaicing in frame sequential type (complementary color filter))
In the surface sequential method, the subject 6 is sequentially irradiated with R light, G light, and B light, and these return lights are sequentially incident on the imaging surface of the image sensor 19. The Y filter transmits these R light, G light, and B light as they are. Therefore, for the Y pixel, the R signal is output in the same manner as the R pixel when the R light is irradiated, the G signal is output as in the G pixel when the G light is irradiated, and the B signal is output in the same manner as the B pixel when the B light is irradiated. To do.

  5A to 5C are explanatory diagrams illustrating signals obtained by irradiation with R light, G light, and B light when the color filter 22 illustrated in FIG. 4A is employed. 5A to 5C, R, G, and B indicate an R signal, a G signal, and a B signal, respectively. As shown in FIG. 5A, in the pattern of FIG. 4A, when R light is irradiated, three pixels excluding Cy pixels out of 2 × 2 pixels output R signals in the same manner as R pixels. Note that the remaining one pixel can be obtained by interpolation processing using a plurality of R signals obtained from pixels around each pixel.

  Further, as shown in FIG. 5B, in the pattern of FIG. 4A, when G light is irradiated, three pixels excluding Mg pixels out of 2 × 2 pixels output G signals in the same manner as the G pixels. Note that the remaining one pixel can be obtained by interpolation processing using a plurality of G signals obtained from pixels around the pixel.

  Further, as shown in FIG. 5C, in the pattern of FIG. 4A, when the B light is irradiated, all pixels of 2 × 2 pixels output B signals in the same manner as the B pixels.

In the present embodiment, the color filter 22 is composed of two types of complementary color filters having different spectral characteristics and a Y filter that does not limit the wavelength band, so that an RGB signal with sufficient resolution can be obtained even when the frame sequential method is adopted. Can be obtained. In particular, at the time of NBI observation, an extremely higher resolution can be obtained than when the Bayer array is used.
Although only the example of the color filter of FIG. 4A has been described in FIGS. 5A to 5C, sufficient resolution can be obtained similarly in the case of the color filter of FIGS. 4B to 4E.

  FIG. 6 is a block diagram showing a specific configuration of the image processing unit 27 that performs demosaicing processing.

The image processing unit 27 includes a control circuit 31 that controls a plurality of circuits in the image processing unit 27. The control circuit 31 can be configured by a processor using a CPU or the like (not shown), and may be configured to control each unit according to a program stored in the memory. The image processing unit 27 receives an imaging signal from the imaging unit 21. The pre-processing circuit 32 clamps OB (optical black) in the imaging signal and determines a black level using the parameter value stored in advance in the memory 31a in the control circuit 31 for the input imaging signal. Processing, correction processing for defective pixels of the image sensor 19, noise reduction processing, and the like are performed.
The output signal of the preprocessing circuit 32 is input to an amplifier 33 for gain adjustment. The amplifier 33 is controlled by the control circuit 31 to amplify the imaging signal and output it to the A / D conversion circuit 34. The control circuit 31 adjusts the brightness of the endoscopic image displayed on the color monitor 5 by adjusting the gain of the amplifier 33 based on the detection value subjected to the light modulation detection. The A / D conversion circuit 34 converts the input analog imaging signal into a digital imaging signal. The output signal of the A / D conversion circuit 34 is given to the demosaicing processing circuit 35.

  When the endoscope is driven simultaneously, the demosaicing processing circuit 35 performs color extraction processing, interpolation processing, and the like to obtain R, G, and B signals. Further, when the endoscope is driven in a frame sequential manner, the demosaicing processing circuit 35 performs an interpolation process on the R, G, B signals input in the frame sequential manner, and then performs the R, G in the frame sequential manner. , B signals are output. In the case of the frame sequential type, it is necessary to synchronize the R, G, B signals input in the frame sequential manner, and in the case of the simultaneous type, this synchronization processing is unnecessary.

  In the present embodiment, since the endoscope can be driven simultaneously and in a frame sequential manner, the output of the demosaicing processing circuit 35 is supplied to the synchronization circuit 38 via the switch 36a. ing. The output of the demosaicing processing circuit 35 is also supplied to the white balance circuit 40 via the switches 36a and 36b.

  The switches 36a and 36b are controlled by the control circuit 31 and when the endoscope is driven in a simultaneous manner, the output of the demosaicing processing circuit 35 is supplied to the synchronization circuit 38, and the endoscope is sequentially operated. , The output of the demosaicing processing circuit 35 is directly supplied to the white balance circuit 40.

  The image processing unit 27 includes a memory unit 39 including, for example, three memories 39R, 39G, and 39B for synchronization. The synchronization circuit 38 stores the imaging signal obtained when the illumination light is R light in the memory 39R, and stores the imaging signal obtained when the illumination light is G light in the memory 39G. The imaging signal obtained in the case of light is stored in the memory 39B. The synchronization circuit 38 reads out the R, G, and B signals acquired at different timings from the memory unit 39 and outputs them to the white balance circuit 40 via the switch 36b.

  The white balance circuit 40 receives R, G, B signals from the demosaicing processing circuit 35 or the synchronization circuit 38. The white balance circuit 40 is composed of, for example, three gain variable amplifiers, adjusts the white balance of the input R, G, and B signals and outputs the adjusted white balance to the post-processing circuit 41 and the dimming circuit 42.

  The post-processing circuit 41 performs, for example, gradation conversion processing, color enhancement processing, and contour enhancement processing using the gradation variable coefficient, color conversion coefficient, and edge enhancement coefficient stored in advance in the memory 31a of the control circuit 31. The processed R, G, B signals are output to the color monitor 5 as display image signals. In this way, the color monitor 5 displays the image of the subject 6 captured by the imaging unit 21.

  The dimming circuit 42 generates a luminance signal from the signal input from the white balance circuit 40, for example, generates a signal having an average value of the luminance signal as a brightness signal, and adjusts a signal that is a difference from the target brightness. It outputs to the LED light emission control circuit 13 as an optical signal. The LED light emission control circuit 13 controls the light emission amounts of the LEDs 12a to 12c based on the dimming signal so that the brightness of the endoscope image becomes a target brightness.

  Next, the operation of the embodiment configured as described above will be described.

  Now, normal observation shall be performed. The light source device 3 controls the LEDs 12a to 12c to emit white light from the condenser lens 15 in the case of the simultaneous type, and to frame R, G, B light from the condenser lens 15 in the case of the frame sequential type. Output every time. The light emitted from the light source device 3 is guided to the distal end of the insertion portion 7 through the light guide 11 and is irradiated onto the subject 6 from the illumination lens 16. The return light from the subject 6 enters the image sensor 19 through the objective lens 18. The color filter 22 provided opposite to the pixels of the image sensor 19 limits the wavelength band of light incident on each pixel of the image sensor 19.

  The color filter 22 includes a primary color filter having two types of spectral characteristics and a color selection filter using a Y filter, or a complementary color filter having two types of spectral characteristics and a color selection filter using a Y filter. In any case, the Y filter is provided to face 50% or more of all pixels. The state where the Y filter is arranged is a state where no color selection filter is arranged or a state where a transparent filter or the like is arranged, and incident light restricts the wavelength band in the Y pixel where the Y filter is arranged. It is incident as it is without being.

  Therefore, in the simultaneous expression, the Y signal obtained by the Y pixel is R + G + B, and R, G, and B are obtained by calculation with two types of signals obtained by the pixels corresponding to the other two primary colors or complementary color filters. G and B signals can be obtained. That is, the image pickup signal from the image pickup device 19 is transmitted to the processor 4 and demosaiced by the demosaicing processing circuit 35 in the image processing unit 27 to obtain R, G, B signals. The output of the demosaicing processing circuit 35 is supplied to the white balance circuit 40 via the switches 36 a and 36 b, subjected to white balance processing, and then supplied to the monitor 5 via the post-processing circuit 41. In this way, an endoscopic image of the subject 6 can be observed on the monitor 5.

  In the field sequential, Y signals obtained by Y pixels are R signals, G signals, or B signals, and two kinds of signals obtained by pixels corresponding to the other primary color or complementary color filters having two kinds of spectral characteristics. Thus, it is possible to obtain R, G, B signals with sufficient resolution in a frame sequential manner. That is, the image pickup signal from the image pickup device 19 is transmitted to the processor 4, and interpolation processing is performed by the demosaicing processing circuit 35 in the image processing unit 27 to obtain R, G, B signals. The output of the demosaicing processing circuit 35 is supplied to the synchronization circuit 38 via the switch 36a and is synchronized. The synchronized R, G, B signals are supplied to the white balance circuit 40 and subjected to white balance processing, and then supplied to the monitor 5 via the post-processing circuit 41. In this way, an endoscopic image of the subject 6 can be observed on the monitor 5.

  In addition, the endoscope uses an observation method that uses only a part of the wavelength of visible light and a band other than visible light. For this reason, the color filter 22 is distributed more in the Y filter than in the other two types of color selection filters. Thereby, an image with high resolution can be obtained even in NBI observation.

  In the NBI observation, the light source device 3 controls the LEDs 12a to 12c to emit the combined light of G and B light from the condensing lens 15 in a simultaneous manner, and the G and B light from the condensing lens 15 in a surface sequential manner. Are output sequentially. Therefore, the Y signal obtained by the Y pixel is G + B in the simultaneous type, and becomes the G signal or the B signal in the frame sequential type. Accordingly, the interpolation processing using the two types of signals obtained by the pixels corresponding to the other two primary colors having the spectral characteristics or the complementary color filter and the calculation with the Y signal are higher than when the Bayer array filter is used. Resolution R, G, B signals can be obtained.

  In particular, when a complementary color filter is used as a color selection filter having two types of spectral characteristics and driving is performed by plane sequential, G and B signals use 3/4 of all pixels in NBI observation. Therefore, extremely high resolution R, G, B signals can be obtained.

  Further, at the time of infrared light (IR) observation, the light source device 3 emits infrared light (IR). In this case, high-sensitivity IR observation is possible by using only the Y signal obtained from the Y pixel.

  As described above, in the present embodiment, the color filter employed in the imager is constituted by the primary color filter and the Y filter having two types of spectral characteristics, or the complementary color filter and the Y filter having two types of spectral characteristics. R, G, B color images can be obtained even when the endoscope is driven either simultaneously or in a sequential manner. In addition, the Y filter is arranged at 50% or more of all pixels, so that the resolution can be improved. Furthermore, imaging with sufficient resolution is also possible in NBI observation, infrared light observation, and the like. As a result, an endoscope image having a sufficient resolution can be obtained by an endoscope equipped with one imager in any of the simultaneous driving method and the frame sequential driving method.

(Second Embodiment)
7 and 8 are timing charts showing the operation employed in the second embodiment of the present invention. Note that the hardware configuration in the present embodiment is the same as that in FIGS. 1 and 6, and a description thereof will be omitted. The configuration of the color filter 22 described in FIGS. 2A to 2C and FIGS. 4A to 4C is the same as that of the first embodiment, and the description thereof is omitted.

  In this embodiment, resolution is improved by combining frame-sequential driving with simultaneous driving. As described above, when frame-sequential driving is performed using an imager that employs a Bayer color filter, the G signal and the B signal can be obtained from only one of the 2 × 2 four pixels. On the other hand, when frame sequential driving is performed using an imager employing the above-described color filter 22, the G signal and the B signal are 2 × as shown in FIGS. 3B, 3C, 5B, and 5C. 2 pixels are obtained from 3 pixels or 4 pixels, respectively, and the resolution is high.

  Therefore, in the present embodiment, simultaneous driving is performed by irradiating white light, and at the same time, the subject is irradiated with colored light for obtaining an image that is desired to be superior in terms of resolution, color reproducibility, and sensitivity. To do. For example, when only B light is irradiated at a predetermined timing, an image superior in terms of resolution, color reproducibility, and sensitivity can be obtained for B light.

  7 and 8 show the start timing of the vertical period by the vertical synchronization signal VD in the upper stage, the illumination period in the middle stage, and the readout period of the imaging signal in the lower stage. 7 and 8, the illumination period and the readout period are indicated by H. FIG. 7 shows an example of the rolling shutter system, and FIG. 8 shows an example of the global shutter system.

  In the rolling shutter system, every time reading of one line is completed, pixels of the line where reading is completed are reset, and exposure possible periods of each line occur at different timings. Accordingly, at a certain point in time, the line before the readout is the exposure possible period of the previous frame, and the line after the readout is the exposure possible period of the subsequent frame. Therefore, in the rolling shutter system, if illumination is performed during readout of an imaging signal, illumination is performed in a state in which illumination for exposure of the previous frame and illumination for exposure of the subsequent frame are mixed. Therefore, when performing imaging by a rolling shutter method in an endoscope, illumination may be performed in a period other than the readout period.

  FIG. 7 shows an example of this case. In the readout period of the imaging signal indicated by H, illumination is not performed as indicated by L, and is indicated by H when the readout period of the imaging signal ends as indicated by L. Illumination is started. And the imaging signal obtained by the imaging device 19 by this illumination is read in the next vertical period.

  FIG. 8 shows an example of the global shutter system, which differs from FIG. 7 only in that the exposure period is set in the same period as the pixel signal readout period.

  7 and 8, the light source device 3 emits white light during an illumination period other than hatching. The drive unit 26 drives the image sensor 19 simultaneously, and the image processing unit 27 performs demosaicing processing on the output of the image sensor 19 to obtain R, G, and B signals.

  In the present embodiment, the light source device 3 emits illumination light of a predetermined color light at a predetermined timing during simultaneous driving. 7 and 8 show an illumination period of illumination light of a predetermined color light by hatching. In the next frame period (hatching period) after the illumination period, the drive unit 26 drives the image pickup device 19 in a frame sequential manner to read out an image pickup signal, and the image processing unit 27 interpolates the output of the image pickup device 19. To do. The image processing unit 27 synthesizes the R, G, B signals obtained by the demosaicing processing at the time of simultaneous driving and the image of a predetermined color obtained by the interpolation processing at the time of frame sequential driving, B signal is acquired.

  For example, B light is emitted from the light source device 3 as illumination light during frame sequential driving. For example, when the color filter 22 including the complementary color filter and the Y filter in FIG. 4A is employed, it is possible to obtain B signals from all the pixels as shown in FIG. 5C by illumination with B light. By synthesizing this B signal with the R, G, B signals obtained by simultaneous driving, an image superior in terms of resolution, color reproducibility and sensitivity can be obtained for the B signal.

  Note that the light to be emitted at the time of the frame sequential driving is not limited to the B light, but may be R light or G light, or color light used for NBI observation or infrared light observation may be emitted. Further, the frame sequential driving may be performed at a predetermined cycle or may be performed at an arbitrary timing that is not periodic.

  As described above, in the present embodiment, R and G obtained by the simultaneous method can be obtained by utilizing the fact that a signal having a sufficient resolution can be obtained regardless of whether the imager is driven by the simultaneous method or the frame sequential method. , B signal and a signal obtained based on a predetermined color light in a frame sequential manner, an image superior in terms of resolution, color reproducibility and sensitivity is obtained for an image corresponding to the predetermined color light. be able to.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment.

    DESCRIPTION OF SYMBOLS 1 ... Endoscopy system, 2 ... Endoscope, 3 ... Light source device, 4 ... Processor, 7 ... Insertion part, 11 ... Light guide, 12a-12c ... LED, 13 ... LED light emission control circuit, 19 ... Imaging element, 21 ... an imaging unit, 22 ... a color filter, 27 ... an image processing unit.

Claims (14)

A solid-state imaging device having a plurality of pixels arranged in a two-dimensional matrix and a color filter in which a color selection filter is arranged corresponding to the pixels;
The solid-state imaging device is attached, and includes an insertion unit that makes return light from a subject enter the pixel,
The color filter is
The first pixel that is 50% or more of all the pixels of the solid-state imaging device is incident on the return light without restricting the wavelength band of the return light, and the second pixel other than the first pixel of the all pixels. A wavelength band of the return light is limited to be incident on the pixel by the first spectral characteristic, and the first spectral characteristic is applied to the remaining third pixels other than the first and second pixels among all the pixels. An endoscope, wherein the wavelength band of the return light is limited by a second spectral characteristic different from that of the return light. The endoscope according to claim 1, wherein the color selection filter that gives the first and second spectral characteristics is a primary color filter. The primary color filter is any one of an R filter that passes a red light band, a G filter that passes a green light band, and a B filter that passes a blue light band. Endoscope. The primary color filter is any one of a first filter that passes the wavelength band of 570-750 nm, a second filter that passes the wavelength band of 385-495 nm, and a third filter that passes the wavelength band of 450-650 nm. The endoscope according to claim 2, wherein the two filters are used. The endoscope according to claim 1, wherein the color selection filter that gives the first and second spectral characteristics is a complementary color filter. 6. The complementary color filter is any one of a Mg filter that passes a magenta light band, a Cy filter that passes a cyan light band, and a Ye filter that passes a yellow light band. The endoscope described. The complementary color filter is a fourth filter that passes the wavelength band of 385-610 nm, a fifth filter that passes the wavelength bands of 385-480 nm and 610-700 nm, and a sixth filter that passes the wavelength band of 470-750 nm. The endoscope according to claim 5, which is any two of the filters. The endoscope according to claim 1, further comprising a neutral density filter that reduces an amount of the return light incident on the first pixel. The endoscope according to any one of claims 1 to 8,
A light source device that generates illumination light that irradiates the subject through the insertion unit;
An endoscope system comprising: a processor that controls the light source device and the solid-state imaging device to drive the endoscope in a simultaneous or frame-sequential manner. The processor is provided with a first pixel signal from the first pixel of the solid-state imaging device, a second pixel signal from the second pixel, and a third pixel signal from the third pixel. The endoscope system according to claim 9, further comprising an image processing circuit that obtains R, G, and B signals by demosaicing processing using the first to third pixel signals. The endoscope system according to claim 10, wherein the image processing circuit obtains the R, G, and B signals by demosaicing processing based on the first pixel signal. In the normal observation mode, the processor causes the light source device to emit white light having a spectral characteristic in a range of 385 to 750 nm,
In the narrow band light observation mode, the light source device is irradiated with irradiation light including G light and B light bands simultaneously or sequentially,
The endoscope system according to claim 9, wherein in the special light observation mode, the light source device is irradiated with light only in a band necessary for special light observation. The endoscope system according to claim 9, wherein the processor controls the light source device and the solid-state imaging device to switch and drive the endoscope between a simultaneous type and a frame sequential type. The endoscope system according to claim 13, wherein the processor performs frame sequential driving during simultaneous driving.

Images