JP6800664B2 - Endoscope device - Google Patents

Description

The present invention relates to an endoscopic device.

Conventionally, special light observation such as NBI (Narrow Band Imaging) or BLI (Blue Laser Imaging) is used in which a narrow band light is irradiated to an observed portion and the reflected light is imaged (for example, Patent Document). (See 1 and 2), which can facilitate the detection of lesions. On the other hand, an image sensor provided with a color filter having an RGB Bayer arrangement is used so that simultaneous observation can be performed while obtaining good color reproduction in white light observation.

In special light observation, B information is important because the information (B information) of the reflected light from purple to blue greatly contributes to the image of the capillaries on the surface layer of the mucous membrane. However, in the RGB Bayer array, the B pixel that detects the reflected light from purple to blue is only one-fourth of all the pixels, so that the resolution of capillaries and the like is insufficient. Patent Documents 1 and 2 try to solve such a problem.

In Patent Document 1, B information is acquired even in the G pixel by the G pixel having a secondary sensitivity region in the purple wavelength region, and B is obtained from the G pixel signal by performing a correlation calculation between the G pixel signal and the R pixel signal. Information is being extracted. In Patent Document 2, when observing narrow-band light with an endoscope using a complementary color image sensor, blue narrow-band light and green narrow-band light are surfaced in order to improve the separability of B information and G information. The light is emitted in a sequential manner.

Japanese Unexamined Patent Publication No. 2012-170639 JP 2015-66062

However, in the case of Patent Document 1, there is a problem that the extraction accuracy of B information becomes low in a region where the correlation between the G pixel signal and the R pixel signal is low at the time of acquiring a white light image. Further, when B information is extracted from the G pixel signal by image processing, a gain is applied to the image signal. There is a problem that the noise of the image is increased by this gain. When the surface sequential method of Patent Document 2 is used, there is a problem that color shift occurs in the narrow band optical image.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an endoscope device capable of acquiring a narrow band optical image and a white light image having good image quality.

In order to achieve the above object, the present invention provides the following means.
One aspect of the present invention includes a light source device that outputs illumination light that illuminates a subject, and a color filter array that includes color filters of a plurality of colors arranged in two dimensions, and photographs the subject illuminated by the illumination light. The light source device includes a first solid-state illuminating element that emits light having a peak intensity in the purple wavelength region, and a second solid-state illuminating element that emits light having a peak intensity in the blue wavelength region. Then, the third solid-state illuminating element that emits light having a peak intensity in the green wavelength region, the first solid-state illuminating element, the second solid-state illuminating element, and the third solid-state illuminating element are turned on and off. The color filter array includes a cyan color filter and a blue color filter, and the cyan color filter has optical characteristics for transmitting green, blue, and purple light. Then, in the first observation mode , the control unit turns on the first solid-state lighting element and the third solid-state lighting element at the same time , turns off the second solid-state lighting element, and makes a second observation. In the mode, it is an endoscope device that turns on the second solid-state lighting element and the third solid-state lighting element in order.

According to one aspect of the present invention, when the subject is irradiated with illumination light including purple, blue and green light, the reflected light of purple, blue and green incident on the image sensor from the subject is transmitted through the color filter array. Then, it is detected by the pixels of the image sensor.

When acquiring a narrow-band optical image, the subject is irradiated with only purple light and green light whose wavelengths are different from each other. At this time, the control unit controls the first and third solid-state lighting elements so that the purple light and the green light are simultaneously irradiated. The purple reflected light containing the information on the capillaries is detected not only by the B pixel corresponding to the blue color filter (B filter) but also by the Cy pixel corresponding to the cyan color filter (Cy filter). As a result, the amount of information of the purple reflected light containing a large amount of information on the capillaries increases, so that a high-quality narrow-band optical image having a high resolution can be acquired.

Since the Cy pixel detects the green reflected light in addition to the purple reflected light, it is necessary to accurately extract the green reflected light information from the Cy pixel in order to improve the color separation. Since the information contained in the B pixel signal is dominated by the purple reflected light information, the B information predicted from the Cy pixel and the B pixel signal located in the vicinity thereof is subtracted from the Cy pixel signal. Therefore, the information of the reflected light of green can be extracted with high accuracy.

On the other hand, when acquiring a white light image, the subject is irradiated with purple, blue and green light. At this time, the control unit controls the first, second, and third solid-state lighting elements so that the blue light and the green light whose wavelengths are close to each other are sequentially irradiated. As a result, the blue reflected light and the green reflected light are detected by the Cy pixels in a time-divided manner, so that the blue reflected light information and the green reflected light information are acquired separately without being mixed. Therefore, it is not necessary to gain the signal by image processing to separate the blue and green reflected light information. As a result, it is possible to acquire a white light image with good color reproduction while suppressing an increase in noise.

In the above aspect, the light source device may include a fourth solid-state lighting element that emits light having a peak intensity in a wavelength region of 580 nm to 700 nm.
By doing so, the information of the red reflected light is acquired with an appropriate intensity by the orange to red light emitted by the fourth solid-state lighting element. As a result, it is possible to obtain a white light image with reduced noise by suppressing the gain applied to the signal of the red reflected light for adjusting the white balance of the white light image.

In the above aspect, the color filter array may include a red color filter.
By doing so, the color reproducibility of the white light image can be improved by detecting the red reflected light from the subject by the pixels corresponding to the red color filter.

In the above aspect, the color filter array may include a magenta color filter.
By doing so, the purple to blue reflected light is detected in addition to the red reflected light by the pixels corresponding to the magenta color filter. This makes it possible to increase the amount of information of the reflected light from purple to blue while improving the color reproducibility of the white light image, and further improve the resolution of the narrow band light image.

In the above aspect, the ratio of the number of the cyan color filters in the color filter array may be 1/4 or more.
By doing so, the number of Cy pixels becomes one-fourth or more of all the pixels of the image sensor. This makes it possible to improve the resolution of the capillaries on the surface layer in the narrow band optical image while maintaining the image quality of the white light image.

According to the present invention, there is an effect that a narrow band optical image and a white light image having good image quality can be acquired.

It is an overall block diagram of the endoscope apparatus which concerns on one Embodiment of this invention. It is a figure which shows the color filter array which the image sensor of the endoscope apparatus of FIG. 1 has. It is a figure which shows the spectral sensitivity characteristic of the image sensor which has the color filter array of FIG. In the white light observation mode of the endoscope device of FIG. 1, the spectral characteristics of the light output from the light source device during the (a) first period and (c) second period, and (b) the first period and (D) It is a figure which shows the spectroscopic characteristic included in the signal output from an image sensor in the 2nd period. It is a figure which shows (a) the spectral characteristic of the light output from a light source apparatus, and (b) the spectral characteristic contained in the signal output from an image sensor in the narrow band light observation mode of the endoscope apparatus of FIG. .. It is an overall block diagram of the 1st modification of the endoscope apparatus of FIG. In the white light observation mode of the endoscope device of FIG. 6, the spectral characteristics of the light output from the light source device during the (a) first period and (c) second period, and (b) the first period and (D) It is a figure which shows the spectroscopic characteristic included in the signal output from an image sensor in the 2nd period. It is a figure which shows the color filter array in the 2nd modification of the endoscope apparatus of FIG. It is a figure which shows the spectral sensitivity characteristic of the image sensor provided with the color filter array of FIG. In the white light observation mode of the endoscope device including the color filter array of FIG. 8, the spectral characteristics of the light output from the light source device during the (a) first period and (c) second period, and (b). It is a figure which shows the spectral characteristic contained in the signal output from the image pickup device in the 1st period and (d) the 2nd period. In the narrow band light observation mode of the endoscope device including the color filter array of FIG. 8, (a) the spectral characteristics of the light output from the light source device and (b) the spectral characteristics included in the signal output from the image sensor. It is a figure which shows.

The endoscope device 1 according to the embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the endoscope device 1 according to the present embodiment has an elongated scope 2 inserted into the body to irradiate a subject with illumination light and acquire an image signal, and a base end of the scope 2. It includes a drive control device 3 that is connected and supplies illumination light to the scope 2 and generates an image, and a monitor 4 that is connected to the drive control device 3.

The scope 2 includes an illumination optical system 5 that irradiates the subject with illumination light, and an imaging optical system 6 that receives reflected light of the illumination light from the subject and obtains an image signal of the subject.
The illumination optical system 5 is arranged in the scope 2 along the longitudinal direction and is ejected from a light guide member 7 that guides the illumination light from the base end to the tip end and a light guide member 7 provided at the tip end of the scope 2. It is provided with an illumination lens 8 that emits the illumination light toward the front of the tip of the scope 2.

The image pickup optical system 6 includes an objective lens 9 provided at the tip of the scope 2 for condensing reflected light from the subject, and an image pickup element 10 for capturing an image of the subject connected by the objective lens 9 and acquiring an image signal. And an analog-to-digital (AD) converter 11 that converts an image signal output from the image sensor 10 into a digital signal.

FIG. 2 shows a part of a color filter array 12 provided on the image pickup surface of the image pickup device 10 and composed of a large number of two-dimensionally arranged color filters. As shown in FIG. 2, the color filter array 12 is a so-called Bayer array in which the G filter is replaced with a Cy filter. Specifically, the color filter array 12 includes a red color filter (R filter), a cyan color filter (Cy filter), and a blue color filter (B filter). One unit array is composed of one squarely arranged R filter, one B filter, and two Cy filters, and the unit array is arranged in the row direction and the column direction.

Each color filter corresponds to one pixel of the image sensor 10. Hereinafter, the pixels corresponding to the R filter, the Cy filter and the B filter are referred to as R pixel, Cy pixel and B pixel, respectively. Therefore, the ratio of the number of Cy pixels to the total number of pixels of the image sensor 10 is halved. The R pixel, Cy pixel, and B pixel detect the light transmitted through the R filter, the Cy filter, and the B filter, respectively, and acquire the R signal, the Cy signal, and the B signal, respectively. The R signal, Cy signal, and B signal are transmitted to the image processing device 14 in the drive control device 3 via the AD converter 11.

FIG. 3 shows the spectral sensitivity characteristics of the R pixel, Cy pixel, and B pixel of the image sensor 10. The spectral sensitivity characteristics of the R pixel, Cy pixel, and B pixel are each obtained by multiplying the transmittance characteristics of the R filter, Cy filter, and B filter with the spectral sensitivity characteristics of the semiconductor serving as the light receiving portion. As shown in FIG. 3, since the R filter selectively transmits red light, the R pixel has high sensitivity in the red wavelength region (R region). Since the Cy filter selectively transmits green, blue and purple light, the Cy pixel has high sensitivity in the green, blue and purple wavelength regions (G region, B region, V region). Since the B filter selectively transmits blue and purple light, the B pixel has high sensitivity in the B region and the V region.

The drive control device 3 includes a light source device 13 that generates illumination light, and an image processing device 14 that processes an image signal received from the imaging optical system 6 to generate an image.
The light source device 13 includes a three-color LED 15V, 15B, 15G, a movable narrow band filter 16, a light source control unit (control unit) 17 that controls lighting and extinguishing of the LEDs 15V, 15B, 15G, and a narrow band filter 16. It is provided with a filter control unit 18 for controlling the movement of the light source.

The V-LED (first solid-state lighting element) 15V emits purple narrow band light (V light) having a peak intensity in the V region (for example, 380 nm to 430 nm). The B-LED (second solid-state lighting element) 15B emits blue narrow band light (B light) having a peak intensity in the B region (for example, 430 nm to 480 nm). The G-LED (third solid-state lighting element) 15G uses a phosphor to emit green broadband light (G light) having a peak intensity in the G region (for example, 480 nm to 580 nm). The G light has a wider spectral width than the V light and the B light, and also has an intensity in the R region.

The V light, B light, and G light emitted from the LEDs 15V, 15B, and 15G are combined with each other by the combined wave optical system 19 and incident on the condenser lens 20 along a single optical axis, and the condenser lens 20 Is focused on the base end surface of the light guide member 7. The combined wave optical system 19 is composed of, for example, a combination of a plurality of dichroic mirrors.

The narrow band filter 16 has a first position on the optical path of G light (see the solid line in FIG. 1) between the G-LED 15G and the combined wave optical system 19 and a second position outside the optical path of G light. (See the two-point chain line in FIG. 1) and is provided so as to be movable. The narrow band filter 16 transmits only light in a narrow wavelength region of green.

The light source control unit 17, the filter control unit 18, and the image processing device 14 operate for each of a white light observation mode for acquiring a white light image and an NBI observation mode (narrow band light observation mode) for acquiring an NBI image. It has become.

In the white light observation mode, the light source control unit 17 lights the V-LED15V, B-LED15B and G-LED15G, and the filter control unit 18 arranges the narrow band filter 16 at the second position. At this time, the light source control unit 17 has a first period in which only the V-LED15V and the B-LED15B are turned on as shown in FIG. 4A, and the G-LED15G as shown in FIG. 4C. The second period in which only the light source is turned on is repeated alternately.

Therefore, in the first period, as shown in FIG. 4A, only the V light Lv and the B light Lb are irradiated to the subject. Then, as shown in FIG. 4B, the reflected light of the V light Lv and the B light Lb is detected by the Cy pixel and the B pixel. In the second period, as shown in FIG. 4C, only G light Lg is applied to the subject. Then, as shown in FIG. 4D, among the reflected light of the G light Lg, the component in the G region is detected by the Cy pixel, and the component in the R region is detected by the R pixel.

In the white light observation mode, the image processing apparatus 14 assigns the Cy signal and the B signal acquired in the first period to the B channel, assigns the Cy signal acquired in the second period to the G channel, and assigns the second period. A white light image is generated by allocating the R signal acquired during the period to the R channel and further performing other processing. The Cy signal acquired in the first period may not be assigned to the B channel, and only the B signal may be assigned to the B channel.

In the NBI observation mode, the light source control unit 17 turns on the V-LED 15V and the G-LED 15G and turns off the B-LED 15B, and the filter control unit 18 arranges the narrow band filter 16 at the first position. At this time, the light source control unit 17 simultaneously lights the V-LED15V and the G-LED15G as shown in FIG. 5A.

Therefore, as shown in FIG. 5A, only the green narrowband light (G'light) Lg'passed through the V light Lv and the narrowband filter 16 is irradiated to the subject. Then, as shown in FIG. 5B, the reflected light of the V light Lv is detected by the B pixel and the Cy pixel, and the reflected light of the G'light Lg'is detected by the Cy pixel. That is, the Cy pixel simultaneously detects the reflected light of both the V light Lv and the G'light Lg', and acquires the Cy signal including the information of the reflected light of both the V light Lv and the G'light Lg'.

In the NBI observation mode, the image processing apparatus 14 assigns the B information predicted from the B signal and the Cy signal to the B channel and the G channel, assigns the difference obtained by subtracting the B information from the Cy signal to the R channel, and further assigns the difference. By adding processing, an NBI image is generated.

As described above, the Cy signal includes information on reflected light of both V light Lv and G'light Lg'. Further, the B signal includes information on both V light Lv and G'light Lg', but since the sensitivity of the B pixel in the G region is sufficiently lower than the sensitivity in the V region, information on the reflected light of V light Lv. Can be obtained with high purity. Therefore, by predicting the B information from the Cy signal and the B signal and subtracting the B information from the Cy signal, the information of the reflected light of the G'light Lg'can be extracted with high accuracy.

The image processing device 14 transmits the generated white light image or NBI image to the monitor 4.
The monitor 4 displays the received white light image or NBI image.

Whether to perform white light observation or NBI observation is determined by the user operating the observation mode changeover switch 21 provided on the scope 2. The observation mode changeover switch 21 allows the user to select one of the white light observation mode and the NBI observation mode. Information on the selected mode is transmitted from the observation mode changeover switch 21 to the light source control unit 17, the filter control unit 18, and the image processing device 14. The light source control unit 17 and the filter control unit 18 execute the above-described control according to the received mode information, and the image processing device 14 generates a white light image or an NBI image according to the received mode information.

Next, the operation of the endoscope device 1 configured in this way will be described.
When the user selects the white light observation mode with the observation mode changeover switch 21, the light source control unit 17 and the filter control unit 18 control the LEDs 15V, 15B, 15G and the narrow band filter 16 in the white light observation mode to obtain V light and B light and G light are alternately applied to the subject. Then, the reflected light of V light and B light and the reflected light of G light are alternately detected by the image pickup device 10, and the B signal and Cy signal including the information of the reflected light of V light and B light and the reflection of G light are reflected. The Cy signal and the R signal including the light information are alternately transmitted to the image processing device 14. In the image processing apparatus 14, the B signal and the Cy signal of the first period are assigned to the B channel, the Cy signal and the R signal of the second period are assigned to the G channel and the R channel, respectively, and other image processing is added. As a result, a white light image is generated and displayed on the monitor 4.

On the other hand, when the user selects the NBI observation mode with the observation mode changeover switch 21, the light source control unit 17 and the filter control unit 18 control the LEDs 15V, 15B, 15G and the narrow band filter 16 in the NBI observation mode to obtain V light and G'light is applied to the subject at the same time. Then, the reflected light of V light and G'light is detected by the image pickup device 10, and the Cy signal including the information of the reflected light of V light and G'light and the B signal including the information of the reflected light of V light Lv are obtained. It is transmitted to the image processing device 14. In the image processing device 14, the B information predicted from the B signal and the Cy signal is assigned to the B channel and the G channel, the G information obtained by subtracting the B information from the Cy signal is assigned to the R channel, and other images. By adding the processing, an NBI image is generated and displayed on the monitor 4.

As described above, according to the present embodiment, by providing the image sensor 10 with a Cy filter that transmits light in the G, B, and V regions, the reflected light of V light including information on the capillaries on the surface layer of the living tissue is generated. , It is also detected by the Cy pixel in addition to the B pixel. Therefore, the information of the reflected light of the V light Lv used for generating the NBI image is increased. This has the advantage that a high resolution NBI image can be acquired.

Further, when observing white light, a white light image having high color reproducibility can be obtained by using a wide band G light Lg having intensity in the R region and detecting the reflected light in the R region by the R pixel. There is an advantage. Further, the first period in which only the V-LED15V and the B-LED15B are turned on and the second period in which only the G-LED15G is turned on are alternately repeated, and the V light, the B light, and the G light are subjected to a surface-sequential method. By irradiating the subject, the Cy pixel detects the reflected light in the V region, the reflected light in the B region, and the reflected light in the G region in a time-divided manner. This has the advantage that the information in the B and V regions and the information in the G region can be separated without applying a gain to the image signal, and a white light image with reduced noise can be obtained.

If the V-LED15V, the B-LED15B, and the G-LED15G are turned on at the same time, since the B pixel also has sensitivity in the G region, the B signal includes a lot of information on the reflected light of the G light Lg. Therefore, the accuracy of separating the information in the V and B regions and the information in the G region from the signal of the Cy pixel is lowered, and the color reproducibility is lowered. Further, when the information in the V and B regions and the information in the G region are separated, it becomes necessary to apply a high gain to the image signal, resulting in a noisy white light image.

In the present embodiment, the color filter array 12 has a unit array consisting of four squarely arranged color filters, but the arrangement pattern of the color filters of the color filter array 12 is not limited to this. ..
For example, in a 2 × 2 unit array, the number of Cy filters may be one and the number of R filters or B filters may be two. Alternatively, the unit array may be composed of 9 color filters of 3 × 3 or 16 color filters of 4 × 4. In any of the array patterns, the ratio of the number of Cy filters to the total number of color filters in the color filter array is preferably 1/4 or more.

In the present embodiment, the case of acquiring an NBI image as a narrow band optical image has been described, but a narrow band optical image other than the NBI image, for example, a BLI image may be acquired.

Next, a modified example of this embodiment will be described.
In the endoscope device 100 according to the first modification of the present embodiment, the light source device 131 further includes an R-LED (fourth solid-state lighting element) 15R as shown in FIG. The R-LED15R emits red narrow band light (R light) having a peak intensity in the orange to red wavelength region (for example, 580 nm to 700 nm).
In this modification, the control by the light source control unit 17 in the NBI observation mode is the same as the above-described embodiment, but the control by the light source control unit 17 in the white light observation mode is different from the above-described embodiment.

In the white light observation mode, the light source control unit 17 lights only the V-LED15V and the B-LED15B during the first period as shown in FIG. 7A, and the light source control unit 17 turns on only the V-LED15V and the B-LED15B as shown in FIG. Only G-LED15G and R-LED15R are turned on during the period of 2. In the first period, as shown in FIG. 7B, the B signal and Cy signal assigned to the B channel are acquired. The Cy signal acquired in the first period may not be assigned to the B channel, and only the B signal may be assigned to the B channel. In the second period, as shown in FIG. 7D, the Cy signal assigned to the G channel and the larger R signal assigned to the R channel are acquired.
In this way, since the R signal has an appropriate magnitude with respect to the signals assigned to the B channel and the G channel, the gain applied to the R signal for adjusting the white balance can be suppressed, and the white light image can be displayed. The noise can be further reduced.

In the endoscope apparatus according to the second modification of the present embodiment, the color filter array 121 has a magenta color filter (Mg filter) instead of the R filter, as shown in FIG. FIG. 9 shows the spectral sensitivity characteristics of the Mg pixel, Cy pixel, and B pixel of the image sensor 10. The Mg pixel is a pixel corresponding to the Mg filter. The Mg filter has two transmission bands in the B and V regions and the R region, and selectively transmits blue and purple light and red light.

In this modification, the light source device 131 includes the four LEDs 15V, 15B, 15G, and 15R described above. The control of the LEDs 15V, 15B, 15G, and 15R by the light source control unit 17 is the same as the first modification as shown in FIGS. 10 (a), 10 (c) and 11 (a).

In the generation of the white light image, the B signal, Cy signal, and Mg signal acquired in the first period are assigned to the B channel. The Cy signal and Mg signal acquired in the first period may not be assigned to the B channel, and only the B signal may be assigned to the B channel. Further, the Cy signal acquired in the second period is assigned to the G channel, and the Mg signal acquired in the second period is assigned to the R channel.
On the other hand, in the generation of the NBI image, the Mg signal is assigned to the B channel and the G channel in addition to the B signal and the Cy signal. In this way, the reflected light of V light Lv is further detected by the Mg pixel, and the amount of information of the reflected light of V light Lv in the NBI image is further increased, so that a higher resolution NBI image of the capillaries is acquired. be able to.

In this modification, the light source device 13 that does not include the R-LED 15R may be used instead of the light source device 131. In this case as well, the resolution of the NBI image can be improved.

1,100 Endoscope device 10 Image sensor 12,121 Color filter array 13,131 Light source device 15V V-LED (first solid-state lighting element)
15BB-LED (second solid-state lighting element)
15G G-LED (third solid-state lighting element)
15RR-LED (4th solid-state lighting element)
17 Light source control unit (control unit)

Claims (5)

A light source device that outputs illumination light to illuminate the subject,
It has a color filter array including color filters of a plurality of colors arranged two-dimensionally, and includes an image pickup element for photographing the subject illuminated by the illumination light.
The light source device
A first solid-state lighting element that emits light with peak intensity in the purple wavelength region,
A second solid-state lighting element that emits light with peak intensity in the blue wavelength region,
A third solid-state lighting element that emits light with peak intensity in the green wavelength region,
A control unit for controlling lighting and extinguishing of the first solid-state lighting element, the second solid-state lighting element, and the third solid-state lighting element is provided.
The color filter array includes a cyan color filter and a blue color filter, and the cyan color filter has an optical property of transmitting green, blue and purple light.
The control unit
In the first observation mode, the first solid-state lighting element and the third solid-state lighting element are turned on at the same time , and the second solid-state lighting element is turned off .
In the second observation mode, an endoscope device that turns on the second solid-state lighting element and the third solid-state lighting element in order. The endoscope device according to claim 1, wherein the light source device includes a fourth solid-state lighting element that emits light having a peak intensity in a wavelength region of 580 nm to 700 nm. The endoscope device according to claim 1 or 2, wherein the color filter array includes a red color filter. The endoscope device according to claim 1 or 2, wherein the color filter array includes a magenta color filter. The endoscope device according to any one of claims 1 to 4, wherein the ratio of the number of the cyan color filters in the color filter array is one-fourth or more.

Images