WO2020026323A1

WO2020026323A1 - Endoscope device, and endoscope device operating method and program

Info

Publication number: WO2020026323A1
Application number: PCT/JP2018/028542
Authority: WO
Inventors: 順平高橋; 恵仁森田
Original assignee: オリンパス株式会社
Priority date: 2018-07-31
Filing date: 2018-07-31
Publication date: 2020-02-06

Abstract

This endoscope device includes a light source 100, an imaging element 240, and a processing circuit 310. The imaging element 240 has a color filter. The processing circuit 310 selects a signal from first narrow-band pixels having sensitivity to a first narrow-band light, and a signal from second narrow-band pixels having sensitivity to second narrow-band light. Of a plurality of filter units in the color filter, half or more comprise filter units transmitting the first narrow-band light, and half or more comprise filter units transmitting the second narrow-band light. Then, the processing circuit 310 executes first interpolation processing wherein the signal from the first narrow-band pixels is processed for interpolation without using the signal from the second narrow-band pixels, and second interpolation processing wherein the signal from the second narrow-band pixels is processed for interpolation without using the signal from the first narrow-band pixels.

Description

Endoscope apparatus, method of operating endoscope apparatus, and program

The present invention relates to an endoscope apparatus, an operation method of the endoscope apparatus, a program, and the like.

内 In medical and industrial fields, endoscope devices are widely used for various examinations. Among them, in the medical field, by inserting a flexible insertion portion having an elongated shape, an in-vivo image in a body cavity can be acquired without incising the subject, thereby reducing the burden on the subject. For this reason, endoscope devices have been widely used. An imaging device having a plurality of pixels is provided at the tip of the insertion section, and an in-vivo image in a body cavity is taken by the imaging device.

As an observation mode of such an endoscope apparatus, a white light observation (WLI: White Light Imaging) mode using white illumination light and a narrow band light observation mode using narrow band illumination light are known. An example of the narrow-band light observation mode is an NBI (Narrow Band Imaging) mode. In the NBI mode, illumination light including narrow-band light included in the wavelength band of blue light and narrow-band light included in the wavelength band of green light is used.

On the other hand, a simultaneous single-chip image sensor is generally used as an image sensor for acquiring an image. A plurality of color filters having different colors are arranged in the simultaneous single-chip image sensor. In an image picked up by the image pickup device, only one color is obtained for one pixel, and other colors are lost. Therefore, the interpolation process is an essential process.

The interpolation processing is, for example, linear interpolation processing or bicubic interpolation processing, and interpolates missing pixels. Therefore, the resolution is reduced by the interpolation processing. In addition, when an image sensor having a complementary color filter is used, a pixel signal in which RGB color signals are mixed is obtained, so that color separation between RGB is reduced.

手法 A method for improving the sense of resolution when a simultaneous imaging device is used is disclosed in, for example, Patent Document 1. In Patent Literature 1, complementary color filters for yellow and cyan and two green primary color filters have a basic array of 2 rows and 2 columns, and the basic array is repeatedly arranged, and two green primary color filters are provided in the basic array. Are arranged side by side in an oblique direction. Then, by performing direction discrimination-type interpolation processing on the G signal obtained from the image provided with the green primary color filter, the resolution is improved.

JP 2003-087804 A

では In the white light observation mode, the G signal is input to the G channel of the display image. According to the technique of Patent Document 1, it is possible to improve the resolution of a G signal in which a blood vessel or a gland duct structure of a living body is clearly depicted. However, in the narrow band light observation mode, the G signal is not always input to the G channel of the display image. For example, in the NBI mode, a B signal is input to a G channel of a display image. For this reason, in the technique of Patent Document 1 in which only the resolution of the G signal is improved, improvement in resolution or color separation in the narrow-band light observation mode cannot be expected. In the NBI mode, a change in color tone such as a blood vessel running pattern on the mucosal surface layer or a brownish area is observed. For example, when distinguishing between cancer and inflammation, the contrast or color change of blood vessels is observed. For example, in bladder observation, when distinguishing an inflammatory lesion from a flat cancer (CIS: Cancer In Situ), an improvement in resolution or color separation is desired.

According to some aspects of the invention, an endoscope apparatus and an endoscope apparatus capable of improving resolution or color separation in a narrow-band light observation mode while maintaining image quality in a white light observation mode. An operation method, a program, and the like can be provided.

One embodiment of the present invention includes a light source that emits narrow-band illumination light having first narrow-band light and second narrow-band light, and a color filter in which each filter unit is provided for each of a plurality of pixels. An image sensor that photoelectrically converts light received by the plurality of pixels, a first narrow band pixel signal having sensitivity to the first narrow band light among the pixel signals from the image sensor, A processing circuit for selecting a second narrow band pixel signal having sensitivity to the band light, wherein the color filter has a filter unit that transmits the first narrow band light at least half of the total number, and (2) a first interpolation process for interpolating the first narrowband pixel signal without using the second narrowband pixel signal, wherein the processing circuit has at least half of the total number of filter units transmitting the narrowband light; And the second narrow band image Signal, related to an endoscope apparatus which performs a second interpolation process of interpolating processing without using the first narrowband pixel signal.

According to another aspect of the present invention, there is provided a light source that emits narrow-band illumination light having first narrow-band light and second narrow-band light, and a color filter in which each filter unit is provided for each of a plurality of pixels. And an image sensor having the filter, wherein the color filter has at least half of filter units that transmit the first narrow band light, and transmits the second narrow band light. When the number of filter units is more than half, among the pixel signals from the image sensor, a first narrowband pixel signal having sensitivity to the first narrowband light and a second narrowband pixel signal having sensitivity to the second narrowband light are provided. A first interpolation process of selecting the first narrowband pixel signal without using the second narrowband pixel signal, and a first interpolation process of interpolating the first narrowband pixel signal without using the second narrowband pixel signal. Compensation without using band pixel signal Relating to the operation method of the endoscope apparatus which performs a second interpolation process that sense.

Another aspect of the present invention is a program for processing an image captured by an image sensor having a color filter provided with each filter unit for each of a plurality of pixels, wherein the color filter has a narrow band illumination. When more than half the filter units that transmit the first narrowband light of light and more than half the filter units that transmit the second narrowband light of the narrowband illumination light, the pixel signal from the image sensor is The first narrowband pixel signal having sensitivity to the first narrowband light and the second narrowband pixel signal having sensitivity to the second narrowband light are selected, and the first narrowband pixel signal is A first interpolation process for interpolating without using the second narrowband pixel signal and a second interpolation process for interpolating the second narrowband pixel signal without using the first narrowband pixel signal are performed. Steps And a program causing a computer to execute.

3 illustrates a configuration example of an endoscope apparatus. FIG. 4 is a diagram illustrating an interlaced imaging operation using a complementary color image sensor. FIG. 4 is a diagram illustrating a process performed by a pixel signal selection unit. 4 illustrates an example of sensitivity of a pixel signal. An example of a color filter that is not a complementary color filter. FIG. 4 is a diagram illustrating a process performed by an interpolation processing unit. FIG. 4 is a diagram illustrating color mixing in pixel signals. FIG. 4 is a diagram illustrating color mixing in pixel signals. FIG. 9 is a diagram illustrating a second example of the direction discrimination type interpolation processing. FIG. 9 is a diagram illustrating a second example of the direction discrimination type interpolation processing. FIG. 9 is a diagram illustrating a second example of the direction discrimination type interpolation processing.

Hereinafter, the present embodiment will be described. The present embodiment described below does not unduly limit the contents of the present invention described in the claims. Further, all of the configurations described in the present embodiment are not necessarily essential components of the invention.

1. Endoscope device

FIG. 1 is a configuration example of an endoscope device. The endoscope device includes an insertion unit 200, a control device 300, a display unit 400, an external I / F unit 500, and a light source 100. As the endoscope device, for example, a flexible endoscope used for a digestive tract and the like, and a rigid endoscope used for a laparoscope and the like can be assumed. Note that the insertion section 200 is also called a scope. Control device 300 is also called a main unit or a processing device. The display unit 400 is also called a display device. The external I / F unit 500 is also called an operation unit or an operation device. The light source 100 is also called a lighting unit or a lighting device.

The endoscope device has a white light observation mode and a narrow band light observation mode. Hereinafter, a case where the narrow-band light observation mode is the NBI mode will be described as an example. However, the narrow-band light observation mode is not limited to the NBI mode, and may be any observation mode that uses a plurality of narrow-band lights having different colors.

The insertion part 200 is a part to be inserted into the body. The insertion section 200 includes a light guide 210, an illumination lens 220, an objective lens 230, an image sensor 240, and an A / D conversion circuit 250. Further, the insertion section 200 can include a memory 260. The image sensor 240 is also called an image sensor. The insertion section 200 has a connector (not shown), and is attached to and detached from the control device 300 by the connector.

The light source 100 includes a white light source 110, a filter 130, and a lens 120. The white light source 110 is, for example, an LED (Light Emitting Diode) or a xenon lamp, and emits white illumination light. In the white light observation mode, the filter 130 is removed from between the white light source 110 and the lens 120, and white illumination light enters the lens 120. In the NBI mode, the filter 130 is inserted between the white light source 110 and the lens 120, the white illumination light passes through the filter 130, and the light passing through the filter 130 enters the lens 120. The lens 120 condenses the light that has entered the lens 120 and causes the light to enter the light guide 210.

The filter 130 passes a first narrow band centered on the wavelength 410 nm and a second narrow band centered on the wavelength 540 nm. The first narrow band is, for example, 390 nm to 445 nm and belongs to the blue wavelength band of white light. The second narrow band is, for example, 530 nm to 550 nm and belongs to the green wavelength band of white light. Hereinafter, the first narrowband light that has passed through the filter 130 is referred to as B narrowband light, and the second narrowband light that has passed through the filter 130 is referred to as G narrowband light. These narrow-band lights have characteristics that are easily absorbed by hemoglobin in blood.

The configuration of the light source 100 is not limited to this. For example, the light source 100 includes a white light source and a narrow band light source, the white light source emits white light in the white light observation mode, and the narrow band light source emits the B narrow band light and the G narrow band light in the NBI observation mode. Is also good. In this case, the filter 130 is omitted.

Light guide 210 guides the illumination light from light source 100 to the tip of insertion section 200. The illumination lens 220 irradiates the subject with the illumination light guided by the light guide 210. In the present embodiment, the subject is a living body. Light reflected from the subject enters the objective lens 230. A subject image is formed by the objective lens 230, and the imaging element 240 captures the subject image.

The image sensor 240 includes a plurality of pixels for subjecting a subject image to photoelectric conversion, and acquires a pixel signal from the plurality of pixels. The image sensor 240 may read out pixel signals for each pixel, or may obtain pixel signals by adding and reading out from a plurality of pixels. The image sensor 240 is a simultaneous single-chip image sensor that can obtain pixel signals of a plurality of colors by one image pickup. The image sensor 240 is, for example, a complementary color image sensor, but the image sensor 240 is not limited to a complementary image sensor.

The A / D conversion circuit 250 A / D converts an analog pixel signal from the image sensor 240 into a digital pixel signal. The A / D conversion circuit 250 may be built in the image sensor 240.

(4) The control device 300 performs signal processing including image processing. The control device 300 controls each part of the endoscope device. The control device 300 includes a processing circuit 310 and a control circuit 320. Each of the processing circuit 310 and the control circuit 320 is realized by, for example, an integrated circuit device or the like. The control circuit 320 may be configured as a separate circuit from the processing circuit 310, or may be included in the processing circuit 310 as a control unit. The processing circuit 310 may be constituted by, for example, a processor described later.

The control circuit 320 controls each section of the endoscope device. For example, the user operates the external I / F unit 500 to set the observation mode. When the white light observation mode is set, the control circuit 320 causes the light source 100 to emit white light, and causes the processing circuit 310 to execute image processing in the white light observation mode. When the NBI mode is set, the control circuit 320 causes the light source 100 to emit B narrowband light and G narrowband light, and causes the processing circuit 310 to execute image processing in the NBI mode.

The memory 260 of the insertion unit 200 stores information on the insertion unit 200. The control circuit 320 controls each unit of the endoscope apparatus based on the information read from the memory 260. For example, the memory 260 stores information about the image sensor 240. The information on the image sensor 240 is, for example, information on the arrangement of color filters. The control circuit 320 causes the processing circuit 310 to perform corresponding image processing based on the information on the image sensor 240 read from the memory 260.

The processing circuit 310 generates a display image by performing image processing based on the pixel signal from the A / D conversion circuit 250, and outputs the display image to the display unit 400. The display unit 400 is, for example, a liquid crystal display device or the like, and displays a display image from the processing circuit 310. The processing circuit 310 includes a pixel signal selection unit 311 and an interpolation processing unit 312.

In the white light observation mode, the interpolation processing unit 312 generates a white light image by performing an interpolation process on the pixel signal, and outputs the white light image as a display image. In the NBI mode, the pixel signal selection unit 311 selects a pixel signal of a part of the plurality of pixel signals as a luminance component pixel signal, and selects a remaining color pixel signal as a non-luminance component pixel signal. The luminance component pixel signal is a pixel signal having sensitivity to the B narrow band light. The non-luminance component pixel signal is a pixel signal having sensitivity to G narrow band light. The interpolation processing unit 312 generates a luminance component image by performing a first interpolation process on the luminance component pixel signal, and generates a non-luminance component image by performing a second interpolation process on the non-luminance component pixel signal. The processing circuit 310 independently performs the first interpolation process and the second interpolation process as separate processes. The first interpolation processing and the second interpolation processing may be the same algorithm, but are independent as processing. For example, when performing direction discrimination-type interpolation processing, the processing circuit 310 performs direction discrimination based on the luminance component pixel signal in the first interpolation processing, and interpolates the luminance component pixel signal based on the discrimination result. In the first interpolation processing, a non-luminance component pixel signal is not used. In addition, the processing circuit 310 performs the direction determination based on the non-luminance component pixel signal in the second interpolation processing, and interpolates the non-luminance component pixel signal based on the determination result. In the second interpolation processing, a luminance component pixel signal is not used. The processing circuit 310 inputs the luminance component image to the G channel and the B channel of the display image, and inputs the non-luminance component image to the R channel of the display image. Thus, the NBI image is output as a display image. The display image may be either a moving image or a still image.

The external I / F unit 500 is an interface for performing input from the user to the endoscope apparatus and the like. That is, it is an interface for operating the endoscope apparatus, an interface for setting operation of the endoscope apparatus, or the like. For example, a button, a dial, a lever, and the like for selecting an observation mode are included.

Hereinafter, operation and processing of the endoscope apparatus according to the present embodiment will be described. The details will be described later. Here, the embodiments will be described with reference to the following embodiments as appropriate.

The endoscope apparatus includes the light source 100, the image sensor 240, and the processing circuit 310. The light source 100 emits narrow-band illumination light having first narrow-band light and second narrow-band light. The image sensor 240 has a color filter in which each filter unit is provided for each of the plurality of pixels, and photoelectrically converts light received by the plurality of pixels. The processing circuit 310 selects a first narrowband pixel signal having sensitivity to the first narrowband light and a second narrowband pixel signal having sensitivity to the second narrowband light among the pixel signals from the image sensor 240. I do. The color filter has at least half of the total number of filter units transmitting the first narrowband light, and at least half of the total number of filter units transmitting the second narrowband light. Then, the processing circuit 310 performs a first interpolation process of interpolating the first narrowband pixel signal without using the second narrowband pixel signal, and uses the first narrowband pixel signal to perform the second narrowband pixel signal. And a second interpolation process for performing the interpolation process.

When the narrow-band light observation mode is the NBI mode, the first narrow-band light is B narrow-band light and the second narrow-band light is G narrow-band light. As described with reference to FIGS. 3 and 4, the processing circuit 310 selects the B narrowband signal as the first narrowband pixel signal, and selects the B narrowband signal as the second narrowband pixel signal. As described with reference to FIG. 2, when the color filter is a complementary color filter, the Mg and Cy filter units transmit B narrow band light, and the Mg, Cy, Ye and G filter units transmit G narrow band light. In each case, it is more than half of the whole. As described with reference to FIG. 6, the processing circuit 310 independently performs the first interpolation process of interpolating the B narrowband signal and the second interpolation process of interpolating the G narrowband signal. In the NBI mode, since the G channel of the display image is generated based on the B narrow band signal, the B narrow band signal is a luminance component pixel signal. However, the first narrowband signal is not limited to a luminance component pixel signal. For example, a luminance component of a display image may be generated based on both the first narrowband pixel signal and the second narrowband pixel signal. Further, the narrow-band light observation mode is not limited to the NBI mode. The narrow-band observation mode may be any mode in which illumination light including a plurality of narrow-band lights is used.

As described in the following equations (1) to (4), conventionally, an image captured by a complementary color image sensor is subjected to interpolation processing by pixel addition or the like, and the image resolution or color separation may be reduced. . On the other hand, in the narrow band light observation mode, it is desirable that the contrast of the blood vessel is high. The contrast of a blood vessel is the resolution or color separation of the blood vessel. According to the present embodiment, the first interpolation processing for interpolating the first narrowband pixel signal and the second interpolation processing for interpolating the second narrowband pixel signal are performed independently. That is, an image obtained by each narrow-band illumination light is interpolated independently without adding pixels. Since pixel addition or the like is unnecessary, the contrast of blood vessels can be improved. For example, it is expected that CIS and inflammation can be easily distinguished in bladder observation. Further, in the above-mentioned Patent Document 1, only the G signal has a high resolution. On the other hand, according to the present embodiment, both of the two narrow-band images have high resolution, so that high-resolution imaging can be realized in the narrow-band light observation mode.

According to the present embodiment, more than half of the filter units that transmit the first narrow-band light are provided in the image sensor 240, and more than half of the filter units that transmit the second narrow-band light are provided. Then, a first narrowband pixel signal having sensitivity to the first narrowband light and a second narrowband pixel signal having sensitivity to the second narrowband light are selected. As a result, of all the pixels forming one image, half of the pixels are pixels of the first narrow band, and the remaining half are pixels of the second narrow band. By performing the first interpolation processing and the second interpolation processing on such an image, an image having a high resolution is obtained.

Further, as described in the following equation (5) and the first modification, the processing circuit 310 includes a first narrowband image obtained by the first interpolation process and a second narrowband image obtained by the second interpolation process. , A color separation process between the first narrowband image and the second narrowband image is performed.

As described with reference to FIGS. 7 and 8, the first narrowband pixel signal is also sensitive to the second narrowband light, and the second narrowband pixel signal is also sensitive to the first narrowband light. That is, the first narrowband pixel signal and the second narrowband pixel signal have a mixed color. According to the present embodiment, since the color separation process is performed between the first narrowband image and the second narrowband image, the color mixture can be reduced. By reducing the color mixture, the color separation in the narrow-band light observation mode can be improved.

Further, as described in the following equations (12) and (13) and the third modification, the processing circuit 310 determines the direction based on the brightness of the first narrowband pixel signal and the brightness of the second narrowband pixel signal. An index value RLB (x, y) indicating the reliability of the direction discrimination in the discrimination type interpolation processing is obtained. The processing circuit 310 performs a direction determination type interpolation process based on the index value RLB (x, y) in the first interpolation process and the second interpolation process.

Specifically, the index value RLB (x, y) indicates the degree of variation in the brightness of the first narrowband pixel signal and the second narrowband pixel signal around the position (x, y). When the degree of variation in the brightness of the first narrowband pixel signal and the second narrowband pixel signal is large, the reliability of the edge direction determination decreases. According to the present embodiment, the processing circuit 310 performs the interpolation processing such that the lower the reliability, the smaller the weight of the interpolation processing result in the edge direction. This can reduce the occurrence of artifacts due to the interpolation processing.

Further, in the present embodiment, in the direction discrimination type interpolation processing, the processing circuit 310 compares the edge direction interpolation processing result and the non-direction discrimination type interpolation processing result with a blend ratio based on the index value RLB (x, y). Blend with. The processing circuit 310 increases the blend ratio of the non-direction discrimination type interpolation processing result as the reliability is lower.

において In the following equation (13), the blend ratio of the result of the non-direction discrimination-type interpolation processing is (1-RLB (x, y)).

According to this embodiment, the lower the reliability, the higher the blending ratio of the non-direction discrimination type interpolation processing result. As a result, the greater the degree of variation in brightness between the first narrowband pixel signal and the second narrowband pixel signal, the greater the blending rate of the non-directional discrimination type interpolation processing result, and the higher the blending rate of the direction discrimination type interpolation processing. Can be reduced.

Further, as described by the following equation (12), the processing circuit 310 calculates the index value RLB (x, y) based on the ratio between the brightness of the first narrowband pixel signal and the brightness of the second narrowband pixel signal. Ask.

比 The ratio of the brightness of the first narrowband pixel signal to the brightness of the second narrowband pixel signal changes according to the degree of variation in the brightness of the first narrowband pixel signal and the second narrowband pixel signal. This makes it possible to control the blending ratio of the non-direction discrimination-type interpolation processing according to the degree of variation in brightness between the first narrowband pixel signal and the second narrowband pixel signal.

As described in FIGS. 7, 8 and the fourth modification, the first narrow-band pixel signal is composed of the first pixel signal and the second pixel signal having higher sensitivity to the first narrow-band light than the first pixel signal. And The second narrowband pixel signal includes a third pixel signal and a fourth pixel signal having a higher sensitivity to the second narrowband light than the third pixel signal. The processing circuit 310 makes the weight of the second pixel signal larger than the weight of the first pixel signal in the first interpolation processing, and makes the weight of the fourth pixel signal larger than the weight of the third pixel signal in the second interpolation processing. Enlarge.

7 and 8, the first pixel signal is an MgYe signal, and the second pixel signal is an MgCy signal. The third pixel signal is a GCy signal, and the fourth pixel signal is a GYe signal.

According to the present embodiment, when the first narrowband image is subjected to the interpolation processing, the weight of the pixel signal having high sensitivity to the first narrowband light is increased, thereby realizing the first interpolation processing with higher resolution. it can. In addition, when performing interpolation processing on the second narrowband image, by increasing the weight of a pixel signal having high sensitivity to the second narrowband light, it is possible to realize second interpolation processing with higher resolution. Therefore, the sense of resolution is improved for the display image obtained by combining these.

Also, in the present embodiment, the first narrowband pixel signal includes a first pixel signal and a second pixel signal corresponding to different colors. The second narrowband pixel signal includes a third pixel signal and a fourth pixel signal corresponding to different colors. In the first interpolation processing, the processing circuit 310 performs the first interpolation based on the average value of the first pixel signal in the peripheral pixels of the first target pixel and the average value of the second pixel signal in the peripheral pixels of the first target pixel. The brightness correction between the pixel signal and the second pixel signal is performed. Further, in the second interpolation processing, the processing circuit 310 performs the second interpolation based on the average value of the third pixel signal in the peripheral pixels of the second target pixel and the average value of the fourth pixel signal in the peripheral pixels of the second target pixel. The brightness correction between the third pixel signal and the fourth pixel signal is performed.

ＢAs shown in FIG. 3, the B narrowband signal, which is the first narrowband pixel signal, includes the MgYe signal as the first pixel signal, and includes the MgCy signal as the second pixel signal. The G narrow-band signal, which is the second narrow-band pixel signal, includes the GCy signal as the third pixel signal, and includes the GYe signal as the fourth pixel signal. For example, the processing circuit 310 multiplies the MgYe signal by the average value of the MgCy signal / the average value of the MgYe signal, and multiplies the GCy signal by the average value of the GYe signal / the average value of the GCy signal. That is, the processing circuit 310 performs the brightness correction based on the average value of the peripheral same color pixels. This brightness correction is performed, for example, before the interpolation processing.

According to the present embodiment, in the first narrowband image, the first pixel signal and the second pixel signal are corrected so that the brightness levels match, and in the second narrowband image, the third pixel signal and the fourth pixel signal are corrected. Correction is performed so that the brightness level of the pixel signal matches. Thereby, the interpolation processing can be appropriately executed. For example, in the direction determination type interpolation processing, the degree of variation in brightness between signals is reduced, so that the accuracy of edge direction determination is improved.

As described with reference to FIG. 4, the processing circuit 310 selects a pixel signal whose sensitivity to the first narrowband light is equal to or higher than the sensitivity to the second narrowband light as the first narrowband pixel signal. A pixel signal whose sensitivity to the second narrowband light is greater than the sensitivity to the first narrowband light is selected as the second narrowband pixel signal.

According to the present embodiment, the first narrowband pixel signal and the second narrowband pixel signal are selected according to the sensitivity to the narrowband light, so that the pixel signals are converted into the first narrowband pixel signal and the second narrowband pixel. Can be separated into signals. As described above, by separating the pixel signal into the first narrowband pixel signal and the second narrowband pixel signal and performing independent interpolation processing on each pixel signal, a display image with high contrast can be obtained.

Further, in the present embodiment, the processing circuit 310 performs the first narrowband pixel signal and the second narrowband pixel signal on the basis of the narrowband pixel signal having a small color mixture among the first narrowband pixel signal and the second narrowband pixel signal. Adjust the brightness level between and.

For example, it is assumed that the color mixture is smaller in the second narrowband pixel signal. At this time, the processing circuit 310 multiplies the first narrowband pixel signal by the average of the second narrowband pixel signal / the average of the first narrowband pixel signal. This brightness level adjustment is performed, for example, before or after the interpolation processing.

(2) As described in FIG. 2, the color filter is a complementary color filter. Specifically, the color filter includes magenta, yellow, cyan, and green filter units.

As described above, in the related art, an image captured by a complementary color image sensor is subjected to interpolation processing by pixel addition or the like. On the other hand, according to the present embodiment, the first interpolation processing for interpolating the first narrowband pixel signal and the second interpolation processing for interpolating the second narrowband pixel signal in the narrowband light observation mode are independently performed. Done. This eliminates the need for pixel addition and the like, and thus can improve the contrast of blood vessels.

As described with reference to FIG. 5, the color filter may be a color filter in which a red filter unit is replaced with a magenta filter unit in a primary color Bayer array color filter.

The present invention can be applied to a case where an image sensor provided with such a color filter is used. Also in this case, in the narrow-band light observation mode, the first interpolation process of interpolating the first narrow-band pixel signal and the second interpolation process of interpolating the second narrow-band pixel signal are performed independently. The contrast of blood vessels can be improved.

Also, in the present embodiment, the processing circuit 310 generates a display image by inputting the first narrowband image obtained by the first interpolation process to the G channel of the display image.

In this case, the first narrowband image is a luminance component image. That is, the processing circuit 310 selects the first narrowband pixel signal as the luminance component pixel signal.

Also, in the present embodiment, the first narrowband light belongs to the blue wavelength band of white light, and has a wavelength band narrower than the blue wavelength band. The second narrowband light belongs to the green wavelength band of the white light, and has a wavelength band narrower than the green wavelength band. The processing circuit 310 inputs the first narrowband image to the G and B channels of the display image, and inputs the second narrowband image obtained by the second interpolation process to the R channel of the display image.

According to the present embodiment, the NBI mode can be realized as the narrow-band light observation mode. In the NBI mode, blood vessels near the surface layer of the mucous membrane can be photographed with high contrast. In addition, a lesion in which blood vessels are densely located near the surface of the mucous membrane can be observed as a brownish area. The brownish area is an area that looks brown. In the present embodiment, a blood vessel can be photographed with high contrast in the NBI mode. For example, although both CIS and inflammation appear in the brownish area, the blood vessels appear to have high contrast, so that it is expected that CIS and inflammation can be easily distinguished.

In the present embodiment, the light source 100 emits white illumination light in the white light observation mode, and emits narrow band illumination light in the narrow band light observation mode. The color filter of the image sensor 240 has a plurality of filter units for capturing a white light image when white illumination light is emitted. The number of filter units transmitting the first narrowband light is at least half of the plurality of filter units, and the number of filter units transmitting the second narrowband light is at least half of the plurality of filter units. In the narrow-band light observation mode, the processing circuit 310 selects the first narrow-band pixel signal and the second narrow-band pixel signal, and performs the first interpolation process and the second interpolation process.

(2) When the image sensor 240 is a complementary color image sensor, the complementary color filter has Mg, Cy, Ye, and G filter units as shown in FIG. These four filter units are for capturing a white light image. Specifically, a white light image is obtained by the interpolation processing of the following equations (1) to (4). At this time, the Mg and Cy filter units transmit the B narrow band light, and the Mg, Cy, Ye and G filter units transmit the G narrow band light. In each case, two or more of the four are half. In the related art, when capturing a white light image, pixel addition or the like is performed as in the following equations (1) to (4). According to the present embodiment, a high-contrast narrow-band light image can be obtained by independently interpolating the first narrow-band pixel signal and the second narrow-band pixel signal in the narrow-band light observation mode.

Note that the control device 300 of the present embodiment may be configured as follows. That is, the control device 300 of the present embodiment includes a memory that stores information, and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The processor includes hardware. When the narrow-band illumination light is emitted, the processor has a first narrow-band pixel signal having sensitivity to the first narrow-band light and a sensitivity to the second narrow-band light among the pixel signals from the image sensor 240. And a second narrowband pixel signal. The processor interpolates the first narrowband pixel signal without using the second narrowband pixel signal, and interpolates the second narrowband pixel signal without using the first narrowband pixel signal. And a second interpolation process.

In the processor, for example, the function of each unit may be realized by individual hardware, or the function of each unit may be realized by integrated hardware. For example, a processor includes hardware, and the hardware can include at least one of a circuit that processes digital signals and a circuit that processes analog signals. For example, the processor can be configured with one or a plurality of circuit devices mounted on a circuit board or one or a plurality of circuit elements. The one or more circuit devices are, for example, ICs. The one or more circuit elements are, for example, resistors, capacitors, and the like. The processor may be, for example, a CPU (Central Processing Unit). However, the processor is not limited to the CPU, and various processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) can be used. Further, the processor may be a hardware circuit based on an ASIC. Further, the processor may include an amplifier circuit and a filter circuit for processing an analog signal. The memory may be a semiconductor memory such as an SRAM or a DRAM, a register, a magnetic storage device such as a hard disk device, or an optical storage device such as an optical disk device. You may. For example, the memory stores a computer-readable instruction, and when the instruction is executed by the processor, the function of each unit of the control device 300 is realized as a process. The instruction here may be an instruction of an instruction set constituting a program or an instruction for instructing a hardware circuit of a processor to operate. For example, the processor realizes the function of the processing circuit 310 in FIG. Alternatively, the processor realizes the functions of the processing circuit 310 and the control circuit 320 in FIG.

Each part of the endoscope apparatus according to the present embodiment may be realized as a module of a program that operates on a processor. For example, the program may include a pixel signal selection module realizing the pixel signal selection unit 311 and an interpolation processing module realizing the interpolation processing unit 312. When the narrow-band illumination light is emitted, the pixel signal selection module outputs a first narrow-band pixel signal having sensitivity to the first narrow-band light and a second narrow-band light among the pixel signals from the image sensor 240. And a second narrowband pixel signal having sensitivity. The interpolation processing module performs a first interpolation process of interpolating the first narrowband pixel signal without using the second narrowband pixel signal, and a second interpolation process of using the second narrowband pixel signal without using the first narrowband pixel signal. And a second interpolation process for performing the interpolation process.

The program that implements the processing performed by each unit of the control device 300 of the present embodiment can be stored in, for example, an information storage medium that is a computer-readable medium. The information storage medium can be realized by, for example, an optical disk, a memory card, an HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing circuit 310 and the control circuit 320 perform various processes of the present embodiment based on programs and data stored in the information storage medium. That is, the information storage medium stores a program for causing a computer to function as each unit of the endoscope apparatus according to the present embodiment. The computer is a device including an input device, a processing unit, a storage unit, and an output unit. The program is a program for causing a computer to execute the processing of each unit. The program is recorded on an information storage medium. Here, as the information recording medium, various recording media readable by an optical detection system, such as an optical disk such as a DVD and a CD, a magneto-optical disk, a hard disk, and a memory such as a nonvolatile memory and a RAM, can be assumed.

{2. Operation and processing

Hereinafter, detailed operations and processes of the endoscope apparatus will be described.

FIG. 2 is a view for explaining an interlaced imaging operation using a complementary color image sensor. In FIG. 2, X indicates a horizontal scanning direction in the image sensor 240, and Y indicates a vertical scanning direction in the image sensor 240.

において In the interlace method, imaging of even-numbered fields and imaging of odd-numbered fields are alternately performed. A frame is composed of an even field and an odd field, and the processing circuit 310 generates a frame image from a pixel signal obtained in the even field and a pixel signal obtained in the odd field.

画素 As shown in FIG. 2, pixels are arranged in a grid on the image sensor 240, and one filter unit is provided for each pixel. Four color filter units of magenta, green, cyan, and yellow are arranged in two rows and two columns. In FIG. 2, Mg indicates magenta, G indicates green, Cy indicates cyan, and Ye indicates yellow. Hereinafter, colors will be described using these codes. Further, for example, a pixel provided with an Mg filter unit is referred to as an Mg pixel.

In the interlaced method, the image sensor 240 adds and reads pixel signals from two pixels arranged in the Y direction. That is, in the even field, the imaging element 240 performs addition reading from the Mg pixels in the first row and the Cy pixels in the second row to obtain the MgCy signal, and obtains the MgCy signal from the G pixels in the first row and the Ye pixels in the second row. The GYe signal is obtained by performing addition reading. A first horizontal line is configured by the MgCy signal and the GYe signal. Similarly, the image sensor 240 obtains the GCy signal and the MgYe signal constituting the third horizontal line by performing addition readout from the pixels in the third and fourth rows. In the odd field, the image sensor 240 reads out the GCy signal and the MgYe signal constituting the second horizontal line by performing addition reading from the pixels in the second and third rows. Further, the image sensor 240 reads out the MgCy signal and the GYe signal constituting the fourth horizontal line by performing addition reading from the pixels in the fourth and fifth rows. The first to fourth horizontal lines constitute an image of one frame.

First, the image generation processing in the white light observation mode will be described. The interpolation processing unit 312 converts a pixel signal into a YCrCb signal according to the following equations (1) to (4). In the following equations (1) to (4), for example, MgCy means the signal value of the MgCy pixel signal. The interpolation processing unit 312 generates a display image by interpolating the YCrCb signal to generate a YCrCb signal for all pixels, and converting the YCrCb signal to an RGB signal.

Next, an image generation process in the NBI mode will be described. FIG. 3 is a diagram illustrating a process performed by the pixel signal selection unit 311. In FIG. 3, x is a position in the horizontal scanning direction, and y is a position in the vertical scanning direction. The positions x and y mean pixel positions on the image when the pixel signals are arranged on the image. The positions x and y are each an integer. For example, a pixel in one row and one column is represented as (x, y) = (1, 1). Hereinafter, the horizontal scanning direction is also referred to as an x direction, and the vertical scanning direction is also referred to as a y direction.

Ｍｇ The MgCy signal, MgYe signal, GCy signal, and GYe signal acquired in FIG. 2 are arranged as shown in FIG. The pixel signal selection unit 311 selects the MgCy signal and the MgYe signal as the B narrowband signal, and selects the GCy signal and the GYe signal as the G narrowband signal. In the NBI mode, since the G channel of the display image is generated based on the B narrowband signal, the B narrowband signal is a luminance component pixel signal. Focusing on the area of 2 × 2 pixels, the B narrow band signal is arranged at (x, y) = (0, 1) and (1, 0), and the G narrow band signal is (x, y) = (0, 0). 0) and (1, 1). For example, a pixel in which a B narrowband signal is arranged is referred to as a B narrowband pixel. As shown in FIG. 3, each of the B narrowband pixel and the G narrowband pixel occupies half of the whole. In both the x direction and the y direction, B narrowband pixels and G narrowband pixels are alternately arranged.

FIG. 4 is an example of the sensitivity of a pixel signal. In FIG. 4, the sensitivity to the G narrow band light and the sensitivity to the B narrow band light are represented by relative numerical values.

(4) The pixel signal selection unit 311 selects the MgCy signal and the MgYe signal whose sensitivity to the B narrow band light is equal to or higher than the sensitivity to the G narrow band light as the B narrow band signal. In addition, the pixel signal selection unit 311 selects the GCy signal and the GYe signal whose sensitivity to the G narrow band light is equal to or higher than the sensitivity to the B narrow band light, as the G narrow band signal. As a result, a narrow-band B pixel and a narrow-band G pixel arranged as shown in FIG. 3 are obtained.

The color filter of the image sensor 240 is not limited to a complementary color filter. FIG. 5 shows an example of a color filter that is not a complementary color filter.

In FIG. 5, the Bayer array R filter is replaced with an Mg filter. That is, a G filter is arranged at (x, y) = (0, 0), (1, 1), a B filter is arranged at (x, y) = (1, 0), and (x, y) = An Mg filter is arranged at (0, 1). The 2 × 2 pixels are repeatedly arranged on the image sensor 240. The image sensor 240 reads out the G signal, the B signal, and the Mg signal from the G pixel, the B pixel, and the Mg pixel, respectively. The B filter and the Mg filter transmit the B narrow band light, and the G filter transmits the G narrow band light. For this reason, the pixel signal selection unit 311 selects the B signal and the Mg signal as the B narrowband signal, and selects the G signal as the G narrowband signal. Thereby, the arrangement of the B narrow band pixel and the G narrow band pixel is the same as that of the complementary color filter. The following interpolation processing can be applied in the same manner as in the case of the complementary color filter.

FIG. 6 is a diagram illustrating the processing performed by the interpolation processing unit 312. The interpolation processing unit 312 independently performs interpolation processing on the B narrowband signal and the G narrowband signal.

The pixel signal input from the pixel signal selection unit 311 to the interpolation processing unit 312 is a pixel signal of one color per pixel. That is, the B narrow band signal is missing in half of the pixels, and the G narrow band signal is missing in the remaining half of the pixels.

The interpolation processing unit 312 performs an interpolation process on the B narrowband signal from the interpolation processing unit 312. The interpolation processing is, for example, a smoothing filter processing of 5 × 5 pixels. Note that the smoothing range of the smoothing filter is not limited to 5 × 5 pixels. By performing the interpolation processing, the B narrowband signal is interpolated with respect to the pixel where the B narrowband signal is missing. The image constituted by the B narrowband signal is hereinafter referred to as a B narrowband image. The interpolation processing unit 312 performs an edge enhancement process on the B narrowband image. That is, the interpolation processing unit 312 extracts a high-frequency component from the B narrow-band image and adds the high-frequency component to the B narrow-band image. The interpolation processing unit 312 extracts a high-frequency component by performing, for example, an adaptive low-pass filter process of 7 × 7 pixels on the B narrowband image, and then performing a high-pass filter process of 7 × 7 pixels on the result. The interpolation processing unit 312 outputs the B narrowband signal after the edge enhancement processing as a final B narrowband signal. Note that the content of the edge enhancement processing is not limited to the above. Further, the edge enhancement processing may be omitted.

(5) The interpolation processing unit 312 performs an interpolation process on the G narrow band signal from the interpolation processing unit 312. The interpolation filter is, for example, a 5 × 5 pixel smoothing filter. Note that the smoothing range of the smoothing filter is not limited to 5 × 5 pixels. By performing the interpolation processing, the G narrow band signal is interpolated with respect to the pixel where the G narrow band signal is missing. The image constituted by the G narrow band signal is also called a G narrow band image. The interpolation processing unit 312 outputs the G narrow band signal after the interpolation processing.

The interpolation processing unit 312 sets the B narrowband image as the G channel and the B channel of the display image, and sets the G narrowband image as the R channel of the display image. Thus, a display image in the NBI mode is generated.

{3. Modified example

Hereinafter, various modifications will be described. Note that the following modifications can be appropriately combined.

In the first modification, the interpolation processing unit 312 performs the color separation processing after the interpolation processing. FIGS. 7 and 8 are diagrams illustrating color mixing in pixel signals. 7 and 8, BN indicates the spectrum of the B narrow band light, and GN indicates the spectrum of the G narrow band light.

FIG. 7 shows the spectra of the MgCy signal and the MgYe signal selected as the B narrowband signal. These signals are sensitive not only to B narrowband light but also to G narrowband light. FIG. 8 shows the spectra of the GCy signal and the GYe signal selected as the G narrow-band signal. These signals are sensitive not only to G narrowband light but also to B narrowband light. Since the B narrow-band light and the G narrow-band light are simultaneously irradiated on the subject, the B narrow-band signal is mixed with the G narrow-band component, and the G narrow-band signal is mixed with the B narrow-band component. .

The interpolation processing unit 312 reduces the above-described color mixture by performing a color separation process represented by the following equation (5). In the following equation (5), B (x, y) is a B narrowband signal at a position (x, y), and G (x, y) is a G narrowband signal at a position (x, y). The interpolation processing unit 312 performs color separation processing of the B narrowband signal between the interpolation processing and the edge enhancement processing in FIG. 6, for example.

In the second modification, the interpolation processing unit 312 performs a direction discrimination type interpolation process. That is, the interpolation processing of FIG. 6 is a direction determination type interpolation processing. Hereinafter, the interpolation process for the B narrowband signal will be described as an example, but the same applies to the interpolation process for the G narrowband signal.

First, a first example of the direction determination type interpolation processing will be described. In this example, interpolation processing is performed using a range of 3 × 3 pixels. The interpolation processing unit 312 determines the edge direction at (x, y), and selects one of the following equations (6) to (8) based on the determination result. (X, y) is the position where the B narrowband signal is missing. When it is determined that the edge direction is the y direction, the interpolation processing unit 312 obtains a B narrowband signal by the following equation (6). When it is determined that the edge direction is the x direction, the interpolation processing unit 312 obtains a B narrowband signal by the following equation (7). When it is determined that there is no edge, the interpolation processing unit 312 obtains a B narrowband signal by the following equation (8).

Next, a description will be given of a second example of the direction determination type interpolation processing. In this example, the interpolation processing is performed using a range wider than 3 × 3 pixels. 9 to 11 are diagrams for explaining a second example of the direction discrimination-type interpolation processing. 9 to 11, the range of pixels used for the interpolation processing is indicated by a dotted line. The bold square in the center indicates a pixel to be interpolated.

The interpolation processing unit 312 determines the edge direction at (x, y), and selects one of the following equations (9) to (11) based on the determination result. When it is determined that the edge direction is the y direction, the interpolation processing unit 312 obtains a B narrowband signal by using the pixels in the range shown in FIG. When it is determined that the edge direction is the x direction, the interpolation processing unit 312 obtains a B narrowband signal by using the pixels in the range shown in FIG. If it is determined that there is no edge, the interpolation processing unit 312 obtains a B narrowband signal using the pixels in the range shown in FIG.

In the third modification, the result of the interpolation processing in the edge direction and the result of the interpolation processing without using the direction determination are blended based on the reliability. Hereinafter, the interpolation process for the B narrowband signal will be described as an example, but the same applies to the interpolation process for the G narrowband signal.

The interpolation processing unit 312 obtains the reliability index value RLB (x, y) by the following equation (12). ave_MgCy is the average value of the MgCy signal at (x, y), and is the average value obtained from the MgCy signals of the peripheral pixels at (x, y). The same applies to ave_MgYe. The index value RLB (x, y) is a numerical value in the range of 0 <RLB (x, y) ≦ 1. The larger the index value RLB (x, y), the higher the reliability.

The interpolation processing unit 312 obtains a B narrowband signal at (x, y) by the following equation (13). Bededge (x, y) is the result of interpolation processing in the edge direction. When the direction discriminating type interpolation processing of the above equations (6) to (8) is used, Bedge (x, y) is the calculation result of the above equation (6) or (7). Bave (x, y) is a result of interpolation processing without using direction discrimination, and is, for example, a calculation result of the above equation (8).

In the fourth modification, the weight of the pixel signal in the interpolation processing is set based on the sensitivity of the pixel signal to narrowband light. Hereinafter, the interpolation process for the B narrowband signal will be described as an example, but the same applies to the interpolation process for the G narrowband signal.

感度 As shown in FIG. 7, when comparing the MgCy signal and the MgYe signal, the sensitivity to the B narrowband light is higher for the MgCy signal. For this reason, in the interpolation processing for the B narrowband signal, the weight of the MgCy signal is set higher than the weight of the MgYe signal. For example, in the above equations (9) to (11), the term of MgCy is multiplied by the weight W_MgCy, and the term of MgYe is multiplied by the weight W_MgYe. W_MgCy> W_MgYe. W_MgCy and W_MgYe are set in advance based on the spectrum of the pixel signal and the like.

As described above, the embodiments to which the present invention is applied and the modifications thereof have been described. However, the present invention is not limited to the embodiments and the modifications thereof as it is. Can be embodied by modifying the components. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in each of the above embodiments and modifications. For example, some components may be deleted from all the components described in the embodiments and the modifications. Furthermore, the components described in the different embodiments and modified examples may be appropriately combined. Thus, various modifications and applications are possible without departing from the gist of the invention. Further, in the specification or the drawings, a term described at least once together with a broader or synonymous different term can be replaced with the different term in any part of the specification or the drawing.

100 ° light source, 110 ° white light source, 120 ° lens, 130 ° filter, 200 ° insertion section, 210 ° light guide, 220 ° illumination lens, 230 ° objective lens, 240 ° image sensor, 250 ° A / D conversion circuit, 260 ° memory, 300 ° control device, 310 ° processing circuit Pixel signal selection section, 311 interpolation processing section, 320 control circuit, 400 display section, 500 external I / F section, Mg, Cy, Ye, G filter unit, MgCy, MgYe, GCy, GYe pixel signal

Claims

A light source that emits narrow-band illumination light having first narrow-band light and second narrow-band light;
An image sensor that has a color filter in which each filter unit is provided for each of the plurality of pixels, and that photoelectrically converts light received by the plurality of pixels;
A processing circuit that selects a first narrow-band pixel signal having sensitivity to the first narrow-band light and a second narrow-band pixel signal having sensitivity to the second narrow-band light among the pixel signals from the image sensor; When,
Including
The color filter,
The filter includes a filter unit that transmits the first narrow band light, the filter unit transmits half of the total number of filter units,
The processing circuit includes:
A first interpolation process of interpolating the first narrowband pixel signal without using the second narrowband pixel signal, and interpolating the second narrowband pixel signal without using the first narrowband pixel signal An endoscope apparatus, which performs a second interpolation process.
In claim 1,
The processing circuit includes:
Color separation between the first narrowband image and the second narrowband image based on a first narrowband image obtained by the first interpolation process and a second narrowband image obtained by the second interpolation process An endoscope apparatus for performing processing.
In claim 1,
The processing circuit includes:
Based on the brightness of the first narrowband pixel signal and the brightness of the second narrowband pixel signal, determine an index value indicating the reliability of direction determination in direction determination type interpolation processing,
An endoscope apparatus, wherein in the first interpolation processing and the second interpolation processing, the direction discriminating-type interpolation processing is performed based on the index value.
In claim 3,
The processing circuit includes:
In the direction discrimination type interpolation processing, the interpolation processing result in the edge direction and the non-direction discrimination type interpolation processing result are blended at a blend ratio based on the index value,
The endoscope apparatus according to claim 1, wherein the lower the reliability, the larger the blending ratio of the result of the non-direction discrimination-type interpolation processing.
In claim 3,
The processing circuit includes:
The endoscope apparatus according to claim 1, wherein the index value is obtained based on a ratio between the brightness of the first narrowband pixel signal and the brightness of the second narrowband pixel signal.
In claim 1,
The first narrowband pixel signal includes a first pixel signal, and a second pixel signal having a higher sensitivity to the first narrowband light than the first pixel signal,
The second narrowband pixel signal includes a third pixel signal, and a fourth pixel signal having a higher sensitivity to the second narrowband light than the third pixel signal,
The processing circuit includes:
In the first interpolation processing, the weight of the second pixel signal is made larger than the weight of the first pixel signal, and in the second interpolation processing, the weight of the fourth pixel signal is made larger than the weight of the third pixel signal. An endoscope apparatus characterized in that the size of the endoscope is also increased.
In claim 1,
The first narrowband pixel signal includes a first pixel signal and a second pixel signal corresponding to different colors,
The second narrowband pixel signal includes a third pixel signal and a fourth pixel signal corresponding to different colors,
The processing circuit includes:
In the first interpolation processing, based on an average value of the first pixel signal in peripheral pixels of the first target pixel and an average value of the second pixel signal in peripheral pixels of the first target pixel, Performing a brightness correction between the pixel signal and the second pixel signal,
In the second interpolation processing, the third interpolation processing is performed based on an average value of the third pixel signal in peripheral pixels of the second target pixel and an average value of the fourth pixel signal in peripheral pixels of the second target pixel. An endoscope apparatus for performing brightness correction between a pixel signal and the fourth pixel signal.
In claim 1,
The processing circuit includes:
A pixel signal whose sensitivity to the first narrowband light is equal to or greater than the sensitivity to the second narrowband light is selected as the first narrowband pixel signal, and the sensitivity to the second narrowband light is the first narrowband light. An endoscope apparatus which selects a pixel signal having a higher sensitivity to the second narrowband pixel signal.
In claim 1,
The processing circuit includes:
A brightness level between the first narrow-band pixel signal and the second narrow-band pixel signal with reference to a narrow-band pixel signal having a small color mixture among the first narrow-band pixel signal and the second narrow-band pixel signal. An endoscope apparatus characterized by adjusting the following.
In claim 1,
The endoscope apparatus, wherein the color filter is a complementary color filter.
In claim 10,
The color filter,
An endoscope apparatus comprising magenta, yellow, cyan, and green filter units.
In claim 1,
The color filter,
An endoscope apparatus characterized in that in a color filter of a primary color Bayer array, a red filter unit is replaced by a magenta filter unit.
In claim 1,
The processing circuit includes:
An endoscope apparatus wherein the display image is generated by inputting a first narrowband image obtained by the first interpolation processing to a G channel of a display image.
In claim 13,
The first narrowband light belongs to a blue wavelength band in white light, and has a wavelength band narrower than the blue wavelength band,
The second narrowband light belongs to a green wavelength band in white light, and has a wavelength band narrower than the green wavelength band,
The processing circuit includes:
The first narrowband image is input to the G channel and the B channel of the display image, and the second narrowband image obtained by the second interpolation processing is input to the R channel of the display image. Endoscope device.
In claim 1,
The light source is
In the white light observation mode, emit white illumination light, in the narrow band light observation mode, emit the narrow band illumination light,
The color filter,
Having a plurality of filter units for capturing a white light image when the white illumination light is emitted,
The filter unit that transmits the first narrowband light is at least half of the plurality of filter units, and the filter unit that transmits the second narrowband light is at least half of the plurality of filter units,
The processing circuit, in the narrowband light observation mode, selects the first narrowband pixel signal and the second narrowband pixel signal, and performs the first interpolation process and the second interpolation process. Endoscope device.
Endoscope including a light source that emits narrow-band illumination light having first narrow-band light and second narrow-band light, and an image sensor having a color filter in which each filter unit is provided for each of the plurality of pixels A method of operating a mirror device,
In the case where the color filter has half or more filter units transmitting the first narrow band light, and has half or more filter units transmitting the second narrow band light,
Among the pixel signals from the imaging element, a first narrowband pixel signal having sensitivity to the first narrowband light and a second narrowband pixel signal having sensitivity to the second narrowband light are selected,
A first interpolation process of interpolating the first narrowband pixel signal without using the second narrowband pixel signal, and interpolating the second narrowband pixel signal without using the first narrowband pixel signal And a second interpolation process for processing.
A program for processing an image captured by an image sensor having a color filter provided with each filter unit for each of a plurality of pixels,
In the case where the color filter has half or more filter units that transmit the first narrowband light of the narrowband illumination light, and has half or more filter units that transmit the second narrowband light of the narrowband illumination light,
Among the pixel signals from the imaging element, a first narrowband pixel signal having sensitivity to the first narrowband light and a second narrowband pixel signal having sensitivity to the second narrowband light are selected,
A first interpolation process of interpolating the first narrowband pixel signal without using the second narrowband pixel signal, and interpolating the second narrowband pixel signal without using the first narrowband pixel signal Performing a second interpolation process to be processed,
A program to be executed by a computer.