WO2017046876A1 - Endoscope system, image processing apparatus, and image processing method - Google Patents

Abstract

Provided are an endoscope system, an image processing apparatus, and an image processing method with which a high-resolution enhanced image can be obtained without amplifying noise even when a subject is irradiated with narrow-band illumination light. This endoscope system 1 includes an extraction unit 412 which extracts components of narrow-band light obtained at least from green light from pixel values generated from each of a plurality of pixels, said extraction being carried out by subtracting, from a pixel value obtained from the pixel with the least sensitivity to blue light, which constitutes the narrow-band light, among a plurality of pixels with sensitivity to blue light, a pixel value obtained from a pixel with high sensitivity to blue wavelength light multiplied by a coefficient with an absolute value less than or equal to 1.

Description

Endoscope system, image processing apparatus, and image processing method

The present invention is an endoscope that generates an enhanced image in which capillaries on a mucosal surface layer in a subject are emphasized using image data that is introduced into a living body and output from an endoscope that acquires the in-vivo image. The present invention relates to a system, an image processing apparatus, and an image processing method.

In recent years, in endoscopic systems, narrow-band light (hereinafter referred to as “narrow-band illumination light”) that is narrower than the wavelength band of light emitted from a white light source and included in the blue and green wavelength bands is referred to as an observation site of a subject. A technique for highlighting the capillaries and mucosal fine patterns on the surface of the mucosa by imaging the reflected light reflected at the observation site (see Patent Document 1) is known. In this technique, in an image sensor in which a color filter in which three types of filters that transmit red (R), green (G), and blue (B) are arranged in a predetermined pattern is provided on a light receiving surface, a green component In addition to a main sensitivity region sensitive to green light, a sub-sensitivity region sensitive to blue narrow-band illumination light is provided as a characteristic of a filter that transmits light (G filter).

Correlation calculation of pixel values of the pixel (G pixel) that receives the light transmitted through the G filter and the pixel (R pixel) that receives the light transmitted through the filter (R filter) that transmits the red component is performed. By doing this, a component of the sub-sensitivity region is extracted from the pixel value of the G pixel, and a pixel (B pixel) that receives the light transmitted through the filter (B filter) that transmits the extracted component of the sub-sensitivity region and the blue component Based on the pixel value, an enhanced image in which the capillaries and mucous fine patterns on the mucosal surface layer are emphasized is displayed on the display monitor.

JP 2012-170639 A

Incidentally, in Patent Document 1 described above, when the subject is irradiated with narrow-band illumination light, since the pixel value of the R pixel is very small, the gain of the pixel value of the R pixel is increased. At this time, since the noise component included in the pixel value of the R pixel is amplified as the sensitivity difference with respect to the G pixel of the R image in the green light of the narrowband illumination light increases, the SN ratio of the enhanced image obtained by the correlation calculation is increased. There was a problem of being lowered.

The present invention has been made in view of the above, and is an endoscope that can obtain a high-resolution enhanced image without amplifying noise even when a subject is irradiated with narrow-band illumination light. An object is to provide a mirror system, an image processing apparatus, and an image processing method.

In order to solve the above-described problems and achieve the object, an endoscope system according to the present invention includes at least a narrow-band first light included in a blue wavelength band and a narrow-band included in a green wavelength band. A light source device that emits narrow-band light, and an imaging device that generates an electrical signal by photoelectrically converting light received by each of a plurality of pixels arranged in a two-dimensional grid. The first filter that transmits light in the blue wavelength band, the second filter that transmits light in the blue wavelength band and the green wavelength band, and transmits light in at least the red wavelength band A first filter unit comprising a plurality of filters having a third filter, or a first filter that transmits light in the blue wavelength band, and light in the blue wavelength band and the red wavelength. Transmits light in the band A second filter unit comprising a plurality of filters having a fourth filter that transmits at least the light in the green wavelength band, and the light in the green wavelength band. The first filter unit in which the number of filters to be transmitted is more than half of the total number of filters, and the number of filters that transmit light in the blue wavelength band is greater than or equal to the number of filters that transmit light in the green wavelength band Or a color filter formed by arranging a second filter unit corresponding to the plurality of pixels, and an arithmetic processing for extracting the narrowband light component from the pixel value generated by each of the plurality of pixels, From the pixel value of the pixel having the lowest sensitivity to the first light among the plurality of pixels having sensitivity to the first light, the first of the plurality of pixels. By subtracting the product of the pixel value of the pixel having the highest sensitivity to light and the first coefficient having an absolute value of 1 or less, at least the component of the narrowband light obtained by the second light is extracted. And an extraction unit that performs arithmetic processing.

The endoscope system according to the present invention is the endoscope system according to the above invention, wherein the extraction unit uses the second value among the plurality of pixels based on a pixel value of the pixel having sensitivity to light in at least the red wavelength band. By further subtracting the product of the pixel value of the pixel having the highest sensitivity to light and the second coefficient whose absolute value is 1 or less, the component of the narrowband light obtained by the first light is further extracted. It is characterized by that.

In the endoscope system according to the present invention, in the above invention, the extraction unit extracts the narrowband light component obtained by the second light and then obtains the narrow light obtained by the first light. A band light component is extracted.

The endoscope system according to the present invention may further include an interpolation unit that generates an interpolation value of a channel for each color by performing an interpolation process on the pixel value generated by each of the plurality of pixels. And the extraction unit extracts the narrowband light component from the interpolation value generated by the interpolation unit performing the interpolation process.

Further, in the endoscope system according to the present invention, in the above invention, the pixel in which the first filter is arranged has the highest sensitivity to the first light among the plurality of pixels. The pixel in which the second filter is disposed or the pixel in which the fifth filter is disposed has the highest sensitivity to the second light among the plurality of pixels. .

The endoscope system according to the present invention is characterized in that, in the above-mentioned invention, the endoscope system further includes a display image generation unit that generates a display image signal based on the narrowband light component extracted by the extraction unit. To do.

In addition, the image processing apparatus according to the present invention is generated by an endoscope including an image sensor that generates an electric signal by photoelectrically converting light received by each of a plurality of pixels arranged in a two-dimensional grid. An image processing device that performs image processing on image data, wherein the first light of the narrow band included in the blue wavelength band emitted by the light source device from the pixel value generated by each of the plurality of pixels , A processing for extracting a narrowband light component comprising a second narrowband light included in the green wavelength band, wherein the first of the plurality of pixels having sensitivity to the first light. The pixel value and absolute value of the pixel having the highest sensitivity to the first light out of the plurality of pixels is 1 or less from the pixel value of the pixel having the lowest sensitivity to the light of 1 By subtracting the product with the coefficient And an extraction unit that performs calculation processing to extract at least a component of the narrowband light obtained by the second light, and the endoscope includes a first filter that transmits light in the blue wavelength band, A second filter configured to transmit light in the blue wavelength band and the green wavelength band, and a third filter configured to transmit at least light in the red wavelength band. One filter unit, or a first filter that transmits light in the blue wavelength band, a fourth filter that transmits light in the blue wavelength band and light in the red wavelength band, and at least the green filter A second filter unit comprising a plurality of filters having a fifth filter that transmits light in the wavelength band, and the number of filters that transmit light in the green wavelength band is the total number of filters. And a plurality of the first filter units or the second filter units, wherein the number of filters that transmit light in the blue wavelength band is greater than or equal to the number of filters that transmit light in the green wavelength band. It is characterized by comprising a color filter arranged corresponding to the pixels.

In addition, the image processing method according to the present invention is generated by an endoscope including an image sensor that generates an electric signal by photoelectrically converting light received by each of a plurality of pixels arranged in a two-dimensional grid. An image processing method executed by an image processing apparatus that performs image processing on image data, wherein a narrow band included in a blue wavelength band emitted by a light source device from a pixel value generated by each of the plurality of pixels The first light and a narrowband second light included in the green wavelength band, and extracting the narrowband light component. The pixel value and the absolute value of the pixel having the highest sensitivity to the first light among the plurality of pixels are 1 from the pixel value of the pixel having the lowest sensitivity to the first light among the pixels of A first coefficient which is An extraction step of performing an arithmetic process of extracting at least a component of the narrowband light obtained by the second light by subtracting a product, wherein the endoscope transmits light in the blue wavelength band A plurality of filters having a first filter, a second filter that transmits light in the blue wavelength band and the green wavelength band, and a third filter that transmits light in at least the red wavelength band Or a first filter that transmits light in the blue wavelength band, and a fourth filter that transmits light in the blue wavelength band and light in the red wavelength band. A second filter unit comprising a plurality of filters having at least a fifth filter that transmits light in the green wavelength band, and transmits light in the green wavelength band A first filter unit, wherein the number of filters is equal to or greater than half of the total number of filters, and the number of filters that transmit light in the blue wavelength band is equal to or greater than the number of filters that transmit light in the green wavelength band; It is provided with a color filter formed by arranging a second filter unit corresponding to the plurality of pixels.

According to the present invention, it is possible to obtain a high-resolution enhanced image without amplifying noise even when the subject is irradiated with narrow-band illumination light.

FIG. 1 is a schematic diagram showing a schematic configuration of an endoscope system according to Embodiment 1 of the present invention. FIG. 2 is a schematic diagram showing an example of the configuration of the color filter according to Embodiment 1 of the present invention. FIG. 3 is a diagram showing the relationship between the wavelength and the sensitivity characteristic of each of the B pixel, Cy pixel, and Mg pixel according to Embodiment 1 of the present invention. FIG. 4 is a diagram showing sensitivity characteristics between the narrowband light and each pixel according to the first embodiment of the present invention. FIG. 5 is a schematic diagram showing the structure in the layer direction of the biological tissue observed by the endoscope system according to Embodiment 1 of the present invention. FIG. 6A is a diagram illustrating an example of interpolation processing performed on Cy pixels by the interpolation unit according to Embodiment 1 of the present invention. FIG. 6B is a diagram illustrating an example of interpolation processing performed on the Cy pixels by the interpolation unit according to Embodiment 1 of the present invention. FIG. 7A is a diagram illustrating an example of an interpolation process performed on a B pixel or an Mg pixel by the interpolation unit according to Embodiment 1 of the present invention. FIG. 7B is a diagram illustrating an example of interpolation processing performed on the B pixel or the Mg pixel by the interpolation unit according to Embodiment 1 of the present invention. FIG. 7C is a diagram illustrating an example of interpolation processing performed on the B pixel or the Mg pixel by the interpolation unit according to Embodiment 1 of the present invention. FIG. 8 is a diagram showing an outline of processing for extracting a signal value generated by each narrowband light performed by the extraction unit according to Embodiment 1 of the present invention from the Cy channel or the Mg channel. FIG. 9A is a diagram illustrating an area of a B pixel in a narrow band. FIG. 9B is a diagram illustrating an area of a Cy pixel in a narrow band. FIG. 9C is a diagram illustrating the areas of B pixels and Cy pixels in a narrow band. FIG. 10 is a schematic diagram showing an example of the configuration of the color filter according to Embodiment 2 of the present invention. FIG. 11 is a diagram showing the relationship between the wavelength and the sensitivity characteristic of each of the B pixel, the G pixel, and the Mg pixel according to Embodiment 2 of the present invention. FIG. 12 is a diagram showing sensitivity characteristics of each narrowband light and each pixel according to the second embodiment of the present invention. FIG. 13 is a diagram showing an outline of processing for extracting a signal value generated by each narrowband light from the Mg channel performed by the extraction unit according to Embodiment 2 of the present invention. FIG. 14 is a schematic diagram showing an example of the configuration of the color filter according to Embodiment 3 of the present invention. FIG. 15 is a diagram showing the relationship between the wavelength and the sensitivity characteristic of each of the B pixel, the Cy pixel, and the R pixel according to the third embodiment of the present invention. FIG. 16 is a diagram showing sensitivity characteristics of each narrowband light and each pixel according to the third embodiment of the present invention. FIG. 17 is a diagram illustrating an outline of a process of extracting, from the Cy channel, a signal generated by each narrowband light performed by the extraction unit according to Embodiment 3 of the present invention. FIG. 18 is a diagram showing sensitivity characteristics of each narrowband light and each pixel according to the fourth embodiment of the present invention. FIG. 19 is a diagram showing an outline of processing for extracting a signal generated by each narrowband light performed by the extraction unit according to Embodiment 4 of the present invention from the Cy channel and the Mg channel.

Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described. In the present embodiment, a medical endoscope system that captures and displays an image of a body cavity of a subject such as a patient will be described. Further, the present invention is not limited by the present embodiment. Furthermore, in the description of the drawings, the same portions will be described with the same reference numerals.

(Embodiment 1)
FIG. 1 is a schematic diagram showing a schematic configuration of an endoscope system according to Embodiment 1 of the present invention. An endoscope system 1 shown in FIG. 1 includes an endoscope 2 that acquires an image signal in a body cavity by being inserted into a subject, and a light source device 3 that generates illumination light emitted from the distal end of the endoscope 2. The processor unit 4 performs predetermined image processing on the image signal acquired by the endoscope 2 and controls the overall operation of the endoscope system 1. The processor unit 4 corresponds to the image signal input from the processor unit 4. And a display unit 5 for displaying an image to be displayed.

[Configuration of endoscope]
First, the configuration of the endoscope 2 will be described.
The endoscope 2 includes an operation unit 200, an imaging lens 201, an imaging unit 202, a light guide 203, an illumination lens 204, an A / D conversion unit 205, and an imaging information storage unit 206.

The operation unit 200 operates peripheral devices such as an air supply unit, a water supply unit, and a gas supply unit in addition to an instruction signal for instructing the light source device 3 to switch the illumination light and an instruction signal for changing the control content of the processor unit 4. An input of an instruction signal for instructing is received.

The imaging lens 201 is provided at the distal end of the endoscope 2 and collects at least reflected light from the subject. The imaging lens 201 is configured using one or a plurality of lenses and a prism.

The imaging unit 202 receives the light collected by the imaging lens 201 and performs photoelectric conversion, thereby generating an image signal of the subject, and outputs the image signal to the A / D conversion unit 205. The imaging unit 202 includes an imaging element 202a and a color filter 202b (first color filter).

The imaging element 202a is realized by using an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image sensor 202a has a plurality of pixels arranged in a two-dimensional lattice.

The color filter 202b has a plurality of transmission filters that transmit light in wavelength bands that are individually set. Specifically, a first filter that transmits light in the blue wavelength band, a second filter that transmits light in the blue wavelength band and the green wavelength band, and light in at least the red wavelength band A filter unit comprising a plurality of filters, wherein the number of filters transmitting light in the green wavelength band is more than half of the total number of filters, and the blue wavelength band A filter unit in which the number of filters that transmit light is equal to or greater than the number of filters that transmit light in the green wavelength band is arranged in correspondence with a plurality of pixels of the image sensor 202a. Here, the configuration of the color filter 202b will be described. FIG. 2 is a schematic diagram illustrating an example of the configuration of the color filter 202b.

As shown in FIG. 2, the color filter 202b includes a filter B (first filter) that transmits light in the blue wavelength band, and a filter Cy (that transmits light in the blue wavelength band and light in the green wavelength band). A second filter) and a filter Mg (third filter) that transmits light in the red wavelength band and light in the blue wavelength band. The arrangement pattern of the color filter 202b is basically a filter B on the upper left, a filter Cy on the lower left and upper right, and a filter Mg on the lower right based on 2 × 2 pixels. The color filter 202b is arranged over all the pixels of the image sensor 202a according to the arrangement pattern described above. In the following, pixels in which each of the filter B, the filter Cy, and the filter Mg are arranged are denoted as B pixel, Cy pixel, and Mg pixel.

The sensitivity characteristics with respect to the wavelength of light of each of the B pixel, the Cy pixel, and the Mg pixel configured as described above will be described. FIG. 3 is a diagram illustrating the relationship between the sensitivity characteristic with respect to the wavelength of light and the wavelength of each of the B pixel, the Cy pixel, and the Mg pixel. In FIG. 3, the vertical axis represents sensitivity, and the horizontal axis represents wavelength. In FIG. 3, each of a curve L _B (solid line), a curve L _Cy (one-dot chain line), and a curve L _Mg (dotted line) indicates sensitivity characteristics of the B pixel, the Cy pixel, and the Mg pixel.

As shown by the curve L _B in FIG. 3, B pixel has a first peak sensitivity in the wavelength band of blue light (400nm ~ 470nm). Further, as shown by the curve L _Cy in FIG. 3, the Cy pixel has a first peak of sensitivity for green light (500 nm to 550 nm) and a second peak of sensitivity for blue light (400 nm to 470 nm). Have. Further, as shown by the curve L _Mg in FIG. 3, the Mg pixel has a first peak of sensitivity for red light (600 nm to 650 nm) and a second peak of sensitivity for blue light (400 nm to 470 nm). Have.

Returning to FIG. 1, the description of the configuration of the endoscope 2 will be continued.
The light guide 203 is configured using glass fiber or the like, and guides light emitted from the light source device 3 described later.

The illumination lens 204 is provided at the tip of the light guide 203, diffuses the light guided by the light guide 203, and emits the light from the tip of the endoscope 2 to the outside.

The A / D conversion unit 205 performs A / D conversion on the analog image signal generated by the image sensor 202 a to generate a digital image signal, and outputs the image signal to the processor unit 4.

The imaging information storage unit 206 records data including various programs for operating the endoscope 2, various parameters necessary for the operation of the endoscope 2, identification information of the endoscope 2, and the like. In addition, the imaging information storage unit 206 includes an identification information storage unit 206a that stores identification information. The identification information includes unique information (ID) of the endoscope 2, year, specification information, transmission method, filter arrangement information for the color filter 202 b, and the like. The imaging information storage unit 206 is realized using a flash memory or the like.

[Configuration of light source device]
Next, the configuration of the light source device 3 will be described.
The light source device 3 includes an illumination unit 31 and an illumination control unit 32.

The illumination unit 31 switches and emits a plurality of illumination lights having different wavelength bands under the control of the illumination control unit 32. The illumination unit 31 includes a light source unit 311, a light source driver 312, a switching filter 313, a drive unit 314, a drive driver 315, and a condenser lens 316.

The light source unit 311 emits white light including red, green, and blue light. White light emitted from the light source unit 311 is emitted to the outside from the distal end of the endoscope 2 via the switching filter 313, the condenser lens 316, and the light guide 203. The light source unit 311 is realized using an LED (Light Emitting Diode), a xenon lamp, or the like.

The light source driver 312 supplies white light to the light source unit 311 by supplying power to the light source unit 311 under the control of the illumination control unit 32.

The switching filter 313 is detachably disposed on the optical path of white light emitted from the light source unit 311. The switching filter 313 transmits only the blue narrow-band light and the green narrow-band light among the white light emitted from the light source unit 311. Specifically, the switching filter 313 transmits narrowband light composed of light in a narrow band T _B (eg, 390 nm to 445 nm) included in white light and light in a narrow band T _G (eg, 530 nm to 550 nm). To do. The light in each of the narrow band T _B and the narrow band T _G has a blue and green wavelength band that is easily absorbed by hemoglobin in the blood.

FIG. 4 is a diagram showing sensitivity characteristics between narrowband light and each pixel. In FIG. 4, the vertical axis indicates sensitivity, and the horizontal axis indicates wavelength.

As shown in FIG. 4, the sensitivity of the narrow band _TG is the highest for Cy pixels. Also, the sensitivity of the narrowband T _B is, Cy pixel is the lowest, the difference is set so as not to occur between the B pixel and the Mg pixel.

Returning to FIG. 1, the description of the configuration of the light source device 3 will be continued.
The drive unit 314 inserts or displaces the switching filter 313 on the optical path of white light emitted from the light source unit 311. The drive unit 314 is realized using a stepping motor, a DC motor, or the like.

The drive driver 315 supplies predetermined power to the drive unit 314 under the control of the illumination control unit 32.

The condensing lens 316 collects the white light emitted from the light source unit 311 or the narrowband light transmitted through the switching filter 313 and emits the light to the outside of the light source device 3 (light guide 203).

The illumination control unit 32 controls the light source driver 312 to turn on / off the light source unit 311. In addition, the illumination control unit 32 controls the drive driver 315 to insert and remove the switching filter 313 with respect to the optical path of white light emitted from the light source unit 311, thereby changing the illumination light emitted from the illumination unit 31 to white. light, and narrow-band T _B narrowband T switch to narrowband light control consisting of the _G light is carried out.

[Processor configuration]
Next, the configuration of the processor unit 4 will be described.
The processor unit 4 includes an image processing unit 41, a display image generation unit 42, an input unit 43, a storage unit 44, and a control unit 45. In the first embodiment, the processor unit 4 functions as an image processing apparatus.

The image processing unit 41 performs predetermined image processing on the image signal input from the A / D conversion unit 205 and outputs the image signal to the display image generation unit 42. The image processing unit 41 includes an interpolation unit 411 and an extraction unit 412.

The interpolation unit 411 performs an interpolation process on the image signal input from the A / D conversion unit 205. Specifically, the interpolation unit 411 performs an interpolation process based on direction information (for example, an edge) on each of the B pixel, the Cy pixel, and the Mg pixel, thereby each of the B channel, the Cy channel, and the Mg channel. The interpolation values of the B channel, the Cy channel, and the Mg channel are output to the extraction unit 412. Further, the interpolation unit 411 generates the B channel and the Mg channel with high resolution by using the direction information of the predetermined pixels for each of the B pixel, the Cy pixel, and the Mg pixel under the observation of the narrow band light.

Extraction unit 412, the observation of a narrow-band light, a processing for extracting a component of the narrow-band light from the pixel values, each generated in a plurality of pixels constituting the image pickup element 202a, the narrow-band T _B light sensitivity from the pixel value of the pixel having the lowest sensitivity sensitivity to light of a narrow band T _B of the plurality of pixels with respect to the pixel value of the highest pixel sensitivity to light of a narrow band T _B of the plurality of pixels By subtracting the product of the first coefficient having an absolute value of 1 or less, a calculation process for extracting at least a narrowband light component obtained by the narrowband _TG is performed, and the calculation result is displayed on the display image generation unit 42. Output to. Specifically, the extraction unit 412 extracts a predetermined signal value from each of the interpolation values of the B channel, the Cy channel, and the Mg channel input from the interpolation unit 411, and sends the extracted signal value to the display image generation unit 42. Output. For example, the extraction unit 412 extracts the signal values of the narrow band T _B and the narrow band T _G constituting the narrow band light by the arithmetic processing between the channels, and outputs the extracted signal value to the display image generation unit 42. To do. In addition, the extraction unit 412 uses a color conversion matrix or a conversion table for the interpolation values of the B channel, the Cy channel, and the Mg channel under white light observation, so that the B pixel, the G pixel, and the R pixel are used. Extract the signal value of each of the images.

The display image generation unit 42 generates a color image and a pseudo color image based on the signal value input from the image processing unit 41 and outputs the color image and the pseudo color image to the display unit 5. Specifically, the display image generating unit 42, under the narrow band light, each of the signal values of the narrow-band T _B and narrowband T _G constituting the narrowband light input from the image processing unit 41, and a control unit A pseudo color is generated based on the color filter information input from 45. For example, the display image generating unit 42 assigns to each of the B channel and the G channel of the display image signal a signal value of the narrow band T _B, allocating a signal value of the narrow band T _G to R channel of the display image signal. In addition, the display image generation unit 42 generates a color image using the B channel, G channel, and R channel signals output from the image processing unit 41 under white light. Further, the display image generation unit 42 performs gradation conversion processing, enlargement processing, structure enhancement processing, and the like on the pseudo color and the color image, generates a display image, and outputs the display image to the display unit 5.

The input unit 43 receives input of instruction signals for instructing various operations related to the endoscope system 1. For example, the input unit 43 receives an input of an instruction signal that instructs illumination light emitted from the light source device 3.

The storage unit 44 stores various programs for operating the endoscope system 1 and data including various parameters necessary for the operation of the endoscope system 1. In addition, the storage unit 44 stores identification information of the endoscope 2, for example, a relationship table between unique information of the image sensor 202a and information related to the filter arrangement of the color filter 202b. The storage unit 44 is realized using a semiconductor memory such as a flash memory or a DRAM.

The control unit 45 performs drive control of each component including the endoscope 2 and the light source device 3 and information input / output control for each component. Further, the control unit 45 acquires setting data for imaging control, control data related to imaging, and control data related to illumination stored in the storage unit 44, and control signals corresponding to the acquired setting data and control data, respectively. Is output to the endoscope 2 and the light source device 3 via a predetermined signal line or bus. In addition, the control unit 45 outputs the color filter information output via the imaging information storage unit 206 to each of the image processing unit 41 and the display image generation unit 42. The control unit 45 is configured using a CPU (Central Processing Unit) or the like.

[Configuration of display section]
Next, the display unit 5 will be described.
The display unit 5 displays an image corresponding to the display image signal input from the processor unit 4. The display unit 5 is realized using liquid crystal, organic EL (Electro Luminescence), or the like.

In the endoscope system 1 having the above configuration, an outline of high resolution processing by the extraction unit 412 of the image processing unit 41 will be described in detail. FIG. 5 is a schematic diagram showing a layered structure of a biological tissue observed by the endoscope system 1.

As shown in FIG. 5, the living tissue 71 as the subject has blood vessels with different depths. Specifically, many capillaries 72 are mainly distributed in the vicinity of the mucous membrane surface layer of the living tissue 71. Further, in the middle layer deeper than the surface layer of the mucous membrane of the living tissue 71, blood vessels 73 that are thicker than the capillaries 72 are distributed in addition to the capillaries 72. Furthermore, a thicker blood vessel 74 is distributed in the deep layer of the biological tissue 71 than the thick blood vessel 73 in the middle layer.

Further, the depth of light reaching the living tissue 71 depends on the wavelength. For example, narrow band of light T _B is reflected in the vicinity of the mucosal surface, the light of the narrow-band T _G reaches to a position deeper than the mucosal surface layer. That is, the endoscope system 1, the narrow-band light is irradiated to the living body tissue 71, by the light of the narrow-band T _B which is reflected to the imaging (light) in the living tissue 71, the surface of the capillary 72 in the body tissue 71 Information (hereinafter referred to as “surface layer information”), and by imaging (receiving) the light of the narrow band _TG reflected by the living tissue 71, the thick blood vessel 73 in the middle layer in the living tissue 71 and Information on each of the deep blood vessels 74 (hereinafter referred to as “mid-deep layer information”) can be acquired.

Also, B pixels, in the observation of a narrow-band light, sensitive to narrowband T _B. For this reason, the surface layer information is included in the pixel value of the B pixel. Furthermore, each of the Cy pixel and Mg image has a sensitivity to light of the light and narrowband T _G narrowband T _B. For this reason, each of the pixel value of the Cy pixel and the pixel value of the Mg pixel includes surface layer information and mid-deep layer information.

Thus, in the first embodiment, B channel interpolation unit 411 performs interpolation processing, and outputs the interpolated value of Cy channel and Mg channel to the extraction unit 412, extraction unit 412 narrowband T _B and narrowband A signal value obtained by each light of _TG is extracted from the Cy channel or the Mg channel, and is output to the display image generation unit 42.

[Interpolation by the interpolation unit]
Next, an interpolation process performed by the interpolation unit 411 will be described. 6A and 6B are diagrams illustrating an example of an interpolation process performed by the interpolation unit 411 on the Cy pixel. 7A to 7C are diagrams illustrating an example of interpolation processing performed by the interpolation unit 411 for the B pixel or the Mg pixel.

As shown in FIGS. 6A and 6B, first, the interpolation unit 411 performs vertical (see arrow A1) and horizontal (see arrow A1) Cy pixels near the interpolation target pixel P1 (target pixel). The Cy channel is interpolated by calculating the difference between the pixel values and performing weighted averaging so as to give priority to the pixel value in the direction in which the difference is small (FIG. 6A → FIG. 6B).

As shown in FIGS. 7A to 7C, the interpolating unit 411 first calculates the difference between the pixel values in the diagonal direction (see the arrow A2) for the B pixel in the vicinity of the interpolation target pixel P1 (target pixel). A B channel having a checkered value is created by performing weighted averaging so as to give priority to pixel values in a direction where the difference is small (FIG. 7A → FIG. 7B).

Subsequently, the interpolation unit 411 obtains a difference between pixel values in the vertical direction (see the arrow A3) and the horizontal direction (see the arrow A3) for the B pixel in the vicinity of the interpolation target pixel (target pixel). The B channel signal value is interpolated by performing weighted averaging so as to give priority to the pixel value in the direction of smaller B. In addition, the interpolation unit 411 interpolates the Mg channel by the above-described interpolation processing as in the case of the B pixel. (FIG. 7B → FIG. 7C). Incidentally, the interpolation unit 411, in addition to the interpolation process described above also, B pixels, Cy pixel and Mg pixel has a sensitivity to narrowband T _B, due to a high mutual correlation, based on the direction information of the Cy pixel, You may perform interpolation of B pixel and Mg pixel.

[Processing of the extraction unit]
Next, the extraction unit 412 will be described the process for extracting a signal value generated by the light of the light and narrowband T _G narrowband T _B from each of the interpolated values of Cy channel or Mg channel. FIG. 8 is a diagram illustrating an outline of processing performed by the extraction unit 412 to extract a signal value generated by each narrow-band light from the Cy channel or the Mg channel. In FIG. 8, B-channel, the respective interpolated value of Cy channel and Mg channels, indicated as P _B, P _Cy and P _Mg. Further, the signal value of the narrow band T _B included in P _B is b, and the signal values of the narrow band T _B and the narrow band T _G included in P _Cy are b ′ and g ′, respectively, and are included in _PMg. The signal values of the narrow band T _B and the narrow band T _G are b ″ and g ″.

As shown in FIG. 8, the imaging device 202a of the coordinates (x, y) each of the interpolated value P _B of each channel in the (pixel of interest), P _Cy and P _Mg has the following formula (1-1) to Formula ( 1-3).
P _B (x, y) = b (x, y) (1-1)
P _Cy (x, y) = g ′ (x, y) + b ′ (x, y) (1-2)
P _Mg (x, y) = g ″ (x, y) + b ″ (x, y) (1-3)
Further, b ′ is expressed by the following equation (1-4) using the coefficient α.
b ′ (x, y) = αb (x, y) (1-4)

Here, the coefficient α will be described with reference to FIGS. 9A to 9C. FIG. 9A is a diagram showing the area of the B pixel (curve L _B ) in the narrow band T _B. Figure 9B is a diagram showing the area of the Cy pixel in the narrow-band T _B (curve L _Cy). FIG. 9C is a diagram illustrating the areas of the B pixel (curve L _B ) and the Cy pixel (curve L _Cy ) in the narrow band T _B. In FIG. 9A ~ Figure 9C, the area of S _B of the curve L _B in the narrow-band T _B, the area of the curve L _Cy and S _Cy in narrowband T _B.

As shown in FIGS. 9A ~ FIG 9C, the coefficient alpha, the ratio between the area S _Cy narrow area of the curve L _B in the band T _B S _B and the curve L _Cy, described in equation (1-5) Is done.
α = S _Cy / S _B (1-5)
The Figure 4 described above, the sensitivity of the B pixels included in the narrowband T _B is always higher than the sensitivity of the Cy pixel. Therefore, the coefficient α is 0 <α ≦ 1. Therefore, from the above formula (1-1) and formula (1-4),
b ′ (x, y) = αP _B (x, y) (1-6)
It becomes. Since the coefficient α is 0 <α ≦ 1, b ′ can be obtained from the equation (1-6) without amplifying the noise included in P _B. Further, according to the above-described formula (1-2) and formula (1-6),
g ′ (x, y) = P _Cy (x, y) −αP _B (x, y) (1-7)
Thus, g ′ included in P _Cy can be obtained.

Further, the sensitivity ratio in the narrow band _TG of each of the Cy pixel and the Mg pixel is expressed by the following expression (1-8) using the coefficient β.
g ″ (x, y) = βg ′ (x, y) (1-8)
Here, the coefficient beta, the ratio of narrowband T area of the curve L _Cy in _G S _Cy 'and curve L _Mg area S _Mg', represented by the following formula (1-9).
β = S _Mg '/ S _Cy ' (1-9)
From FIG. 4 described above, the sensitivity of the Cy pixel with respect to the narrow band _TG is always higher than the sensitivity of the Mg pixel. Therefore, the coefficient β is 0 <β ≦ 1. Therefore, from the above formulas (1-7) and (1-8),
g ″ (x, y) = β (P _Cy (x, y) −αP _B (x, y))
... (1-10)
From the equations (1-3) and (1-10),
b ″ (x, y) = _PMg (x, y)
-Β (P _Cy (x, y) -αP _B (x, y))
... (1-11)
It becomes. Since each of the coefficient α and the coefficient β is 0 <α ≦ 1 and 0 <β ≦ 1, b ″ included in P _Mg is not amplified without amplifying the noise included in each of P _B and P _Cy. Can be sought.

As described above, the extraction unit 412 obtains the signal value b and the signal value b ″ in all the pixels of the image sensor 202a using the above formulas (1-1) and (1-11). Extract surface layer information of living tissue. Further, the extraction unit 412 extracts the mid-deep layer information by obtaining the signal value g ′ in all the pixels of the image sensor 202a using the above-described equation (1-7). Extraction unit 412 outputs of each B pixel and Mg pixels' and the signal value g of the narrow band T _G of Cy pixel 'narrowband signal value b and the signal value b of T _B' to the display image generating unit 42 a .

[Processing of display image generation unit]
Next, processing performed by the display image generation unit 42 will be described.
The display image generation unit 42 generates a pseudo color image using the signal values b, b ″, and g ′ output from the extraction unit 412. The pseudo color image includes three channels of R, B, and G. First, in generating the B channel and the G channel, the display image generating unit 42 may generate the B channel and assign the same information to the G channel.

Specifically, first, the display image generation unit 42 outputs each of the signal value b and the signal value b ″ input from the extraction unit 412 on the B channel based on the color filter information input from the control unit 45. To the B pixel position and Mg pixel position. In this case, the display image generation unit 42 multiplies the signal value b ″ by the coefficient k to make the brightness of the signal value b and the signal value b ″ uniform. The coefficient k can be expressed by the following equation (1-12) based on the sensitivity ratio between the B pixel and the Mg pixel.
k = S _B / S _Mg (1-12)
Wherein each of S _B and S _Mg is the area of the curve L _B and the curve L _Mg in narrowband T _B. Also, each B pixel and Mg pixel, since a small difference in sensitivity in the narrow-band T _B, a small increase in noise in the signal value b '' by a factor k.

It should be noted that the coefficient k of the sensitivity ratio between the B pixel and the Mg pixel preferably satisfies the following condition (1-13).
1 ≦ k ≦ 2 (1-13)
Furthermore, it is preferable that the following condition (1-14) is satisfied.
k ≦ 1 / β (1 + α) (1-14)
Furthermore, it is preferable to satisfy the following condition (1-15).
β (1 + α) ≦ 1 (1-15)
Here, α is the value of the above equation (1-5), and β is the value of the above equation (1-9).

Thereafter, the display image generation unit 42 generates a surface layer information image for the position of the Cy pixel on the B channel using the signal value b at the position of the neighboring B pixel and the signal value b '' at the position of the Mg pixel. . Here, in generating the surface layer information image, interpolation processing using direction information may be used, or linear interpolation processing may be used. Alternatively, the priority order may be set without using the interpolation process, and either of the signal values b and b ″ may be used. After the display image generation unit 42 generates the B channel, the same information as the B channel is generated. Are assigned to the G channel.

Further, the display image generation unit 42 generates a mid-depth information image by assigning the signal value g ′ input from the extraction unit 412 to the position of each pixel on the R channel.

According to the first embodiment of the present invention described above, the extraction unit 412 is B channels, by performing arithmetic processing as much as possible not to perform amplification of noise to Cy channels and Mg channel interpolation value P _Cy and interpolated values it is possible to extract high-quality narrow-band signal from the P _Mg, while suppressing amplification of noise, the mucosal surface in the observation region of a subject under the narrow band light, the middle layer and deep high-resolution image acquisition can do.

Further, according to the first embodiment of the present invention, the extraction unit 412 determines the Cy channel with the highest sensitivity to the light in the narrow band _TG among the plurality of pixels constituting the image sensor 202a from the interpolation value of the Mg channel. by subtracting the product of the interpolation value and the beta, since further extracts the narrowband light components obtained by the optical narrow-band T _B, while suppressing amplification of noise, the subject under the narrow band light observation High-resolution images of the mucosal surface layer, middle layer and deep layer at the site can be acquired.

Furthermore, according to the first embodiment of the present invention, after the extraction unit 412 extracts a narrow-band light components obtained by the optical narrow-band T _G, narrowband T component of the narrow-band light obtained by the light of _B Therefore, a high-resolution image can be acquired.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. The endoscope system according to the second embodiment has a configuration different from that of the color filter 202b according to the first embodiment described above, and a process performed by each of the extraction unit and the display image generation unit. In the following, after describing the configuration of the color filter according to the second embodiment, the processing executed by the extraction unit and the display image generation unit according to the second embodiment will be described. In addition, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and description is abbreviate | omitted.

FIG. 10 is a schematic diagram showing an example of the configuration of the color filter according to Embodiment 2 of the present invention.

The color filter 202c (second color filter) illustrated in FIG. 10 includes a filter B (first filter), a filter G (fifth filter) that transmits light in the green wavelength band, and a filter Mg (fourth filter). Filter). The arrangement pattern of the color filter 202c is based on 2 × 2 pixels, and is a filter B on the upper left, a filter G on the lower left and upper right, and a filter Mg on the lower right filter. The color filter 202c is arranged over all the pixels of the image sensor 202a according to the arrangement pattern described above. In the following, the pixels in which each of the filter B, the filter G, and the filter Mg are arranged are denoted as a B pixel, a G pixel, and an Mg pixel.

The sensitivity characteristics with respect to the wavelength of light of each of the B pixel, the G pixel, and the Mg pixel configured as described above will be described. FIG. 11 is a diagram illustrating the relationship between the sensitivity characteristic with respect to the wavelength of light and the wavelength of each of the B pixel, the G pixel, and the Mg pixel. In FIG. 11, the vertical axis indicates the sensitivity, and the horizontal axis indicates the wavelength. In FIG. 11, each of a curve L _B (solid line), a curve L _G (dashed line), and a curve L _Mg (dotted line) indicates sensitivity characteristics of the B pixel, the G pixel, and the Mg pixel.

As shown by the curve L _B in FIG. 11, B pixel has a first peak sensitivity in the wavelength band of blue light (400nm ~ 470nm). Further, as shown by the curve _{L g} of FIG. 11, G pixel has a first peak of the sensitivity to green light (500nm ~ 550nm). Further, as shown by a curve L _Mg in FIG. 11, the Mg pixel has a first peak of sensitivity for red light (600 nm to 650 nm) and a second peak of sensitivity for blue light (400 nm to 470 nm). Have.

FIG. 12 is a diagram showing the narrowband light and the sensitivity characteristics of each pixel. In FIG. 12, the horizontal axis indicates the wavelength, and the vertical axis indicates the sensitivity.

As shown in FIG. 12, the G pixel has the highest sensitivity at each wavelength in the narrow band _TG . Further, the sensitivity at each wavelength in the narrow band T _B, the difference is designed so as not to occur between the B pixel and the Mg pixel.

Interpolation processing in which the interpolation unit 411 performs interpolation using the pixel values of the B pixel, the G pixel, and the Mg pixel with respect to the image data generated by the imaging element 202a in which the color filter 202c configured as described above is arranged. I do. In this case, the interpolation unit 411 performs predetermined interpolation using linear interpolation or the like using the pixel values of the B pixel, the G pixel, and the Mg pixel. The interpolation unit 411 outputs the interpolation values of the B channel, the G channel, and the Mg channel generated from the pixel values of the B pixel, the G pixel, and the Mg pixel to the extraction unit 412.

[Processing of the extraction unit]
Next, processing performed by the extraction unit 412 will be described. FIG. 13 is a diagram illustrating an outline of processing performed by the extraction unit 412 to extract a signal value generated by each narrowband light from the Mg channel. In Figure 13, B-channel, each of the interpolation value of the G channel and Mg channels, indicated as P _B, P _G, and P _Mg. Further, the signal value of the narrow band T _B included in P _B is set as b, the signal value of each of the narrow bands T _G included in P _G is set as g ′, and the narrow band T _B and the narrow band T included in P _Mg are set. Let each signal value of _{G be} b ″ and g ″.

As shown in FIG. 13, each of P _B , P _G, and P _Mg at the coordinates (x, y) (target pixel) of the image sensor 202a is expressed by the following equations (2-1) to (2-3). Is done.
P _B (x, y) = b (x, y) (2-1)
P _G (x, y) = g ′ (x, y) (2-2)
P _Mg (x, y) = g ″ (x, y) + b ″ (x, y (2-3)
Further, g ′ is expressed by the following equation (2-4) using a coefficient γ based on each wavelength for the narrow band _TG of each of the G pixel and the Mg pixel.
g ″ (x, y) = γP _G (x, y) (2-4)
From the above formulas (2-3) and (2-4),
b '' (x, y) = P Mg (x, y) -γP G (x, y) ··· (2-5)

Here, as shown in FIG. 12, the sensitivity of the G pixel is always higher than the sensitivity of the Mg pixel in each wavelength component included in the narrow band _TG . Therefore, the coefficient γ is 0 <γ ≦ 1. Therefore, without amplifying the noise contained in the P _Mg by formula (2-5) described above, can be obtained b '' contained in P _Mg.

As described above, the extraction unit 412 obtains the signal value b and the signal value b ″ in all the pixels of the image sensor 202a using the above-described equations (2-1) and (2-5). Extract surface layer information of living tissue. Further, the extraction unit 412 extracts the mid-deep layer information by obtaining the signal value g ′ in all the pixels of the image sensor 202a using the above-described equation (2-2). The extraction unit 412 outputs the signal value b and the signal value b ″ of the narrow band T _B of each of the B pixel and the Mg pixel and the signal value g ′ of the narrow band T _G of the G pixel to the display image generation unit 42. .

[Processing of display image generation unit]
Next, processing of the display image generation unit 42 will be described.
The display image generation unit 42 generates a pseudo color image using the signal values b, b ″, g ′ output from the extraction unit 412. The pseudo color image is composed of three channels of R, B, and G. First, in generating the B channel and the G channel, the display image generation unit 42 may generate the B channel and assign the same information to the G channel.

Based on the color filter information input from the control unit 45, the display image generation unit 42 converts each of the signal value b and the signal value b ″ input from the extraction unit 412 to the position of the B pixel on the B channel and Mg Assign to pixel location. In this case, the display image generation unit 42 multiplies b ″ by a coefficient k to make the brightness of the signal value b and the signal value b ″ uniform. The coefficient k can be expressed by the above equation (1-12) based on the sensitivity ratio between the B pixel and the Mg pixel. Thereafter, the display image generation unit 42 generates a surface layer information image using the signal value b at the position of the neighboring B pixel and the signal value b '' at the position of the Mg pixel at the position of the G pixel. Here, in generating the surface layer information image, interpolation processing using direction information may be used, or linear interpolation processing may be used. Alternatively, the priority order may be set without using interpolation, and either of the signal values b and b ″ may be used. The display image generation unit 42 generates the same information as the B channel after generating the B channel. Further, the display image generation unit 42 generates a mid-depth information image by assigning the signal value g ′ input from the extraction unit 412 to the position of each pixel on the R channel.

According to the second embodiment of the present invention described above, B channels, by performing arithmetic processing as much as possible not to perform amplification of noise to the G channel and Mg channels, high-quality narrowband P _G and P _Mg Can be extracted. Thereby, it is possible to acquire high-resolution images of the mucosal surface layer, the middle layer, and the deep layer at the observation site of the subject under narrow band light without amplifying noise.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. The endoscope system according to the third embodiment has a configuration different from that of the color filter 202b according to the first embodiment described above, and different processes performed by each of the extraction unit and the display image generation unit. In the following, after describing the configuration of the color filter according to the third embodiment, the processing executed by the extraction unit and the display image generation unit according to the third embodiment will be described. In addition, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and description is abbreviate | omitted.

FIG. 14 is a schematic diagram showing an example of the configuration of a color filter according to Embodiment 3 of the present invention.

The color filter 202d illustrated in FIG. 14 includes a filter B (first filter), a filter Cy (second filter), and a filter R (third filter) that transmits light in the red wavelength band. . The arrangement pattern of the color filter 202d is basically a filter B on the upper left, a filter Cy on the lower left and upper right, and a filter R on the lower right, based on 2 × 2 pixels. The color filter 202d is arranged over all the pixels of the image sensor 202a according to the arrangement pattern described above. In the following, a pixel in which each of the filter B, the filter Cy, and the filter R is arranged is denoted as a B pixel, a Cy pixel, and an R pixel.

The sensitivity characteristics with respect to the wavelength of light of each of the B pixel, the Cy pixel, and the R pixel configured as described above will be described. FIG. 15 is a diagram illustrating the relationship between the sensitivity characteristic with respect to the wavelength of light and the wavelength of each of the B pixel, the Cy pixel, and the R pixel. In FIG. 15, the vertical axis indicates sensitivity, and the horizontal axis indicates wavelength. In FIG. 15, each of a curve L _B (solid line), a curve L _Cy (one-dot chain line), and a curve L _R (dotted line) indicates the sensitivity characteristics of the B pixel, the Cy pixel, and the R pixel.

As shown by the curve L _B in FIG. 15, B pixel has a first peak sensitivity in the wavelength band of blue light (400nm ~ 470nm). Further, as shown by the curve L _Cy in FIG. 15, the Cy pixel has a first peak of sensitivity for green light (500 nm to 550 nm) and a second peak of sensitivity for blue light (400 nm to 470 nm). Have. Further, as shown by the curve L _R in Figure 15, R pixel has a first peak of sensitivity to red light (600nm ~ 650nm).

FIG. 16 is a diagram showing narrowband light and sensitivity characteristics of each pixel. In FIG. 16, the horizontal axis indicates the wavelength, and the vertical axis indicates the sensitivity.

As shown in FIG. 16, the Cy pixel has the highest sensitivity at each wavelength in the narrow band _TG . Further, the sensitivity at each wavelength in the narrow band T _B is, B pixels is highest.

The interpolation unit 411 performs an interpolation process based on the direction information of each of the B pixel, the Cy pixel, and the R pixel with respect to the image data generated by the imaging element 202a in which the color filter 202d configured as described above is arranged. Each B pixel and Cy pixels, the narrow-band light observation under for sensitive to narrowband T _B, by using the direction information of the Cy pixel, it is possible to generate a B-channel at high resolution. Further, the interpolation unit 411 may perform the interpolation process using the direction information of each pixel other than using the direction information of the Cy pixel.

[Processing of the extraction unit]
Next, processing performed by the extraction unit 412 will be described.
FIG. 17 is a diagram illustrating an outline of processing performed by the extraction unit 412 to extract the signal value generated by each narrowband light from the Cy channel. In FIG. 17, the interpolation values of the B channel and the Cy channel are respectively expressed as P _B and P _Cy . Further, the signal value of the narrow band T _B included in P _B is b, and the signal values of the narrow band T _B and the narrow band T _G included in P _Cy are b ′ and g ′.

As shown in FIG. 17, P _B and P _Cy at the coordinates (x, y) (pixel of interest) of the image sensor 202a are expressed by the following equations (3-1) and (3-2). .
P _B (x, y) = b (x, y) (3-1)
P _Cy (x, y) = g ′ (x, y) + b ′ (x, y) (3-2)
Further, b ', using the α coefficient obtained based on the sensitivity ratio of the respective wavelength for narrow band T _B of each B pixel and Cy pixel is expressed by the following equation (3-3).
b ′ (x, y) = αb (x, y) (3-3)
It can be expressed. From formula (3-2) and formula (3-3),
g ′ (x, y) = P _Cy (x, y) −αP _B (x, y) (3-4)

Here, as shown in FIG. 16, the sensitivity of the B pixels, at each wavelength component contained in the narrowband T _B, higher always than the sensitivity of the Cy pixel. Therefore, the coefficient α is 0 <α ≦ 1. Therefore, g ′ included in P _Cy can be obtained without amplifying the noise included in P _B by the above-described equation (3-4).

As described above, the extraction unit 412 obtains the surface information of the living tissue by obtaining the signal value b in all the pixels of the image sensor 202a using the above-described equations (3-1) and (3-3). Extract. Further, the extraction unit 412 extracts the mid-deep layer information by obtaining the signal value g ′ in all the pixels of the image sensor 202a using the above formula (3-4). Extraction unit 412 outputs the signal value b of the narrow band T _B of each B pixel and Cy pixel, the display image generating unit 42 a signal value g 'of narrow-band T _G of the G pixel.

[Processing of display image generation unit]
Next, processing of the display image generation unit 42 will be described.
The display image generation unit 42 generates a pseudo color image using the signal values b and g ′ output from the extraction unit 412. The pseudo color image is composed of three channels of R, B, and G. In generating the B channel and the G channel, the B channel may be generated first, and similar information may be assigned to the G channel.

The display image generation unit 42 assigns the signal value b input from the extraction unit 412 to each pixel position on the B channel and the G channel based on the color filter information input from the control unit 45, thereby A tissue surface information image is generated. Further, the display image generation unit 42 generates a mid-depth information image by assigning the signal value g ′ input from the extraction unit 412 to each pixel position on the R channel.

According to the third embodiment of the present invention described above, high-quality narrow-band signals are obtained from P _B and P _Cy by performing arithmetic processing that does not amplify noise as much as possible for the B channel and the Cy channel. Can be extracted. As a result, high-resolution images of the mucosal surface layer, middle layer, and deep layer at the observation site of the subject under narrow band light can be acquired while suppressing noise amplification.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, the sensitivity characteristics of each filter constituting the color filter 202b according to the first embodiment described above, and the processes of the extraction unit and the display image generation unit are different. In the following, after describing the sensitivity characteristics of each pixel according to the fourth embodiment, the processing executed by the extraction unit and the display image generation unit according to the fourth embodiment will be described. In addition, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and description is abbreviate | omitted.

FIG. 18 is a diagram showing narrowband light and sensitivity characteristics of each pixel according to Embodiment 4 of the present invention. In FIG. 18, the horizontal axis indicates the wavelength, and the vertical axis indicates the sensitivity. In FIG. 18, each of a curve L _B (solid line), a curve L _Cy (one-dot chain line), and a curve L _Mg (dotted line) indicates the sensitivity characteristics of the B pixel, the Cy pixel, and the Mg pixel.

As shown in FIG. 18, the Mg pixel is so small that the sensitivity at each wavelength of the narrow band _TG can be ignored.

[Processing of the extraction unit]
Next, processing performed by the extraction unit 412 will be described.
FIG. 19 is a diagram illustrating an outline of processing performed by the extraction unit 412 to extract the signal value generated by each narrow band light from the Cy channel and the Mg channel. In Figure 19, B-channel, the respective interpolated value of Cy channel and Mg channels, indicated as P _B, P _Cy and P _Mg. Further, the signal value of the narrow band T _B included in P _B is b, and the signal values of the narrow band T _B and the narrow band T _G included in P _Cy are b ′ and g ′, respectively, and are included in _PMg. The signal values of the narrow band T _B and the narrow band T _G are assumed to be b ″.

As shown in FIG. 18, each of P _B , P _Cy and P _Mg at the coordinates (x, y) (target pixel) of the image sensor 202a is expressed by the following equations (4-1) to (4-3). expressed.
P _B (x, y) = b (x, y) (4-1)
P _Cy (x, y) = g ′ (x, y) + b ′ (x, y) (4-2)
P _Mg (x, y) = b ′ ′ (x, y) (4-3)
Further, b ′ is expressed by the following equation (4-4) using a coefficient α obtained based on the sensitivity ratio of each wavelength to the narrow band TB of each of the B pixel and the Cy pixel.
b ′ (x, y) = αb (x, y) (4-4)

Here, as shown in FIG. 18, the sensitivity of the B pixels, at each wavelength component contained in the narrowband T _B, higher always than the sensitivity of the Cy pixel. Therefore, the coefficient α is 0 <α ≦ 1. Therefore, according to the above formulas (4-3) and (4-4),
b ′ (x, y) = αP _B (x, y) (4-5)
It becomes. From the above formulas (4-2) and (4-5),
g ′ (x, y) = P _Cy (x, y) −αP _B (x, y) (4-6)
It becomes.

Therefore, g ′ contained in P _Cy can be obtained without amplifying the noise contained in P _B by the above-described equations (4-5) and (4-6). Further, as shown in FIG. 18, Mg pixel, because it has no sensitivity to light of a narrow band T _G, under the narrow band light, can be treated as a value generated by light as narrow-band T _B .

As described above, the extraction unit 412 obtains the signal value b and the signal value b ″ in all the pixels of the image sensor 202a using the above-described equations (4-1) and (4-5). Extract surface layer information of living tissue. Further, the extraction unit 412 extracts the mid-deep layer information by obtaining the signal value g ′ in all the pixels of the image sensor 202a using the above-described formula (4-6). Extraction unit 412 outputs of each B pixel and Mg pixels' and the signal value g of the narrow band T _G of Cy pixel 'narrowband signal value b and the signal value b of T _B' to the display image generating unit 42 a .

[Processing of display image generation unit]
Next, processing of the display image generation unit 42 will be described.
The display image generation unit 42 generates a pseudo color image using the signal values b, b ″, g ′ output from the extraction unit 412. The pseudo color image is composed of three channels of R, B, and G. First, when generating the B channel and the G channel, the display image generation unit 42 may generate the B channel and assign the same information to the G channel.

Based on the color filter information input from the control unit 45, the display image generation unit 42 sets the signal value b and the signal value b ″ input from the extraction unit 412 to the position of the B pixel and the position of the Mg pixel, respectively. assign. In this case, the display image generation unit 42 multiplies b ″ by a coefficient k to make the brightness of the signal value b and the signal value b ″ uniform. The coefficient k can be expressed by the above equation (1-12) based on the sensitivity ratio between the B pixel and the Mg pixel. Thereafter, the display image generation unit 42 generates a surface layer information image using the signal value b at the position of the neighboring B pixel and the signal value b '' at the position of the Mg pixel at the position of the G pixel. Here, the display image generation unit 42 may use an interpolation process using direction information or a linear interpolation process. Further, the display image generation unit 42 may set the priority order without using the interpolation process, and may use either of the signal values b and b ″. The display image generation unit 42 generates the B channel. After that, the same information as the B channel is assigned to the G channel, and the display image generating unit 42 assigns the signal value g ′ input from the extracting unit 412 to the position of each pixel on the R channel, so that the intermediate depth information Generate an image.

According to the fourth embodiment of the present invention described above, B channels, by performing arithmetic processing as much as possible not to perform amplification of noise to Cy channels and Mg channels, high-quality narrowband P _Cy and P _Mg Can be extracted. As a result, high-resolution images of the mucosal surface layer, middle layer, and deep layer at the observation site of the subject under narrow band light can be acquired without amplifying noise.

(Other embodiments)
In addition to the arrangement described above, the color filter according to the present invention can be appropriately changed as long as the arrangement satisfies the above conditions.

In the present invention, the color filter having a plurality of transmission filters that each transmit light in a predetermined wavelength band is provided on the light receiving surface of the image sensor. However, each transmission filter corresponds to each pixel of the image sensor. May be provided individually. Specifically, in the present invention, a 2 × 2 filter unit has been described, but an n (integer) × m (integer) filter unit, for example, a 4 × 4 filter unit, can also be applied.

In addition, the endoscope according to the present invention is applicable to an ultrasonic endoscope in which an imaging element and an ultrasonic transducer are built in the tip, and a capsule endoscope that can be introduced into a subject. can do. When applied to a capsule endoscope, when switching between two light sources to emit either white illumination light or narrow-band illumination light, for example, a light source, a color filter, and an image sensor are provided in a capsule housing Just do it.

In the description of the processing in the present specification, the context is clearly shown using expressions such as “first”, “subsequent”, and “follow”, but the order of processing necessary to implement the present invention is described. Are not uniquely defined by their representation. That is, the order of processing in the processing described in this specification can be changed within a consistent range.

As described above, the present invention can include various embodiments not described herein, and various design changes can be made within the scope of the technical idea specified by the claims. It is.

DESCRIPTION OF SYMBOLS 1 Endoscope system 2 Endoscope 3 Light source device 4 Processor part 5 Display part 31 Illumination part 32 Illumination control part 41 Image processing part 42 Display image generation part 43 Input part 44 Storage part 45 Control part 71 Biological tissue 72 Capillary vessel 73 , 74 Thick blood vessel 200 Operation unit 201 Imaging lens 202 Imaging unit 202a Imaging device 202a Imaging device 202b, 202c, 202d Color filter 203 Light guide 204 Illumination lens 205 A / D conversion unit 206 Imaging information storage unit 206a Identification information storage unit 311 Light source Unit 312 light source driver 313 switching filter 314 drive unit 315 drive driver 316 condenser lens 411 interpolation unit 412 extraction unit

Claims (8)

1. A light source device that emits narrowband light including at least narrowband first light included in at least a blue wavelength band and narrowband second light included in a green wavelength band;
   An imaging device that generates an electrical signal by photoelectrically converting light received by each of a plurality of pixels arranged in a two-dimensional grid; and
   A first filter that transmits light in the blue wavelength band; a second filter that transmits light in the blue wavelength band and the green wavelength band; and a first filter that transmits light in at least the red wavelength band. A first filter unit including a plurality of filters, or a first filter that transmits light in the blue wavelength band, and light in the blue wavelength band and the red wavelength band. A second filter unit comprising a plurality of filters having a fourth filter that transmits light of the first wavelength and a fifth filter that transmits light of at least the green wavelength band, wherein the green wavelength The number of filters that transmit light in the band is more than half of the total number of filters, and the number of filters that transmit light in the blue wavelength band is less than the number of filters that transmit light in the green wavelength band. A color filter of the first filter unit or the second filter unit formed by arranging in correspondence to the plurality of pixels is,
   Computation processing for extracting a component of the narrowband light from a pixel value generated by each of the plurality of pixels, wherein the sensitivity to the first light among the plurality of pixels having sensitivity to the first light is The product of the pixel value of the pixel having the highest sensitivity to the first light and the first coefficient having an absolute value of 1 or less is subtracted from the pixel value of the pixel having the lowest sensitivity. An extraction unit that performs arithmetic processing to extract at least a component of the narrowband light obtained by the second light,
   An endoscope system comprising:
2. The extraction unit has a pixel value and an absolute value of 1 that are the highest in sensitivity to the second light among the plurality of pixels from a pixel value of the pixel that is sensitive to light in at least the red wavelength band. The endoscope system according to claim 1, wherein a component of the narrowband light obtained by the first light is further extracted by subtracting a product with a second coefficient which is the following.
3. The extraction unit extracts the narrowband light component obtained by the first light after extracting the narrowband light component obtained by the second light. The endoscope system described.
4. An interpolation unit that generates an interpolation value of a channel for each color by performing an interpolation process on a pixel value generated by each of the plurality of pixels;
   The endoscope according to any one of claims 1 to 3, wherein the extraction unit extracts the narrowband light component from the interpolation value generated by the interpolation unit performing the interpolation process. Mirror system.
5. The pixel formed by arranging the first filter has the highest sensitivity to the first light among the plurality of pixels,
   The pixel in which the second filter is disposed or the pixel in which the fifth filter is disposed has the highest sensitivity to the second light among the plurality of pixels. The endoscope system according to any one of claims 1 to 4.
6. The endoscopy according to any one of claims 1 to 5, further comprising a display image generation unit that generates a display image signal based on the narrowband light component extracted by the extraction unit. Mirror system.
7. Image processing that performs image processing on image data generated by an endoscope that includes an image sensor that generates an electrical signal by photoelectrically converting light received by each of a plurality of pixels arranged in a two-dimensional grid. A device,
   From the pixel values generated by each of the plurality of pixels, the first light in the narrow band included in the blue wavelength band and the second light in the narrow band included in the green wavelength band emitted by the light source device And a pixel value of the pixel having the lowest sensitivity to the first light among the plurality of pixels having sensitivity to the first light. From the plurality of pixels, by subtracting the product of the pixel value of the pixel having the highest sensitivity to the first light and the first coefficient having an absolute value of 1 or less, at least the second light An extraction unit that performs arithmetic processing to extract the component of the narrowband light obtained by
   The endoscope is
   A first filter that transmits light in the blue wavelength band; a second filter that transmits light in the blue wavelength band and the green wavelength band; and a first filter that transmits light in at least the red wavelength band. A first filter unit including a plurality of filters, or a first filter that transmits light in the blue wavelength band, and light in the blue wavelength band and the red wavelength band. A second filter unit comprising a plurality of filters having a fourth filter that transmits light of the first wavelength and a fifth filter that transmits light of at least the green wavelength band, wherein the green wavelength The number of filters that transmit light in the band is more than half of the total number of filters, and the number of filters that transmit light in the blue wavelength band is less than the number of filters that transmit light in the green wavelength band. The image processing apparatus according to the first filter unit or further comprising a color filter of the second filter unit formed by arranging in correspondence to the plurality of pixels is.
8. Image processing that performs image processing on image data generated by an endoscope that includes an image sensor that generates an electrical signal by photoelectrically converting light received by each of a plurality of pixels arranged in a two-dimensional grid. An image processing method executed by an apparatus,
   From the pixel values generated by each of the plurality of pixels, the first light in the narrow band included in the blue wavelength band and the second light in the narrow band included in the green wavelength band emitted by the light source device And a pixel value of the pixel having the lowest sensitivity to the first light among the plurality of pixels having sensitivity to the first light. From the plurality of pixels, by subtracting the product of the pixel value of the pixel having the highest sensitivity to the first light and the first coefficient having an absolute value of 1 or less, at least the second light An extraction step of performing an arithmetic process of extracting the narrowband light component obtained by:
   The endoscope is
   A first filter that transmits light in the blue wavelength band; a second filter that transmits light in the blue wavelength band and the green wavelength band; and a first filter that transmits light in at least the red wavelength band. A first filter unit including a plurality of filters, or a first filter that transmits light in the blue wavelength band, and light in the blue wavelength band and the red wavelength band. A second filter unit comprising a plurality of filters having a fourth filter that transmits light of the first wavelength and a fifth filter that transmits light of at least the green wavelength band, wherein the green wavelength The number of filters that transmit light in the band is more than half of the total number of filters, and the number of filters that transmit light in the blue wavelength band is less than the number of filters that transmit light in the green wavelength band. Image processing method characterized by the first filter unit or the second filter unit is provided with a color filter formed by arranging in correspondence to the plurality of pixels.

Images