WO2019180983A1

WO2019180983A1 - Endoscope system, image processing method, and program

Info

Publication number: WO2019180983A1
Application number: PCT/JP2018/029313
Authority: WO
Inventors: 理足立
Original assignee: オリンパス株式会社
Priority date: 2018-03-23
Filing date: 2018-08-03
Publication date: 2019-09-26

Abstract

Provided are an endoscope system, an image processing method, and a program that are capable of obtaining high-quality, color drift-free color images and special images. The endoscope system 1 comprises: an imaging element 244 having a color filter 2442 laminated upon the light-reception surface of each pixel and capable of generating image data by capturing images of a subject; a light source 31 capable of irradiating light in a first wavelength band and light in a second wavelength band; an illumination control unit 33 that causes the light source 31 to alternately irradiate light in the first wavelength band and light in the second wavelength band; and an image processing unit 51 that synthesizes first image data generated by the imaging element 244 when the light source 31 irradiates light in the first wavelength band and second image data generated by the imaging element 244 when the light source 31 irradiates light in the second wavelength band.

Description

Endoscope system, image processing method and program

The present disclosure relates to an endoscope system, an image processing method, and a program that are introduced into a living body of a subject and capture an image in the living body.

2. Description of the Related Art Conventionally, in an imaging apparatus such as a stomach camera, white and green transmission filters are provided between a light source and a subject, and yellow and cyan color separation filters are arranged in a mosaic pattern on the imaging light-receiving surface, and white or green Technology to prevent color shifting by obtaining one high-quality color image in two fields, compared to the conventional case of obtaining one color image in three fields by switching the transmission filter Is known (see Patent Document 1).

In recent years, as an endoscope observation method, a white light observation method (WLI: White Light Imaging) using white illumination light (white light) and two narrow bands included in the blue and green wavelength bands, respectively. Special light observation methods such as a narrow band light observation method (NBI: Narrow Band Imaging) using illumination light (narrow band light) made of light are widely known. In the white light observation method, a color image can be obtained by irradiating white light. In the narrow band light observation method, by irradiating narrow band light, it is possible to obtain a special image that highlights capillaries and mucous membrane fine patterns existing on the surface of the mucosa of the living body.

JP-A-6-225316

Incidentally, in recent years, it has been desired for endoscopes to obtain color images and special images. However, in Patent Document 1 described above, only a color image is considered, and no consideration is given to obtaining a special image. For this reason, a technique capable of obtaining each of a color image and a special image with high image quality and no color shift has been desired.

The present disclosure has been made in view of the above, and an object of the present disclosure is to provide an endoscope system, an image processing method, and a program capable of obtaining each of a color image and a special image with high image quality and no color misregistration. And

In order to solve the above-described problems and achieve the object, an endoscope system according to the present disclosure includes a red wavelength band light, a green wavelength band light, and a blue wavelength band light. An image sensor that is formed by stacking color filters configured using two types of filters that transmit light of two different wavelength bands on the light receiving surface of each pixel, and that can generate image data by imaging a subject; Light in a first wavelength band that includes light in at least one of the two wavelength bands and light in a second wavelength band that includes light in at least the other wavelength band of the two wavelength bands A light source unit that can irradiate the light source unit, an illumination control unit that alternately irradiates light of the first wavelength band and light of the second wavelength band to the light source unit, and the light source unit has the first wavelength The imaging when irradiated with light of the band An image processing unit that synthesizes the first image data generated by the child and the second image data generated by the imaging device when the light source unit emits light of the second wavelength band; It is characterized by providing.

Further, in the endoscope system according to the present disclosure, in the above disclosure, the two types of filters include a cyan filter that transmits light in the green wavelength band and light in the blue wavelength band, and a red wavelength band. A yellow filter that transmits light and light in the green wavelength band, and the light source unit emits light in the red wavelength band and first light source that can emit light in the green wavelength band Two light sources and a third light source capable of irradiating light in the blue wavelength band, and the illumination control unit supplies the second light source with light in the green wavelength band as light in the first wavelength band. While irradiating light, as the light of the second wavelength band, the first light source is simultaneously irradiated with light of the red wavelength band and the third light source of light of the blue wavelength band, and the image processing unit Is included in the second image data. A separation unit that generates red image data and blue image data by separating the pixel value of the light in the red wavelength band and the pixel value of the light in the blue wavelength band from the corresponding second image; And a combining unit that generates white image data by combining the red image data generated by the separating unit, the blue image data, and the first image data.

In the endoscope system according to the present disclosure, in the above disclosure, the combining unit generates special image data by combining the blue image data generated by the separating unit and the first image data. It is characterized by doing.

Further, in the endoscope system according to the present disclosure, in the above disclosure, the two types of filters include a cyan filter that transmits light in the green wavelength band and light in the blue wavelength band, and a red wavelength band. A magenta filter that transmits light and light in the blue wavelength band, wherein the light source unit includes a first light source capable of irradiating light in the red wavelength band, and a first light source capable of irradiating light in the green wavelength band. Two light sources and a third light source capable of irradiating light in the blue wavelength band, and the illumination control unit supplies the third light source with light in the blue wavelength band as light in the first wavelength band. While irradiating light, as the light of the second wavelength band, the first light source is simultaneously irradiated with light of the red wavelength band and light of the green wavelength band to the second light source, and the image processing unit Is included in the second image data. A separation unit that generates red image data and green image data by separating the pixel value of the light in the red wavelength band and the pixel value of the light in the green wavelength band from the corresponding second image; And a combining unit that generates white image data by combining the red image data generated by the separation unit, the green image data, and the first image data.

In the endoscope system according to the present disclosure, in the above disclosure, the synthesis unit generates special image data by synthesizing the green image data separated by the separation unit and the first image data. It is characterized by doing.

In addition, in the endoscope system according to the present disclosure, in the above disclosure, when the light source unit emits light in the first wavelength band, a pixel value of each pixel constituting the imaging element is set to a predetermined number of pixels. Pixels of pixels that can receive light of the same wavelength band among the pixels that constitute the image sensor when the light source unit emits light of the second wavelength band while adding and outputting each time An imaging control unit that adds and outputs a value for each predetermined number of pixels is provided.

Further, in the endoscope system according to the present disclosure, in the above disclosure, the light source unit includes a fourth light source capable of emitting light in a purple wavelength band, and a fifth light source capable of emitting light in an orange wavelength band. The illumination control unit irradiates the light of the second wavelength band to the light source unit, and then causes the fourth light source to emit light of the purple wavelength band and the fifth light source to orange. It is characterized in that the light of the wavelength band is simultaneously emitted.

Further, the image processing method according to the present disclosure transmits two kinds of light in two different wavelength bands among light in the red wavelength band, light in the green wavelength band, and light in the blue wavelength band. A color filter configured using a filter is stacked on the light-receiving surface of each pixel, and an image sensor that can generate image data by imaging a subject, and at least one wavelength band of light in the two wavelength bands And a light source unit capable of irradiating light in a first wavelength band including the light in the second wavelength band and light in a second wavelength band including light in at least the other wavelength band of the light in the two wavelength bands. A control method executed by an endoscope system, wherein the light source unit alternately irradiates light of the first wavelength band and light of the second wavelength band to the light source unit, and the light source unit Irradiates light of 1 wavelength band The first image data generated by the image sensor when the light source unit emits light of the second wavelength band, and the second image data generated by the image sensor And an image processing step.

Further, the program according to the present disclosure includes two types of filters that transmit light in two different wavelength bands among light in a red wavelength band, light in a green wavelength band, and light in a blue wavelength band. A color filter formed using the light-receiving surface of each pixel, and an image sensor capable of generating image data by imaging a subject, and light in at least one wavelength band of the two wavelength bands And a light source unit capable of irradiating light in a first wavelength band including light and light in a second wavelength band including light in at least the other wavelength band of the two wavelength bands. An illumination control step for causing the light source unit to alternately irradiate the light of the first wavelength band and the light of the second wavelength band to the mirror system, and the light source unit irradiated the light of the first wavelength band When the image sensor An image processing step of combining the generated first image data and the second image data generated by the imaging device when the light source unit emits light of the second wavelength band. It is made to perform.

According to the present disclosure, it is possible to obtain each of a color image and a special image with high image quality and no color misregistration.

FIG. 1 is a schematic configuration diagram of an endoscope system according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the first embodiment. FIG. 3 is a diagram schematically showing the configuration of the color filter. FIG. 4 is a diagram schematically illustrating spectral characteristics of the color filter and each light emitted from the light source device. FIG. 5 is a diagram illustrating an example of an image generated by the imaging device when the light source device emits light in the green wavelength band. FIG. 6 is a diagram illustrating an example of an image generated by the imaging device when the light source device simultaneously emits light in the red wavelength band and light in the blue wavelength band. FIG. 7 is a flowchart illustrating an outline of processing executed by the endoscope system according to the first embodiment. FIG. 8 is a diagram schematically illustrating an image generated by the image processing unit. FIG. 9 is a diagram schematically illustrating a configuration of a color filter according to a modification of the first embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of an image generated by the imaging device when the light source device emits light in the green wavelength band. FIG. 11 is a diagram illustrating an example of an image generated by the imaging device when the light source device simultaneously emits light in the red wavelength band and light in the blue wavelength band. FIG. 12 is a circuit diagram schematically showing the configuration of the image sensor according to the second embodiment. FIG. 13 is a diagram schematically illustrating an image sensor addition method. FIG. 14 is a diagram schematically illustrating an image sensor addition method. FIG. 15 is a diagram schematically illustrating an image sensor addition method. FIG. 16 is a diagram schematically illustrating a configuration of a color filter according to the third embodiment. FIG. 17 is a diagram illustrating an example of an image generated by the imaging device when the light source device simultaneously emits light in the green wavelength band and light in the red wavelength band. FIG. 18 is a diagram illustrating an example of an image generated by the imaging device when the light source device emits light in a blue wavelength band. FIG. 19 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the fourth embodiment. FIG. 20 is a diagram schematically illustrating spectral characteristics of the color filter 2442 and each light emitted from the light source device 3C. FIG. 21 is a diagram illustrating an example of an image generated by the imaging device when the light source device emits light in the green wavelength band. FIG. 22 is a diagram illustrating an example of an image generated by the imaging device when the light source device simultaneously emits light in the blue wavelength band and light in the red wavelength band. FIG. 23 is a diagram illustrating an example of an image generated by the imaging device when the light source device emits light in a purple wavelength band and light in an orange wavelength band. FIG. 24 is a timing chart illustrating an outline of processing executed by the endoscope system according to the fourth embodiment.

Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as “embodiments”) will be described. In the embodiment of the present disclosure, a medical endoscope system that captures and displays an image of a body cavity of a subject such as a patient will be described. Moreover, this invention is not limited by this embodiment. Furthermore, in the description of the drawings, the same portions will be described with the same reference numerals.

(Embodiment 1)
[Configuration of endoscope system]
FIG. 1 is a schematic configuration diagram of an endoscope system according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the first embodiment. An endoscope system 1 shown in FIGS. 1 and 2 inserts an endoscope into a subject such as a patient, images the inside of the subject, and outputs the captured image data to an external display device. A user such as a doctor examines the presence or absence of each of a bleeding site, a tumor site, and an abnormal site, which are detection target sites, by observing the in-vivo images displayed on the display device. The endoscope system 1 includes an endoscope 2, a light source device 3, a display device 4, and a processing device 5 (processor).

[Configuration of endoscope]
First, the configuration of the endoscope 2 will be described.
The endoscope 2 captures an image of the inside of the subject, generates image data (RAW data), and outputs the generated image data to the processing device 5. The endoscope 2 includes an insertion unit 21, an operation unit 22, and a universal cord 23.

The insertion part 21 has an elongated shape having flexibility. The insertion portion 21 is connected to a distal end portion 24 incorporating an imaging element 244 described later, a bendable bending portion 25 constituted by a plurality of bending pieces, and a proximal end side of the bending portion 25, and has flexibility. And a long flexible tube portion 26.

The distal end portion 24 is configured using a glass fiber or the like, and forms a light guide path for light emitted from the light source device 3. An illumination lens 242 provided at the distal end of the light guide 241. A system 243, and an image pickup device 244 provided at an image forming position of the optical system 243, in which a plurality of pixels that the optical system 243 collects light, receives light, and photoelectrically converts it into an electrical signal are arranged two-dimensionally; An endoscope recording unit 245 that records various types of information related to the endoscope 2 and an imaging control unit 246 that controls the imaging element 244 are provided.

The image sensor 244 is configured using an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Specifically, the imaging element 244 receives a light and performs photoelectric conversion to arrange a plurality of pixels that output an electrical signal in a two-dimensional array, and images a subject (body cavity) at a predetermined frame rate. Image data (RAW data) is output. The imaging element 244 includes a pixel portion 2441 and a color filter 2442.

The pixel portion 2441 is formed by arranging a plurality of pixels each having a photodiode for accumulating charges according to the amount of light and an amplifier for amplifying the charges accumulated by the photodiodes in a two-dimensional matrix.

The color filter 2442 includes a cyan filter (hereinafter simply referred to as a “Cy filter”) that transmits light in a green wavelength band (500 nm to 600 nm) and light in a blue wavelength band (390 nm to 500 nm), and a green wavelength band. And a yellow filter (hereinafter simply referred to as “Ye filter”) that transmits light and light in a red wavelength band (600 nm to 700 nm). Specifically, as shown in FIG. 3, the color filter 2442 is formed by a filter unit U1 using two Cy filters and two Ye filters, and the filter unit U1 is composed of two vertical and horizontal units (2 × 2). It is formed. The color filter 2442 is formed by arranging a Ye filter and a Cy filter in a checkered pattern. As the color filter 2442, a Cy filter or a Ye filter is stacked on the light receiving surface of each pixel of the pixel portion 2441. The spectral characteristics of the Cy filter and Ye filter will be described later.

The endoscope recording unit 245 records various information related to the endoscope 2. For example, the endoscope recording unit 245 records identification information for identifying the endoscope 2, identification information for the imaging element 244, and the like. The endoscope recording unit 245 is configured using a nonvolatile memory or the like.

The imaging control unit 246 controls the operation of the imaging element 244 based on the instruction information input from the processing device 5. Specifically, the imaging control unit 246 controls the frame rate and imaging timing of the imaging element 244 based on the instruction information input from the processing device 5. For example, the imaging control unit 246 causes the imaging element 244 to generate and output image data at 120 fps.

The operation unit 22 includes a bending knob 221 that bends the bending unit 25 in the vertical direction and the left-right direction, a treatment tool insertion unit 222 that inserts a treatment tool such as a biological forceps, a laser knife, and an inspection probe into the body cavity, and the light source device 3. In addition to the processing device 5, a plurality of switches that are operation input units for inputting operation instruction signals of peripheral devices such as an air supply means, a water supply means, and a gas supply means and a pre-freeze signal that instructs the image pickup device 244 to take a still image 223. The treatment tool inserted from the treatment tool insertion portion 222 is exposed from the opening (not shown) via the treatment tool channel (not shown) of the distal end portion 24.

The universal cord 23 includes at least a light guide 241 and a condensing cable in which one or a plurality of cables are collected. The collective cable is a signal line for transmitting and receiving signals between the endoscope 2 and the light source device 3 and the processing device 5, and is a signal line for transmitting and receiving setting data, a signal line for transmitting and receiving image data, A signal line for transmitting and receiving a driving timing signal for driving the image sensor 244 is included. The universal cord 23 has a connector portion 27 that can be attached to and detached from the light source device 3. The connector 27 includes a coiled coil cable 27a and a connector 28 that can be attached to and detached from the processing apparatus 5 at the extended end of the coil cable 27a.

[Configuration of light source device]
Next, the configuration of the light source device 3 will be described.
The light source device 3 supplies illumination light for irradiating the subject from the distal end portion 24 of the endoscope 2. The light source device 3 includes a light source unit 31, a light source driver 32, and an illumination control unit 33.

The light source unit 31 emits illumination light that irradiates the subject. Specifically, the light source unit 31 has at least one wavelength band of two different wavelength bands among the light in the red wavelength band, the light in the green wavelength band, and the light in the blue wavelength band. At least the other of the light in the first wavelength band including light, the light in the red wavelength band, the light in the green wavelength band, and the light in the two wavelength bands different from each other in the blue wavelength band And light in a second wavelength band including light in the wavelength band. The light source unit 31 includes a condenser lens 311, a first light source 312, a second light source 313, and a third light source 314.

The condensing lens 311 is configured using one or a plurality of lenses. The condensing lens 311 condenses the illumination light emitted from each of the first light source 312, the second light source 313, and the third light source 314 and emits it to the light guide 241.

The first light source 312 is configured using a red LED (Light Emitting Diode) lamp. The first light source 312 emits light in the red wavelength band (hereinafter simply referred to as “R light”) based on the current supplied from the light source driver 32.

The second light source 313 is configured using a green LED lamp. The second light source 313 emits light in the green wavelength band (hereinafter simply referred to as “G light”) based on the current supplied from the light source driver 32.

The third light source 314 is configured using a blue LED lamp. The third light source 314 emits light in the blue wavelength band (hereinafter simply referred to as “B light”) based on the current supplied from the light source driver 32.

The light source driver 32 emits only G light to the second light source 313 by supplying current to the first light source 312, the second light source 313, and the third light source 314 under the control of the illumination control unit 33. In addition, by causing the first light source 312 and the third light source 314 to emit R light and B light simultaneously, magenta light (hereinafter simply referred to as “Mg light”) including R light and B light is emitted. Let In the first embodiment, G light corresponds to light in the first wavelength band, and Mg light corresponds to light in the second wavelength band.

The illumination control unit 33 controls the lighting timing of the light source unit 31 based on the instruction signal received from the processing device 5. Specifically, the illumination control unit 33 supplies G light to the endoscope 2 by causing the second light source 313 to emit G light at a predetermined cycle, while supplying R light and a third light source to the first light source 312. Mg light is supplied to the endoscope 2 by causing the 314 to emit B light simultaneously. In this case, the illumination control unit 33 causes the light source unit 31 to emit G light and Mg light alternately and intermittently. The illumination control unit 33 is configured using a CPU (Central Processing Unit) or the like.

[Configuration of display device]
Next, the configuration of the display device 4 will be described.
The display device 4 displays an image corresponding to the image data generated by the endoscope 2 received from the processing device 5. The display device 4 displays various information related to the endoscope system 1. The display device 4 is configured using a display panel such as liquid crystal or organic EL (Electro Luminescence).

[Configuration of processing equipment]
Next, the configuration of the processing device 5 will be described.
The processing device 5 receives the image data generated by the endoscope 2, performs predetermined image processing on the received image data, and outputs it to the display device 4. Further, the processing device 5 comprehensively controls the operation of the entire endoscope system 1. The processing device 5 includes an image processing unit 51, an input unit 52, a recording unit 53, and a processing control unit 54.

The image processing unit 51 receives the image data generated by the endoscope 2 under the control of the processing control unit 54, performs predetermined image processing on the received image data, and outputs the image data to the display device 4. Specifically, the image processing unit 51 includes the first image data generated by the image sensor 244 when the light source unit 31 emits G light, and the image sensor 244 when the light source unit 31 emits Mg light. And the second image data generated by the above. Here, as predetermined image processing, separation processing, composition processing, interpolation processing, OB clamping processing, gain adjustment processing, format conversion processing, and the like are performed. The image processing unit 51 is configured using a GPU (Graphics Processing Unit), a DSP (Digital Signal Processing), or an FPGA (Field Programmable Gate Array). The image processing unit 51 includes at least a separation unit 511, an interpolation unit 512, and a synthesis unit 513.

The separation unit 511 performs color separation on the image corresponding to the image data generated by the endoscope 2. Specifically, the separation unit 511 separates the pixel value of the R pixel and the pixel value of the B pixel from the image corresponding to the image data generated by the image sensor 244 when the light source device 3 emits Mg light.

The interpolation unit 512 performs known interpolation processing using the pixel values of the R pixels separated from the RB image by the separation unit 511, thereby generating R image data in which the pixel values of the R pixels are interpolated for all the pixels. In addition, the interpolation unit 512 generates B image data having pixel values of B pixels in all pixels by performing a known interpolation process using the pixel values of B pixels separated from the RB image by the separation unit 511. Here, the interpolation processing includes bilinear interpolation processing, direction discrimination interpolation processing, and the like. Of course, the interpolation unit 512 may generate the R image data and the B image data using another interpolation process.

When the endoscope system 1 performs the white light observation method (normal observation), the synthesizing unit 513 performs the imaging element 244 when the R image data and B image data generated by the interpolation unit 512 and the light source device 3 emit G light. The G image data generated by is generated to generate color image data, and the generated color image data is output to the display device 4. Further, when the endoscope system 1 performs the NBI observation method, the N image is generated by synthesizing the G image data generated by the imaging device 244 when the light source device 3 emits the G light with the B image data generated by the interpolation unit 512. Image data is generated, and the generated NBI image data is output to the display device 4.

The input unit 52 receives an input of an instruction signal for instructing the operation of the endoscope system 1 and outputs the received instruction signal to the processing control unit 54. For example, the input unit 52 receives an input of an instruction signal instructing either the white light observation method or the NBI observation method, and outputs the received instruction signal to the processing control unit 54. The input unit 52 is configured using switches, buttons, a touch panel, and the like.

The recording unit 53 records various programs executed by the endoscope system 1, data being executed by the endoscope system 1, and image data generated by the endoscope 2. The recording unit 53 is configured using a volatile memory, a nonvolatile memory, a memory card, and the like. The recording unit 53 includes a program recording unit 531 that records various programs executed by the endoscope system 1.

The process control unit 54 is configured using a CPU. The process control unit 54 controls each unit constituting the endoscope system 1. For example, when the instruction signal for switching the illumination light emitted from the light source device 3 is input from the input unit 52, the processing control unit 54 controls the illumination control unit 33 to switch the illumination light emitted from the light source device 3. .

[Spectral characteristics of color filters and illumination light]
Next, spectral characteristics of each light emitted from the color filter 2442 and the light source device 3 will be described. FIG. 4 is a diagram schematically showing the spectral characteristics of the color filter 2442 and each light emitted from the light source device 3. In FIG. 4, the horizontal axis indicates the wavelength, the right vertical axis indicates the sensitivity of each pixel (filter transmittance), and the left vertical axis indicates the intensity of each light. In FIG. 4, a curve L _Cy indicates the spectral sensitivity characteristic of the Cy pixel, and a curve L _Ye indicates the spectral sensitivity characteristic of the Ye pixel. The curve L _B represents a wavelength characteristic of the B light, the curve L _G represents a wavelength characteristic of the G light, the curve L _R represents the wavelength characteristic of the R light.

As shown by the curve L _Ye in FIG. 4, the Ye pixel has sensitivity to R light and G light (the transmittance of the filter is high). The Cy pixel has sensitivity to B light and G light (filter transmittance is high). Therefore, as shown in FIG. 5, when the light source apparatus 3 is emitted from the G light, the image pickup device 244 can generate an image P _G all the pixels have a G pixel. As shown in FIG. 6, when the light source device 3 emits Mg, the image sensor 244 can generate an image P _{Mg in} which R pixels and B pixels are in a checkered pattern. Thus, in the endoscope system 1, under the control of the processing device 5, the light source device 3 supplies G light or Mg light to the endoscope 2, and the imaging device 244 is irradiated with G light or Mg light. The subject is sequentially imaged.

[Endoscope system processing]
Next, processing executed by the endoscope system 1 will be described. FIG. 7 is a flowchart showing an outline of processing executed by the endoscope system 1. FIG. 8 is a diagram schematically illustrating an image generated by the image processing unit 51.

As shown in FIG. 7, first, the process control unit 54 causes the light source device 3 to emit G light (step S101), and causes the image sensor 244 to image the subject irradiated with the G light (step S102). In this case, as shown in FIG. 8, the image pickup device 244 generates an image P _G corresponding to the image data by imaging the subject G light is irradiated.

Subsequently, the process control unit 54 causes the light source device 3 to emit Mg light (step S103), and causes the image sensor 244 to image the subject irradiated with the Mg light (step S104). In this case, as illustrated in FIG. 8, the imaging element 244 generates an image _PMg corresponding to the image data by imaging the subject irradiated with the Mg light.

Thereafter, the separation unit 511 performs color separation of the pixel value of the R pixel and the pixel value of the B pixel on the image _PMg (Step S105). In this case, as illustrated in FIG. 8, the separation unit 511 separates the pixel value of the R pixel from the image P _Mg corresponding to the image data generated by the image sensor 244 and the image P _R1 and the image P of the R pixel. An image P _{B1 of} B pixels obtained by separating the pixel values of B pixels from _Mg is generated.

Subsequently, the interpolation unit 512 performs interpolation processing on each of the image _{P R1} and the image _{P B1} (step S106). Specifically, as illustrated in FIG. 8, the interpolation unit 512 performs an interpolation process for interpolating the pixel values of the other R pixels using the pixel values of the image _PR1 , thereby _causing the pixel values of the R pixels to be applied to all the pixels. Is generated as an interpolated image _PR2 . Further, the interpolation unit 512 generates an image P _B2 in which the pixel values of B pixels are interpolated in all pixels by performing an interpolation process that interpolates the pixel values of other B pixels using the pixel values of the image P _B1. .

Thereafter, when the white light observation method is set in the endoscope system 1 (step S107: Yes), the synthesis unit 513 generates a white image (step S108). Specifically, as illustrated in FIG. 8, the combining unit 513 generates a white image P _W (color image) by combining the image P _R2 , the image P _B2, and the image G. As a result, the endoscope system 1 can generate a white image in two frames (two fields) by two irradiations of G light and Mg light.

Subsequently, when an instruction signal for instructing termination is input from the operation unit 22 or the input unit 52 (step S109: Yes), the endoscope system 1 ends this process. On the other hand, when the instruction signal for instructing the end is not input from the operation unit 22 or the input unit 52 (step S100: No), the endoscope system 1 returns to step S101.

In step S107, when the white observation method is not set in the endoscope system 1 (step S107: No), the synthesis unit 513 generates an NBI image (step S110). Specifically, as shown in FIG. 8, the synthesis unit 513 generates an NBI image _{P NBI} by combining the images _{P G} and the image _{P B2.} As a result, the endoscope system 1 can generate an NBI image P _NBI in two frames (two fields) by two irradiations of G light and Mg light. In the first embodiment, either one of the white image and the NBI image is generated according to the setting of the endoscope system 1. However, the present invention is not limited to this, and the white image and the NBI image are simultaneously generated. It may be generated. In the first embodiment, the white image and the NBI image may be simultaneously displayed on the display device 4, or the reduced NBI image may be superimposed on the white image and displayed on the display device 4, or the NBI may be displayed. A reduced white image may be superimposed on the image and displayed on the display device 4. After step S110, the endoscope system 1 proceeds to step S109.

According to the first embodiment described above, the first image data generated by the image sensor 244 when the image processing unit 51 is irradiated with G light by the light source device 3, and Mg light is irradiated by the light source device 3. And the second image data generated by the image sensor 244 at the time. Thereby, the endoscope system 1 can generate a white image and an NBI image in two frames (two fields) by two irradiations of G light and Mg light.

(Modification of Embodiment 1)
Next, a modification of the first embodiment will be described. The modification of the first embodiment is different in configuration from the color filter 2442 according to the first embodiment described above. Hereinafter, a configuration of the color filter according to the modification of the first embodiment will be described. In addition, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and detailed description is abbreviate | omitted suitably.

[Color filter configuration]
FIG. 9 is a diagram schematically illustrating a configuration of a color filter according to a modification of the first embodiment.

The color filter 2442A shown in FIG. 9 is formed by a filter unit U2 using one Ye filter and three Cy filters, and this filter unit U2 is formed by two vertical and horizontal units (2 × 2). The color filter 2442A is arranged such that the number of Ye filters is smaller than the number of Cy filters. In the color filter 2442A, each of the Ye filter and the Cy filter is stacked on the light receiving surface of each pixel of the pixel portion 2441.

In the endoscope system 1 configured as described above, the light source device 3 supplies G light or Mg light to the endoscope 2 under the control of the processing device 5, and the imaging element 244 is G light or Mg light. The irradiated subject is sequentially imaged. The Ye filter and the Cy filter transmit G light. Therefore, as shown in FIG. 10, the image pickup device 244 can generate an image P _G corresponding to the image data. The Ye filter transmits R light, and the Cy filter transmits B light. For this reason, as shown in FIG. 11, the image sensor 244 can generate an image P _Mg2 corresponding to image data composed of R pixels and B pixels. R light has few high frequency components in the spatial frequency. For this reason, the color filter 2442A is provided with a 1/4 Ye filter as a whole, so that the resolution of the color signal can be optimized. Furthermore, since the color filter 2442A is provided with a 3/4 Cy filter as a whole, a high-resolution NBI image can be obtained when the narrow-band light observation method is performed.

According to the modification of the first embodiment described above, the 3/4 Cy filter is provided for the entire color filter 2442A. Therefore, when performing the narrow-band light observation method, a high-resolution NBI image is obtained. be able to.

(Embodiment 2)
Next, a second embodiment will be described. In the second embodiment, resolution and sensitivity are improved by pixel addition in which a predetermined number of same color pixels are added. Below, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and description is abbreviate | omitted.

FIG. 12 is a circuit diagram schematically showing the configuration of the image sensor according to the second embodiment. An imaging element 244B illustrated in FIG. 12 includes a pixel portion 2441B and a color filter 2442. In FIG. 12, the color filter 2442 is indicated by a dotted line, and is stacked on the light receiving surface of each pixel.

The pixel portion 2441B shown in FIG. 12 is 8 pixels and shares the charge conversion portion, the reset transistor, the amplification transistor, and the selection transistor. Specifically, the pixel portion 2441B illustrated in FIG. 12 includes eight photoelectric conversion elements 301 (photodiodes), a transfer transistor 302 provided in each of the eight photoelectric conversion elements 301, a charge conversion unit 303, and a reset. The pixel includes a transistor 304, a pixel source follower transistor 305, a selection transistor 306, and a vertical transfer line 307.

The photoelectric conversion element 301 photoelectrically converts incident light into a signal charge amount corresponding to the amount of light and accumulates it. The photoelectric conversion element 301 has a cathode side connected to one end side of the transfer transistor 302 and an anode side connected to the ground.

The transfer transistor 302 transfers charges from the photoelectric conversion element 301 to the charge conversion unit 303. The transfer transistor 302 has a gate connected to a signal line to which a drive signal ΦTn (n = 1, 2, 3,..., N−1, n) is supplied, a photoelectric conversion element 301 connected to one end side, and the other end side. The charge conversion unit 303 is connected to. The transfer transistor 302 is turned on when a drive signal ΦT is supplied from a vertical scanning unit (not shown) via a signal line, and transfers charge from the photoelectric conversion element 301 to the charge conversion unit 303.

The charge conversion unit 303 includes a floating diffusion capacitor (FD), and converts the charge accumulated in the photoelectric conversion element 301 into a voltage.

The reset transistor 304 resets the charge conversion unit 303 to a predetermined potential. The reset transistor 304 has one end connected to the power supply voltage VDD, the other end connected to the charge converter 303, and a gate connected to a signal line to which the drive signal ΦR is supplied. The reset transistor 304 is turned on when a drive signal ΦR is supplied from a vertical scanning unit (not shown) via a signal line, and releases the signal charge accumulated in the charge conversion unit 303, thereby causing the charge conversion unit 303 to be predetermined. Reset to potential.

The pixel source follower transistor 305 has one end connected to the selection transistor 306, the other end connected to the vertical transfer line 307, and a gate connected to a signal line for transmitting a signal (image signal) converted by the charge converter 303. Is done.

The selection transistor 306 has one end connected to the power supply voltage VDD, the other end connected to the pixel source follower transistor 305, and a gate connected to a signal line to which a row selection signal ΦX is supplied. The selection transistor 306 is turned on when the row selection signal ΦX is supplied, and transfers the signal (image signal) output from the pixel source follower transistor 305 to the vertical transfer line 307.

[Driving method of image sensor]
Next, a method for driving the image sensor 244B will be described.
First, the case of progressive reading (sequential scanning) of the image sensor 244B will be described. When the image sensor 244B performs progressive reading, the light source device 3 emits G light. For this reason, the image sensor 244B images the subject irradiated with the G light.

As illustrated in FIG. 12, the imaging control unit 246 inputs a transfer pulse to the signal line ΦT1 and reads the signal charges of the Cy pixels in the first row. Subsequently, the imaging control unit 246 inputs a transfer pulse to the signal line ΦT2 and reads the signal charge of the Ye pixels in the first row. In this manner, the imaging control unit 246 sequentially reads out the signal charges for each row of the imaging element 244B. In this case, as shown in FIG. 12, the imaging element 244B, since the light source apparatus 3 is emitted G light, it is possible that all the pixels to produce an image P _G is a G pixel. At this time, as shown in FIG. 13, the combining unit 513 _combines four G pixels as one G pixel _PGA1 . As a result, a G image with increased sensitivity can be generated.

Next, the case of pixel mixture readout of the image sensor 244B will be described. Note that, when the image sensor 244B performs pixel mixture readout, the light source device 3 emits Mg light. For this reason, the image sensor 244B images the subject irradiated with the Mg light.

As illustrated in FIG. 12, the imaging control unit 246 inputs a transfer pulse to the signal line ΦT1 and the signal line ΦT4, and reads the signal charges of the Cy pixels in the first row and the signal charges of the Cy pixels in the second row. Subsequently, the imaging control unit 246 inputs a transfer pulse to the signal line ΦT2 and the signal line ΦT3, and reads the signal charge of the Ye pixel in the first row and the signal charge of the Ye pixel in the second row. As described above, the imaging control unit 246 reads out signal charges from pixels of the same color every two rows of the imaging element 244B. In this case, as shown in FIGS. 14 and 15, the imaging element 244B, since the light source apparatus 3 is emitted Mg light, it is possible to generate an image P _Mg1 which R and B pixels is checkered pattern . At this time, as illustrated in FIGS. 14 and 15, the combining unit 513 combines two pixels of the same color that are diagonally adjacent to each other as one pixel. For example, as illustrated in FIG. 14, the combining unit 513 combines two diagonally adjacent R pixels as one R pixel _PRA1 . Further, as illustrated in FIG. 15, the combining unit 513 combines two diagonally adjacent B pixels as one B pixel P _BA1 . Thereby, it is possible to generate an R image and a B image with increased sensitivity.

According to the second embodiment described above, when the imaging control unit 246 is irradiated with G light from the light source device 3, the pixel values of the G pixels constituting the imaging element 244B are added for each predetermined number of pixels and output. Let Furthermore, the pixel value of the B pixel or the pixel value of the R pixel that can receive light in the same wavelength band among the pixels that constitute the imaging element 244B when the imaging control unit 246 is irradiated with Mg light by the light source device 3. Are added for each predetermined number of pixels and output. As a result, it is possible to generate a white image and an NBI image with increased sensitivity.

In the second embodiment, the imaging control unit 246 adds the pixel values of the pixels that can receive the light in the same wavelength band constituting the imaging element 244B by the charge conversion unit 303 for each predetermined number of pixels, and performs the conversion. By outputting the pixel value, the synthesis process by the synthesis unit 513 may be omitted.

(Embodiment 3)
Next, Embodiment 3 will be described. The third embodiment is different in configuration from the color filter 2442 according to the first embodiment described above, and is different in the type of illumination light emitted from the light source device 3. Hereinafter, the configuration of the color filter according to Embodiment 3 will be described. In addition, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and detailed description is abbreviate | omitted suitably.

[Color filter configuration]
FIG. 16 is a diagram schematically illustrating a configuration of a color filter according to the third embodiment.

16 is configured using a magenta filter (hereinafter simply referred to as “Mg filter”) that transmits light in the red wavelength band and light in the blue wavelength band, and a Cy filter. The color filter 2442B is formed by a filter unit U3 using two Mg filters and two Cy filters, and the filter unit U3 is formed by two vertical and horizontal units (2xx). The color filter 2442B is formed by arranging Mg filters and Cy filters in a checkered pattern. As the color filter 2442B, an Mg filter or a Cy filter is stacked on the light receiving surface of each pixel of the pixel portion 2441.

In the endoscope system 1 configured as described above, the light source device 3 emits G light and R light at the same time under the control of the processing device 5 to emit light in the red wavelength band and light in the green wavelength. Including light (hereinafter simply referred to as “Ye light”) or B light is supplied to the endoscope 2, and the imaging element 244 sequentially images the subject irradiated with the Ye light or B light. In the third embodiment, B light corresponds to light in the first wavelength band, and Ye light corresponds to light in the second wavelength band.

The Mg filter transmits R light, and the Cy filter transmits G light. For this reason, as shown in FIG. 17, the image sensor 244 can generate an image P _{Ye in} which the R pixel and the G pixel are in a checkered pattern. Further, the Mg filter and the Cy filter transmit B light. For this reason, as shown in FIG. 18, the image sensor 244 can generate an image P _B. The image processing unit 51 performs the same process as that in the first embodiment. Thereby, when performing the narrow band observation method, since the pixel value of the B pixel is large, it is possible to improve the resolution of the NBI image.

According to the third embodiment described above, the first image data generated by the imaging device 244 when the image processing unit 51 is irradiated with the B light by the light source device 3 and the Ye light is irradiated by the light source device 3. And the second image data generated by the image sensor 244 at the time. Thereby, the endoscope system 1 can generate a white image and an NBI image in two frames (two fields) by two irradiations of B light and Ye light.

(Embodiment 4)
Next, a fourth embodiment will be described. The fourth embodiment is different from the light source device 3 according to the first embodiment described above. In the following, the configuration of the endoscope system according to the fourth embodiment will be described. In addition, the same code | symbol is attached | subjected to the structure same as the endoscope system 1 which concerns on Embodiment 1 mentioned above, and the abbreviate | omitted description is abbreviate | omitted suitably.

[Configuration of endoscope system]
FIG. 19 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the fourth embodiment. An endoscope system 1C shown in FIG. 19 includes a light source device 3C instead of the light source device 3 according to the first embodiment described above.

The light source device 3C includes a light source unit 31C instead of the light source unit 31 of the light source device 3 according to Embodiment 1 described above. The light source unit 31C includes a fourth light source 315 and a fifth light source 316 in addition to the configuration of the light source unit 31 according to Embodiment 1 described above.

The fourth light source 315 is configured using a purple LED lamp. The fourth light source 315 emits light in the violet wavelength band (400 nm to 435 nm) (hereinafter simply referred to as “V light”) based on the current supplied from the light source driver 32.

The fifth light source 316 is configured using an orange LED lamp. The fifth light source 316 emits light in the orange (Amber) wavelength band (585 nm to 620 nm) (hereinafter simply referred to as “A light”) based on the current supplied from the light source driver 32.

[Spectral characteristics of color filters and illumination light]
Next, spectral characteristics of the color filter 2442 and each light emitted from the light source device 3C will be described. FIG. 20 is a diagram schematically illustrating spectral characteristics of the color filter 2442 and each light emitted from the light source device 3C. In FIG. 20, the horizontal axis indicates the wavelength, the right vertical axis indicates the sensitivity of each pixel, and the left vertical axis indicates the intensity of each light. In FIG. 20, a curve L _Cy indicates the spectral sensitivity characteristic of the Cy pixel, and a curve L _Ye indicates the spectral sensitivity characteristic of the Ye pixel. The curve L _V represents a wavelength characteristic of the V light, curve L _B represents a wavelength characteristic of the B light, the curve L _G represents a wavelength characteristic of the G light, the curve L _A represents a wavelength characteristic of the A light, curve L _R represents the wavelength characteristic of the R light.

As indicated by a curve L _Ye in FIG. 20, the Ye filter transmits R light, A light, and G light. The Cy filter transmits V light, B light, and G light. Therefore, as shown in FIG. 21, when the light source device 3C is emitted from the G light, the image pickup device 244 can generate an image P _G all the pixels have a G pixel. In addition, as illustrated in FIG. 22, when the light source device 3 </ b> C emits B light and R light, the imaging element 244 can generate an image P _{BR in} which the R pixel and the B pixel are checkered. Furthermore, as shown in FIG. 23, when the light source device 3C is emitted from the V light and A light, imaging device 244 may generate an image P _AV where A pixel and V pixel a checkered pattern. That is, according to the fourth embodiment, a plurality of images having different uses, for example, three or more types of images can be generated in three frames.

[Endoscope system processing]
Next, processing executed by the endoscope system 1C will be described. FIG. 24 is a timing chart illustrating an outline of processing executed by the endoscope system 1C. In the following description, the light source device 3C has a G light irradiation frame for irradiating G light, a B + R light irradiation frame for simultaneously irradiating B light and R light, and a V + A light for simultaneously irradiating V light and A light. Three patterns of irradiation frames are performed. In FIG. 24, the horizontal axis indicates time. Further, in FIG. 24, a case will be described in which the time obtained by adding the irradiation time and the signal readout time is one cycle, and the time of this one cycle is 1/90 seconds (90 fps). Note that the time of one cycle can be changed as appropriate, and may be 1/120 seconds or 1/240 seconds, for example, or can be changed as appropriate depending on the subject, the examination content, the site, and the like.

24, first, the light source device 3C supplies G light to the endoscope 2, so that the endoscope 2 irradiates the subject with G light. In this case, the image sensor 244 images the subject irradiated with the G light. Then, the imaging control unit 246 reads a signal from the imaging element 244 and outputs the signal to the image processing unit 51.

Subsequently, the light source device 3C supplies the B light + R light to the endoscope 2 so that the endoscope 2 irradiates the subject with the B light + R light. In this case, the image sensor 244 images the subject irradiated with the B light + R light. Then, the imaging control unit 246 reads a signal from the imaging element 244 and outputs the signal to the image processing unit 51. At this time, the image processing unit 51 performs the separation process and the interpolation process similar to those of the first embodiment described above on the image P _BR to generate the image P _B2 and the image _PR2 , and the image P _{G and the} image P _A white image is generated by performing a combining process for combining _B2 and the image _PR2 .

Thereafter, the light source device 3C supplies V light + A light to the endoscope 2 so that the endoscope 2 irradiates the subject with V light + A light. In this case, the image sensor 244 images the subject irradiated with the V light + A light. Then, the imaging control unit 246 reads a signal from the imaging element 244 and outputs the signal to the image processing unit 51.

Subsequently, the light source device 3 </ b> C supplies G light to the endoscope 2, so that the endoscope 2 irradiates the subject with G light. In this case, the image sensor 244 images the subject irradiated with the G light. Then, the imaging control unit 246 reads a signal from the imaging element 244 and outputs the signal to the image processing unit 51. At this time, the image processing unit 51 generates the image P _V and the image P _A by performing separation processing and interpolation processing on the image P _VA , and the generated image P _V and the image P _A. generating a MBI image (Multiband Imaging) by combining G and B lights + R light image _{P G} generated upon irradiation _of the image _{P B2} and the image _{P R2.} Note that the image P _A is used when extracting the abnormal area having a predetermined feature quantity by learning, not shown. Here, the abnormal areas are a cancer area, a bleeding area, and a lesion area.

As described above, the endoscope system 1C has a plurality of types including a white image and an MBI image by repeating a pattern of one period in three frames of irradiation patterns of G light, B light + R light, and V light + A light. Images can be generated.

According to the fourth embodiment described above, the illumination control unit 33 causes the light source device 3C to repeat the pattern of one cycle in the irradiation pattern of G light, B light + R light, and V light + A light. Multiple types of images can be generated including white and MBI images. For these multiple types of images, a learning device or AI that has learned the feature amount of an abnormal region (for example, cancer, bleeding, etc.) may extract the abnormal region. In this case, the feature part or feature region extracted by the learning device or AI may be superimposed on the white image and displayed on the display device 4.

Further, according to the fourth embodiment, color information necessary for diagnosis can be generated from a small number of frames while achieving both high image quality and downsizing, and color misregistration can be suppressed.

Further, in the fourth embodiment, the frame rate is increased by sequentially irradiating V light, B light, A light, and R light, or by simultaneously irradiating only V light and B light. can do.

In the fourth embodiment, the sensitivity may be increased by adding pixel values of pixels of the same color as in the third embodiment.

(Other embodiments)
Various forms can be formed by appropriately combining a plurality of constituent elements disclosed in the endoscope systems according to the first to fourth embodiments of the present disclosure described above. For example, some components may be deleted from all the components described in the first to fourth embodiments. Furthermore, the constituent elements described in the first to fourth embodiments may be appropriately combined.

In the first to fourth embodiments of the present disclosure, the control device and the light source device are separate, but may be integrally formed.

In Embodiments 1 to 4 of the present disclosure, each light is emitted by the LED lamp. However, the present invention is not limited to this. For example, each light may be emitted by using a laser light source. Of course, each light may be emitted using a white light source and a filter that transmits the wavelength band of each light.

In Embodiments 1 to 4 of the present disclosure, the endoscope system is used. However, for example, a capsule endoscope, a video microscope for imaging a subject, a mobile phone having an imaging function, and an imaging function are provided. Even a tablet terminal can be applied.

In the first to fourth embodiments of the present disclosure, the endoscope system includes a flexible endoscope. However, an endoscope system including a rigid endoscope, and an industrial endoscope are used. Even an endoscopic system provided can be applied.

Further, in Embodiments 1 to 4 of the present disclosure, the endoscope system includes an endoscope that is inserted into a subject. However, an endoscope such as a sinus endoscope and an electric knife or a test probe is used. Even a system can be applied.

In the first to fourth embodiments of the present disclosure, the “unit” described above can be read as “means”, “circuit”, and the like. For example, the control unit can be read as control means or a control circuit.

The program executed by the endoscope system according to the first to fourth embodiments of the present disclosure is a file data in an installable format or an executable format, and is a CD-ROM, a flexible disk (FD), a CD-R, Provided by being recorded on a computer-readable recording medium such as a DVD (Digital Versatile Disk), a USB medium, or a flash memory.

In addition, the program executed by the endoscope system according to Embodiments 1 to 4 of the present disclosure is configured to be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. May be. Furthermore, the program to be executed by the endoscope system according to the first to fourth embodiments of the present disclosure may be provided or distributed via a network such as the Internet.

In addition, in the endoscope systems according to Embodiments 1 to 4 of the present disclosure, signals are transmitted from the endoscope to the processing device using the transmission cable. May be. In this case, an image signal or the like may be transmitted from the endoscope to the processing device according to a predetermined wireless communication standard (for example, Wi-Fi (registered trademark) or Bluetooth (registered trademark)). Of course, wireless communication may be performed according to other wireless communication standards.

In the description of the flowchart in the present specification, the context of the processing between steps is clearly indicated using expressions such as “first”, “after”, “follow”, etc., in order to implement the present invention. The order of processing required is not uniquely determined by their representation. That is, the order of processing in the flowcharts described in this specification can be changed within a consistent range. Further, the program is not limited to such a simple branch process, and more determination items may be comprehensively determined and branched. In that case, an artificial intelligence technique that performs machine learning while prompting the user to perform manual operation and repeating learning may be used in combination. Further, it may be executed by learning an operation pattern performed by many specialists and performing deep learning in a form in which more complicated conditions are inserted.

As described above, some of the embodiments of the present disclosure have been described in detail with reference to the drawings. However, these are merely examples, and various embodiments based on the knowledge of those skilled in the art, including the aspects described in the disclosure section of the present invention. It is possible to carry out the present invention in other forms that have been modified and improved.

DESCRIPTION OF

SYMBOLS

1,1C Endoscope system 2

Endoscope

3,3C Light source apparatus 4 Display apparatus 5 Processing apparatus 21 Insertion part 22 Operation part 23 Universal code 24 Tip part 25 Bending part 26

Flexible tube part

31, 31C Light source part 32 Light source driver DESCRIPTION OF SYMBOLS 33 Illumination control part 51 Image processing part 52 Input part 53 Recording part 54 Processing control part 241 Light guide 242 Illumination lens 243 Optical system 244,244B Image sensor 245 Endoscope recording part 246 Imaging control part 301 Photoelectric conversion element 302 Transfer transistor 303 Charge conversion unit 304 Reset transistor 305 Pixel source follower transistor 306 Selection transistor 307 Vertical transfer line 311 Condensing lens 312 First light source 313 Second light source 314 Third light source 315 Fourth light source 316 Fifth light source 511 Separation unit 512 Interpolation unit 513 Component unit 531

Program recording unit

2441,

2441B Pixel unit

2442, 2442A, 244B Color filter

Claims

Each color filter is configured by using two types of filters that transmit light in two different wavelength bands among light in the red wavelength band, light in the green wavelength band, and light in the blue wavelength band. An image pickup element that is stacked on a light receiving surface of a pixel and that can generate image data by imaging a subject;
Light in a first wavelength band that includes light in at least one of the two wavelength bands and light in a second wavelength band that includes light in at least the other wavelength band of the two wavelength bands A light source unit capable of irradiating
An illumination control unit that causes the light source unit to alternately irradiate light of the first wavelength band and light of the second wavelength band; and
The first image data generated by the image sensor when the light source unit irradiates light of the first wavelength band, and the imaging when the light source unit irradiates light of the second wavelength band. An image processing unit for combining the second image data generated by the element;
An endoscope system comprising:
The two types of filters are:
A cyan filter that transmits light in the green wavelength band and light in the blue wavelength band, and a yellow filter that transmits light in the red wavelength band and light in the green wavelength band;
The light source unit is
A first light source capable of emitting light in the red wavelength band;
A second light source capable of emitting light in the green wavelength band;
A third light source capable of emitting light in the blue wavelength band;
Have
The illumination control unit
The second light source is irradiated with the light of the green wavelength band as the light of the first wavelength band, while the light of the red wavelength band is emitted to the first light source as the light of the second wavelength band. Simultaneously irradiating three light sources with light of the blue wavelength band,
The image processing unit
Red image data and blue image data are obtained by separating a pixel value of the light in the red wavelength band and a pixel value of the light in the blue wavelength band from the second image corresponding to the second image data. A separation unit for generating
A combining unit that generates white image data by combining the red image data generated by the separating unit, the blue image data, and the first image data;
The endoscope system according to claim 1, further comprising:
The endoscope system according to claim 2, wherein the combining unit generates special image data by combining the blue image data generated by the separation unit and the first image data. .
The two types of filters are:
A cyan filter that transmits light in the green wavelength band and light in the blue wavelength band, and a magenta filter that transmits light in the red wavelength band and light in the blue wavelength band;
The light source unit is
A first light source capable of emitting light in the red wavelength band;
A second light source capable of emitting light in the green wavelength band;
A third light source capable of emitting light in the blue wavelength band;
Have
The illumination control unit
The third light source is irradiated with the blue wavelength band light as the first wavelength band light, while the red wavelength band light and the first light wavelength are emitted to the first light source as the second wavelength band light. Simultaneously illuminate two light sources with light in the green wavelength band,
The image processing unit
Separating the pixel value of the light in the red wavelength band and the pixel value of the light in the green wavelength band from the second image corresponding to the second image data, the red image data and the green image data A separation unit for generating
A combining unit that generates white image data by combining the red image data generated by the separating unit, the green image data, and the first image data;
The endoscope system according to claim 1, further comprising:
The endoscope system according to claim 4, wherein the synthesizing unit generates special image data by synthesizing the green image data separated by the separation unit and the first image data. .
When the light source unit emits light of the first wavelength band, the pixel value of each pixel constituting the imaging element is added for each predetermined number of pixels and output, while the light source unit is the second light source. An imaging control unit that outputs a pixel value of a pixel capable of receiving light in the same wavelength band for each predetermined number of pixels among the pixels constituting the imaging element when irradiated with light in a predetermined wavelength band The endoscope system according to any one of claims 1 to 5, further comprising:
The light source unit is
A fourth light source capable of emitting light in a purple wavelength band;
A fifth light source capable of emitting light in an orange wavelength band;
Further comprising
The illumination control unit irradiates the light source unit with light of the second wavelength band, and then causes the fourth light source to emit light of the purple wavelength band and the fifth light source to light of the orange wavelength band. The endoscope system according to claim 2, wherein the endoscope system is simultaneously emitted.
Each color filter is configured by using two types of filters that transmit light in two different wavelength bands among light in the red wavelength band, light in the green wavelength band, and light in the blue wavelength band. An image sensor that is stacked on the light-receiving surface of the pixel and can generate image data by imaging a subject, and light in a first wavelength band including light in at least one of the two wavelength bands And a light source unit capable of irradiating light in a second wavelength band including light in at least the other wavelength band of the light in the two wavelength bands. And
An illumination control step of alternately irradiating the light source unit with the light of the first wavelength band and the light of the second wavelength band;
The first image data generated by the image sensor when the light source unit irradiates light of the first wavelength band, and the imaging when the light source unit irradiates light of the second wavelength band. An image processing step of combining the second image data generated by the element;
An image processing method comprising:
Each color filter is configured by using two types of filters that transmit light in two different wavelength bands among light in the red wavelength band, light in the green wavelength band, and light in the blue wavelength band. An image sensor that is stacked on the light-receiving surface of the pixel and can generate image data by imaging a subject, and light in a first wavelength band including light in at least one of the two wavelength bands And a light source unit capable of irradiating light in a second wavelength band including light in at least the other wavelength band of the light in the two wavelength bands.
An illumination control step of alternately irradiating the light source unit with the light of the first wavelength band and the light of the second wavelength band;
The first image data generated by the image sensor when the light source unit irradiates light of the first wavelength band, and the imaging when the light source unit irradiates light of the second wavelength band. An image processing step of combining the second image data generated by the element;
A program characterized by having executed.