JP2011250926A - Electronic endoscope system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To permit observation images by different rays of illumination light to be simultaneously observed even by using a CMOS type image sensor.SOLUTION: The electronic endoscope 10 has a CMOS type image sensor (CMOS sensor) 23. The vertical scanning circuit 51 of the CMOS sensor 23 is structured so as to enable it to read an imaging signal every other horizontal line and to discharge the signal electric charge of all pixels 56 of the CMOS sensor 23 to a drain (collective resetting of all pixels) through a resetting transistor M3. When a simultaneous photographing mode to obtain each observation image is chosen by irradiating ordinary light (white light) and special light of a narrow wavelength zone, the system turns off the illumination light while it reads the imaging signal every other horizontal line by the vertical scanning circuit 51. The system adjusts the starting timing of electric charge accumulation of the CMOS sensor 23 by carrying out collective resetting of all pixels by the vertical scanning circuit 51 and changes the illumination light by the accumulation period of the CMOS sensor 23.

Description

  The present invention relates to an electronic endoscope system that captures an image of a subject with a CMOS type image sensor while switching the wavelength of illumination light applied to an observation site.

  Conventionally, inspection using an electronic endoscope has been widely used in the medical field. An electronic endoscope has a solid-state image sensor at the tip of an insertion portion that is inserted into the body of a subject (patient). The electronic endoscope is connected to the processor device and the light source device via a cord and a connector.

  The processor device performs various processes on the imaging signal output from the solid-state imaging device, and generates an observation image used for diagnosis. The observation image is displayed on a monitor connected to the processor device. The light source device has a white light source such as a xenon lamp and supplies illumination light for in-subject illumination to the electronic endoscope.

  Examples of the solid-state imaging device mounted on the electronic endoscope include a CMOS type image sensor and a CCD type image sensor. A CMOS image sensor has low power consumption and has an advantage that peripheral circuits can be formed on the same substrate. On the other hand, as shown in FIG. 10, a rolling shutter system that sequentially reads out signal charges for each horizontal line is adopted, and the charge accumulation period of each line is shifted. Therefore, when a moving subject is imaged, the image may be distorted. is there.

  On the other hand, CCD type image sensors have high power consumption and inherent problems due to structures such as smearing and blooming. However, high sensitivity and high quality images can be easily obtained, and signal charges of all pixels can be obtained. Since the same accumulation period can be secured in one image, it is suitable for capturing moving subjects. For this reason, many CCD-type image sensors are employed in electronic endoscopes that capture images of moving subjects.

  However, since a CMOS type image sensor has low power consumption and is excellent in mass productivity, it has been proposed to use a CMOS type image sensor for an electronic endoscope instead of a CCD type image sensor. The CMOS type image sensor has a drawback that it is difficult to obtain a high quality image with relatively large noise due to the individual difference of the amplifier provided for each pixel. However, in recent years, the improvement of the CMOS type image sensor has been improved. As a result, a CMOS image sensor that is equivalent to a CCD image sensor or superior in image quality to a CCD image sensor can be easily obtained. It is being considered.

  By the way, in the field of medical diagnosis using an electronic endoscope, in order to facilitate the discovery of lesions, a narrow wavelength band is used instead of white light (hereinafter referred to as normal light) having broad spectral characteristics in the visible light range. (Hereinafter referred to as “special light”) and the reflected light is imaged (hereinafter, the image thus obtained is referred to as a special image to be distinguished from a normal image by normal light). The method of observation is in the spotlight. According to this technique, it is possible to easily obtain an image in which blood vessels in the submucosal layer are emphasized and an image in which organ structures such as stomach wall and intestinal surface tissue are emphasized (see Patent Document 1).

JP 2007-322348 A

  In the method of obtaining the normal image and special image by irradiating the observation site with normal light and special light, we want to ensure the sameness (identity) of the normal image and the special image, and make a diagnosis while comparing the images of each other There is a request.

  When a CCD image sensor is used, the signal charge accumulation period of all the pixels is the same, and the signal charge accumulated in the previous period can be transferred during the current charge accumulation period. A normal image and a special image can be alternately obtained at intervals of one frame by simply switching the illumination light in accordance with the frame rate break.

  On the other hand, as shown in FIG. 10, the CMOS type image sensor sequentially reads out signal charges for each horizontal line by a rolling shutter system. For this reason, there is a difference of about one frame in the charge accumulation period between the pixels in the first line read out first and the pixels in the nth line read out last. In addition, the line from which the signal charge is read is reset, and the process proceeds to the next exposure.

  Therefore, when a rolling shutter CMOS image sensor is used, when the illumination light is switched, an image in which images of normal light and special light are mixed is generated when the illumination light is switched. For this reason, one frame must be wasted for switching the illumination light, and the normal image and special image such as the normal image acquisition frame, the illumination light switch frame, and the special image acquisition frame are used. There is a space for one frame between images.

  In this way, when switching illumination light using a CMOS image sensor, simply switching illumination light in accordance with the frame rate as in the case of a CCD image sensor, it is possible to switch between normal images and special images. Image simultaneity cannot be ensured.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to enable observation images with different illumination lights to be simultaneously observed while using a CMOS type image sensor.

  In order to achieve the above object, an electronic endoscope system of the present invention is provided at an insertion portion of an electronic endoscope that is inserted into a subject, and a CMOS type image sensor that images the inside of the subject, Illumination means for repeatedly turning on and off the illumination light while switching the wavelength band of illumination light to be irradiated in the specimen, a vertical scanning circuit for selectively reading out an imaging signal from the pixels in the imaging region of the image sensor, and illumination Control means for controlling the operation of the vertical scanning circuit so as to read out the imaging signal while the light is turned off.

  The vertical scanning circuit is configured to selectively read an imaging signal from a selected half or 1/3 of the imaging region of the image sensor. For example, the vertical scanning circuit is constituted by a shift register, and flip-flops of the shift register are connected every other horizontal line or every two horizontal lines, and an image pickup signal is read out every other horizontal line or every two horizontal lines.

  Alternatively, use a flip-flop that has a preset input terminal for forcibly holding data “1” or “0” and a clear input terminal, and perform a preset input to the flip-flop of the read start row selected on the horizontal line. If the clear input is performed to all the flip-flops when the reading of the read end row is completed, only a predetermined number of consecutive rows can be partially read (partial read). Note that, not limited to the flip-flop, a clocked inverter, a combination of a clock train and a decoder, or the like may be used.

  The vertical scanning circuit is configured to read an imaging signal from all the pixels in the imaging region of the image sensor. It is preferable that operation input means for switching whether to selectively read an imaging signal from pixels in the imaging area of the image sensor or to read an imaging signal from all pixels in the imaging area of the image sensor is provided.

  The vertical scanning circuit is configured so that signal charges accumulated in all pixels of the image sensor can be discharged to the drain all at once. Specifically, a flip-flop having a preset input terminal for forcibly holding data “1” or “0” and a clear input terminal is used, and presetting is performed for all flip-flops when charge accumulation to a pixel is started. If input is performed, the charge accumulation start timing of each pixel can be made uniform. The end timing of charge accumulation is made uniform by turning off the illumination light.

  The illumination means emits white light and light having a narrow wavelength band. Alternatively, the illuminating unit emits light in a wavelength band of each color of RGB and light in a narrow wavelength band. In the former case, an image sensor provided with a color filter is used, and one observation image is generated from one type of imaging signal obtained by irradiating white light. In the latter case, a monochrome image sensor is used, and one observation image is generated from three types of imaging signals obtained by irradiating light in the wavelength bands of RGB colors.

  According to the present invention, the imaging signal is selectively read from the pixels in the imaging region of the CMOS image sensor while the illumination light that is turned on while the wavelength band is switched is turned off. While being used, observation images with different illumination lights can be observed simultaneously.

It is an external view which shows the structure of an electronic endoscope system. It is a block diagram which shows the structure of an electronic endoscope system. It is a figure which shows the structure of a CMOS type image sensor. It is a figure which shows the structure of a vertical scanning circuit. It is a figure which shows the structure of a wavelength selection filter. 6 is a timing chart showing the operation of a CMOS image sensor and switching of illumination light when a frame rate priority mode is selected. 6 is a timing chart illustrating the operation of a CMOS image sensor and switching of illumination light when an image quality priority mode is selected. It is a figure which shows another structure of a vertical scanning circuit. It is a figure which shows the structure of the rotation filter at the time of applying a frame sequential imaging system. 6 is a timing chart illustrating an operation of a rolling shutter type CMOS image sensor.

  In FIG. 1, the electronic endoscope system 2 includes an electronic endoscope 10, a processor device 11, and a light source device 12. As is well known, the electronic endoscope 10 includes a flexible insertion portion 13 that is inserted into the body of a subject (patient), an operation portion 14 that is connected to the proximal end portion of the insertion portion 13, and a processor device. 11 and a connector 15 connected to the light source device 12, an operation unit 14, and a universal cord 16 that connects between the connectors 15.

  An observation window 20, an illumination window 21 (both see FIG. 2) and the like are provided at the distal end of the insertion portion 13. In the back of the observation window 20, a CMOS image sensor (hereinafter abbreviated as a CMOS sensor) 23 for in-subject imaging is disposed via an objective optical system 22 composed of a lens group and a prism (sometimes referred to as a CMOS sensor). Also see FIG. The illumination window 21 irradiates the site to be observed with illumination light from the light source device 12 guided by the light guide 24 disposed in the universal cord 16 or the insertion portion 13 and the illumination lens 25 (both see FIG. 2). To do.

  The operation unit 14 includes an angle knob for bending the distal end of the insertion unit 13 in the vertical and horizontal directions, an air supply / water supply button for ejecting air and water from the distal end of the insertion unit 13, and an observation image. Operation members such as a release button for recording an image and a zoom button for instructing enlargement / reduction of the observation image displayed on the monitor 17 are provided.

  Further, a forceps port through which a treatment tool such as an electric knife is inserted is provided on the distal end side of the operation unit 14. The forceps opening communicates with a forceps outlet provided at the distal end of the insertion portion 13 through a forceps channel in the insertion portion 13.

  The processor device 11 is electrically connected to the light source device 12 and comprehensively controls the operation of the electronic endoscope system 2. The processor device 11 supplies power to the electronic endoscope 10 via the universal cord 16 and a transmission cable inserted into the insertion portion 13, and controls the driving of the CMOS sensor 23. In addition, the processor device 11 receives the imaging signal output from the CMOS sensor 23 via the transmission cable, and performs various processes on the received imaging signal to generate image data. The image data generated by the processor device 11 is displayed as an observation image on a monitor 17 connected to the processor device 11 by a cable.

  In FIG. 2, the electronic endoscope 10 is provided with the observation window 20, the illumination window 21, the objective optical system 22, the CMOS sensor 23, and the illumination lens 25 at the distal end of the insertion portion 13. Further, a timing generator (TG) 26 and a CPU 27 are provided in the operation unit 14. The TG 26 may be provided in the CMOS sensor 23.

  The CMOS sensor 23 is arranged so that an image of an observation site in the subject via the observation window 20 and the objective optical system 22 enters the imaging region 50 (see FIG. 3). In the imaging region 50 of the CMOS sensor 23, a color filter composed of a plurality of color segments, for example, a primary color (RGB) or a complementary color (CMY or CMYG) color filter in a Bayer array is formed.

  The imaging signal output from the CMOS sensor 23 is input to the processor device 11 via the universal code 16 and the connector 15, and is temporarily stored in a working memory (not shown) of a digital signal processing circuit (hereinafter abbreviated as DSP) 33. Stored.

  The TG 26 gives a clock signal to the CMOS sensor 23. The CMOS sensor 23 performs an imaging operation according to the clock signal from the TG 26 and outputs an imaging signal. After the electronic endoscope 10 and the processor device 11 are connected, the CPU 27 drives the TG 26 based on an operation start instruction from the CPU 30 of the processor device 11.

  In FIG. 3, the CMOS sensor 23 includes an imaging region 50, a vertical scanning circuit 51, a correlated double sampling (CDS) circuit 52, a column selection transistor 53, an output circuit 54, and a horizontal scanning circuit 55.

  In the imaging region 50, the pixels 56 are arranged in a matrix. The pixel 56 includes a photodiode D1, an amplification transistor M1, a pixel selection transistor M2, and a reset transistor M3. The photodiode D1 generates and accumulates signal charges corresponding to the amount of incident light through photoelectric conversion. The signal charge accumulated in the photodiode D1 is amplified as an imaging signal by the amplifying transistor M1, and is output to the outside of the pixel 56 at a predetermined timing by the pixel selecting transistor M2. The signal charge accumulated in the photodiode D1 is discharged to the drain through the reset transistor M3 at a predetermined timing. The pixel selecting transistor M2 and the resetting transistor M3 are N-channel transistors, which are turned on when a high level “1” is applied to the gate and turned off when a low level “0” is applied.

  In the imaging region 50, a row selection line L1 and a row reset line L2 are wired from the vertical scanning circuit 51 in the horizontal direction (X direction), and a column signal line L3 from the CDS circuit 52 in the vertical direction (Y direction). Wired. The row selection line L1 is connected to the gate of the pixel selection transistor M2, and the row reset line L2 is connected to the gate of the reset transistor M3. The column signal line L3 is connected to the source of the pixel selection transistor M2, and is connected to the column selection transistor 53 of the corresponding column via the CDS circuit 52.

  The CDS circuit 52 holds the imaging signal of the pixel 56 connected to the row selection line L1 selected by the vertical scanning circuit 51 based on the clock signal input from the TG 26, and performs noise removal. The horizontal scanning circuit 55 generates a horizontal scanning signal based on the clock signal input from the TG 26 and performs on / off control of the column selection transistor 53.

  The column selection transistor 53 is provided between the output bus line 57 connected to the output circuit 54 and the CDS circuit 52, and selects a pixel for transferring an imaging signal to the output bus line 57 in accordance with a horizontal scanning signal. To do. The output circuit 54 amplifies the imaging signal sequentially transferred from the CDS circuit 52 to the output bus line 57, performs A / D conversion, and outputs it. The gain of the imaging signal by the output circuit 54 is adjusted by inputting a gain adjustment signal from the CPU 27 to the output circuit 54.

  As shown in FIG. 4, the vertical scanning circuit 51 includes a vertical scanning shift register 60 and a reset shift register 61. The vertical scanning shift register 60 generates a vertical scanning signal based on the clock signal input from the TG 26, selects the row selection line L1 row by row, and outputs the imaging signal to the column signal line L3. Change the line (hereinafter referred to as the horizontal line). The reset shift register 61 selects the horizontal row reset line L2 row by row and changes the horizontal line for discharging the signal charge to the drain via the reset transistor M3.

  Each shift register 60, 61 is a serial input-parallel output type having a plurality of D-type flip-flops 62, 63 for horizontal lines. As is well known, the shift register sequentially transfers the data ("1" or "0") held in each flip-flop 62, 63 to the adjacent flip-flops 62, 63 according to the change of the clock signal (clk).

  When the vertical scanning is performed, the enable signal “1” synchronized with the clock signal is applied to the input terminal D (start) of the flip-flop 62 in the first horizontal line of the vertical scanning shift register 60. Similarly, an enable signal “1” synchronized with the clock signal is applied to the input terminal D (start) of the flip-flop 63 on the first horizontal line of the reset shift register 61 when performing the reset operation.

  The output terminal Q of the flip-flop 62 of the vertical scanning shift register 60 is connected to the row selection lines L1_1, L1_2,..., L1_n−1, L1_n of each horizontal line. The output terminal Q of the flip-flop 63 of the reset shift register 61 is connected to row reset lines L2_1, L2_2,..., L2_n−1, L2_n. As the enable signal “1” sequentially moves between the flip-flops 62 in accordance with the clock signal, “1” is sequentially applied to the row selection line L1, and the pixel selection transistor M2 is sequentially turned on. As a result, the imaging signals of the horizontal lines are sequentially output. Further, the enable signal “1” sequentially shifts between the flip-flops 63 in accordance with the clock signal, so that “1” is sequentially applied to the row reset line L2, and the reset transistor M3 is sequentially turned on. Thereby, the signal charge of each horizontal line is discharged to the drain.

  A changeover switch 64a is connected between the output terminal Q of the flip-flop 62 of the odd-numbered horizontal line of the vertical scanning shift register 60 and the input terminal D of the flip-flop 62 of the even-numbered horizontal line. . A changeover switch 64b is connected between the output terminal Q of the flip-flop 62 of the even-numbered horizontal line and the input terminal D of the flip-flop 62 of the odd-numbered horizontal line. The changeover switches 64a and 64b are interlocked.

  A branch line 65 is wired between the flip-flops 62 of the odd-numbered horizontal lines, and the change-over switches 64a and 64b are input / output of the adjacent flip-flops 62 in accordance with a switch signal (switch) from the CPU 27. Switching between a state in which the terminals D and Q are connected and a state in which the flip-flops 62 on the even-numbered horizontal lines are skipped to connect the flip-flops 62 on the odd-numbered horizontal lines by being tilted to the branch line 65 side. .

  In the state where the flip-flops 62 of the odd-numbered horizontal lines are connected by the change-over switches 64a and 64b, the enable signal is not supplied to the flip-flops 62 of the even-numbered horizontal lines. 62 holds “0”. Accordingly, the data applied to the even-numbered row selection line L1 remains “0”, and the imaging signal of the even-numbered horizontal line is not output. That is, thinning readout is performed in which imaging signals are output every other row.

  The flip-flop 63 of the reset shift register 61 is provided with input terminals P and C of signals (preset and clear) for forcibly holding “1” or “0” data without using a clock signal. ing. When both “0” are input to the input terminals P and C, the data given to the input terminal D is taken in according to the clock signal. In this case, a data shift operation is performed between the flip-flops 63 in accordance with the clock signal, and the signal charges on each horizontal line are sequentially discharged to the drain. On the other hand, when “1” is input to the input terminal P and “0” is input to the input terminal C, “1” is input to all the flip-flops 63 of the reset shift register 61 regardless of the data applied to the input terminal D. "Is retained. In this case, data “1” is applied to all the row reset lines L2, and all pixel collective reset is performed to simultaneously discharge the signal charges of all horizontal lines to the drains. After all the pixels are reset, “0” is input to the input terminal P and “1” is input to the input terminal C. In this case, data “0” is held in all the flip-flops 63 of the reset shift register 61 regardless of the data given to the input terminal D. That is, all flip-flops 63 are cleared.

  Returning to FIG. 2, the CPU 30 controls the overall operation of the processor device 11. The CPU 30 is connected to each unit via a data bus, an address bus, and a control line (not shown). The ROM 31 stores various programs (OS, application programs, etc.) and data (graphic data, etc.) for controlling the operation of the processor device 11. The CPU 30 reads out necessary programs and data from the ROM 31 and develops them in the RAM 32 which is a working memory, and sequentially processes the read programs. Further, the CPU 30 obtains information that changes for each examination, such as examination date and time, character information such as subject and operator information, from a network such as an operation unit 36 or a LAN (Local Area Network) described later, and stores the information in the RAM 32. .

  The DSP 33 performs various signal processing such as color separation, color interpolation, gain correction, white balance adjustment, and gamma correction on the imaging signal from the CMOS sensor 23 to generate image data. Image data generated by the DSP 33 is input to a working memory (not shown) of a digital image processing circuit (hereinafter abbreviated as DIP) 34.

  The DIP 34 performs various types of image processing such as electronic scaling, color enhancement, and edge enhancement on the image data processed by the DSP 33. Image data that has been subjected to various image processing by the DIP 34 is input to the display control circuit 35.

  The display control circuit 35 has a VRAM that stores processed image data from the DIP 34. The display control circuit 35 receives graphic data in the ROM 31 and RAM 32 from the CPU 30. The graphic data includes a display mask that hides the invalid pixel region of the observation image and displays only the effective pixel region, character information such as examination date and time, or information on the subject and the operator, and a graphical user interface (GUI). is there. The display control circuit 35 performs various display control processes such as a display mask, character information, GUI superimposition processing, and drawing processing on the display screen of the monitor 17 on the image data from the DIP 34.

  The display control circuit 35 reads the image data from the VRAM and converts the read image data into a video signal (component signal, composite signal, etc.) corresponding to the display format of the monitor 17. Thereby, an observation image is displayed on the monitor 17.

  The operation unit 36 is a known input device such as an operation panel provided on the housing of the processor device 11, buttons on the operation unit 14 of the electronic endoscope 10, or a mouse or a keyboard. The CPU 30 operates each unit in response to an operation signal from the operation unit 36.

  In addition to the above, the processor unit 11 stores the compressed image data in a CF processing circuit that performs image compression on the image data in a predetermined compression format (for example, JPEG format) and the release button. A media I / F for recording on a removable medium such as a card, a magneto-optical disk (MO), a CD-R, a network I / F for controlling transmission of various data with a network such as a LAN, and the like are provided. These are connected to the CPU 30 via a data bus or the like.

  The light source device 12 includes a light source 40, a wavelength selection filter 41, and a CPU 42. The light source 40 is a xenon lamp or a white LED (light emitting diode) that generates light having a broad wavelength from red to blue (for example, light having a wavelength band of 400 nm to 800 nm, hereinafter referred to as normal light). It is driven by a driver 43. The illumination light emitted from the light source 40 is condensed by the condenser lens 44 and guided to the incident end of the light guide 24.

  The wavelength selection filter 41 is a filter that restricts light emitted from the light source 40 to light in a specific narrow wavelength band (hereinafter referred to as special light). As shown in FIG. 5, the wavelength selection filter 41 has a shape in which a half of the disk is cut out, and is rotated by a motor so as to cross between the light source 40 and the condenser lens 44. Further, the wavelength selection filter 41 is provided with a sensor for detecting the rotational position. While the wavelength selection filter 41 crosses between the light source 40 and the condenser lens 44, the special light is irradiated, and while the notch portion of the wavelength selection filter 41 crosses between the light source 40 and the condenser lens 44. Normal light is irradiated. Examples of the special light include light having a wavelength in the vicinity of 450, 500, 550, 600, and 780 nm.

  Imaging with special light in the vicinity of 450 nm is suitable for observing the fine structure of the surface to be observed, such as blood vessels and pit patterns on the surface layer. With illumination light in the vicinity of 500 nm, it is possible to observe a macro uneven structure such as a depression or a bulge in the observation site. Illumination light in the vicinity of 550 nm has a high absorption rate by hemoglobin, and is suitable for observation of fine blood vessels and redness, and illumination light in the vicinity of 600 nm is suitable for observation of thickening. The deep blood vessels can be observed clearly by injecting a fluorescent substance such as indocyanine green (ICG) intravenously and using illumination light in the vicinity of 780 nm.

  Here, the wavelength selection filter 41 is used. However, instead of or in addition to the wavelength selection filter 41, a plurality of LEDs, laser diodes, etc. that emit light having different wavelength bands are provided as the light source 40, and these are turned on and off. By controlling the above, normal light and special light may be switched. Alternatively, normal light may be generated using a blue laser light source and a phosphor that emits green to yellow excitation light when irradiated with blue laser light, and special light may be generated using a wavelength selection filter.

  The CPU 42 communicates with the CPU 30 of the processor device 11 and controls the operation of the light source driver 43 and the wavelength selection filter 41.

  For example, the light guide 24 is formed by bundling a plurality of quartz optical fibers with a winding tape or the like. The illumination light guided to the emission end of the light guide 24 is diffused by the illumination lens 25 and irradiated to the observation site in the subject through the illumination window 21.

  The electronic endoscope system 2 includes a normal shooting mode in which observation is performed using only normal light, a special shooting mode in which observation is performed using only special light, and simultaneous irradiation with a combination of normal light and special light. Shooting modes are available. In the special photographing mode, it is possible to select light having a wavelength in the vicinity of the aforementioned 450, 500, 550, 600, and 780 nm. Further, the simultaneous shooting mode is provided with a frame rate priority mode and an image quality priority mode. Switching between the modes is performed by operating the operation unit 36.

  The frame rate priority mode reduces the image quality by reducing the number of pixels of the normal image and special image, but obtains the normal image or special image by irradiating normal light or special light. In this mode, observation images are acquired at the same frame rate. The image quality priority mode is a mode for acquiring a high-quality observation image that is usually equal to the special imaging mode while reducing the frame rate as compared with the special imaging mode.

  When the normal shooting mode is selected, the CPU 30 controls the driving of the light source driver 43 via the CPU 42 and turns on the light source 40. Further, the wavelength selection filter 41 is operated based on the output of the rotational position detection sensor, and the notch portion of the wavelength selection filter 41 is positioned between the light source 40 and the condenser lens 44. The illumination light applied to the site to be observed is only normal light. On the other hand, when the special photographing mode is selected, the light source 40 is turned on and the wavelength selection filter 41 is positioned between the light source 40 and the condenser lens 44. Illumination light applied to the site to be observed is only special light.

  In the normal shooting mode and the special shooting mode, the CMOS sensor 23 operates in a rolling shutter system shown in FIG. The change-over switches 64a and 64b are in the state shown in FIG. 4 that connects the input / output terminals D and Q of the adjacent flip-flops 62, and the input terminals P and C of the flip-flop 63 of the reset shift register 61 are both “ “0” is input. In this case, each shift register 60 and 61 sequentially scans the row selection line L1 and the row reset line L2 one row at a time. Since the illumination light cannot be switched, even if the CMOS sensor 23 is operated by the rolling shutter method, an image in which images of normal light and special light are mixed is not generated.

  The pixels 56 in each horizontal line are renewed in response to the reset signal being input to the row reset line L2 to reset the accumulated charge (the input to the reset transistor M3 is changed from “1” to “0”). Charge accumulation starts. Then, the imaging signal is read in response to the input of the vertical scanning signal to the row selection line L1. By repeating this operation in order from the first row to the last horizontal line, an image for one frame is obtained. Hereinafter, an image obtained by photographing with normal illumination light is referred to as a normal image, and an image obtained by photographing with special illumination light is referred to as a special image.

  When the frame rate priority mode is selected in the simultaneous photographing mode, the CMOS sensor 23 operates as shown in FIG. The change-over switches 64a and 64b are connected to the flip-flops 62 of the odd-numbered horizontal lines by the switch signal. As a result, the even-numbered horizontal lines are skipped and the imaging signals of the odd-numbered horizontal lines are sequentially read (thinning-out reading).

  On the other hand, when the image quality priority mode is selected in the simultaneous shooting mode, the CMOS sensor 23 operates as shown in FIG. The change-over switches 64a and 64b are normally in the state shown in FIG. 4 that connects the input / output terminals D and Q of the adjacent flip-flops 62 as in the case of the special photographing mode, and the image pickup signals of the horizontal lines are sequentially read out. (All pixel readout).

  In both the frame rate priority mode and the image quality priority mode, “1” is input to the input terminal P of the flip-flop 63 of the reset shift register 61 after the image signals of all horizontal lines are read (the input terminal C is “0”). ). As a result, signal charges on all horizontal lines are simultaneously discharged to the drain (all pixel batch reset), and charge accumulation on all horizontal lines is started. After all the pixels are collectively reset, “0” is input to the input terminal P of the flip-flop 63 and “1” is input to the input terminal C, and the preset to the flip-flop 63 is cleared.

  The light source 40 is turned off while the imaging signals of all horizontal lines are being read (from the start of reading the first row to the end of reading the last row). Further, the wavelength selection filter 41 is rotated so that the normal light and the special light are alternately irradiated in units of the accumulation period of the CMOS sensor 23. The illumination light applied to the site to be observed is sequentially switched between normal light and special light in units of the accumulation period of the CMOS sensor 23 with the extinguishing period in between. Instead of turning off the light source 40, the wavelength selection filter 41 is provided with a light shielding region for normal light and special light, and the wavelength selection filter 41 is set so that the readout period of the imaging signals for all horizontal lines matches the light shielding region. It may be rotated.

  In the frame rate priority mode, only the imaging signal of the horizontal line of the odd-numbered row is read, so the time required for reading the imaging signal is half that in the image quality priority mode in which all the horizontal lines are read in order. Therefore, the frame rate priority mode can generate an image at a frame rate twice that of the image quality priority mode. However, compared with the frame rate priority mode for thinning readout, the image quality priority mode for all pixel readout provides a high-quality image with high vertical resolution.

  When the frame rate priority mode is selected, the DSP 33 performs pixel interpolation processing for interpolating the image signals of the even-numbered horizontal lines skipped using the image signals of the odd-numbered horizontal lines, Is generated. In the normal shooting mode or the special shooting mode, the display control circuit 35 displays only a moving image or a still image of the normal image or the special image on the monitor 17. In the simultaneous shooting mode, one moving image or still image of a normal image or special image is displayed according to an operation input to the operation unit 36, or a moving image or still image of each image is simultaneously displayed on the monitor 17 (for example, normally, Special images are displayed in parallel, superimposed, and nested (Picture in Picture, PinP)).

  In addition, the display form of each image illustrated here is an example, and various deformation | transformation are possible. For example, a plurality of monitors may be prepared, and a multi-monitor format may be employed, such that the first unit is for displaying a normal image and the second unit is for displaying a special image.

  Next, the operation of the electronic endoscope system 2 configured as described above will be described. When observing the inside of the subject with the electronic endoscope 10, the operator connects the electronic endoscope 10 and the devices 11 and 12, and turns on the power of the devices 11 and 12. Then, the operation unit 36 is operated to input information about the subject and instruct to start the examination.

  After instructing the start of the examination, the surgeon inserts the insertion portion 13 into the subject, and monitors the observation image in the subject by the CMOS sensor 23 while illuminating the subject with illumination light from the light source device 12. Observe at 17.

  The imaging signal output from the CMOS sensor 23 is input to the DSP 33 of the processor device 11. In the DSP 33, various signal processing is performed on the input image pickup signal to generate image data. The image data generated by the DSP 33 is output to the DIP 34.

  In the DIP 34, various image processing is performed on the image data from the DSP 33 under the control of the CPU 30. The image data processed by the DIP 34 is input to the VRAM of the display control circuit 35. In the display control circuit 35, various display control processes are executed in accordance with the graphic data from the CPU 30. As a result, the image data is displayed on the monitor 17 as an observation image.

  When an examination is performed using the electronic endoscope system 2, the object to be observed, such as whether the site to be observed is a mucosa or a submucosal blood vessel, the type of lesion to be observed, etc. Accordingly, the wavelength band of the illumination light that irradiates the site to be observed is selected. The wavelength band of the illumination light selected here is appropriately changed to one that can be easily diagnosed while viewing the observation image during observation.

  Normally, when the special photographing mode is selected by the operation unit 36, the light source 40 is turned on under the instruction of the CPU 30, and the notch portion of the wavelength selection filter 41 is positioned between the light source 40 and the condenser lens 44. Alternatively, the wavelength selection filter 41 is positioned between the light source 40 and the condensing lens 44, and the observation site is irradiated with only normal light or special light. Further, the CMOS sensor 23 is operated by a rolling shutter system. The monitor 17 displays only a normal image or a moving image or a still image of a special image.

  On the other hand, when the simultaneous photographing mode is selected, the wavelength selection filter 41 is rotated so that the normal light and the special light are alternately irradiated in units of the accumulation period of the CMOS sensor 23. Further, the light source 40 is turned off while the imaging signal is being read out by the CMOS sensor 23.

  When the frame rate priority mode is selected, thinning-out readout for reading out only the imaging signals of the horizontal lines of the odd-numbered rows of the CMOS sensor 23 is executed. When the image quality priority mode is selected, all pixel readout is executed. In any mode, all pixel collective reset is executed, and charge accumulation in all pixels 56 is started simultaneously. On the monitor 17, a moving image or still image of one of a normal image and a special image, or a moving image or still image of each image is simultaneously displayed.

  When inserting the insertion unit 13 into the subject, the operator selects the normal imaging mode and performs the insertion operation while observing the normal image displayed on the monitor 17. When a lesion that requires detailed observation is found, the special imaging mode is selected, and a special image obtained by irradiating special light in a wavelength band corresponding to the type of the lesion is observed. When it is desired to compare the normal image and the special image for observation, the simultaneous shooting mode is selected. The frame rate priority mode is selected when importance is attached to the simultaneity of the normal image and the special image, and the image quality priority mode is selected when importance is attached to the image quality. Then, as needed, one of the special images is usually acquired. When treatment is required at the site to be observed, various treatment tools are inserted into the forceps channel of the electronic endoscope 10 to perform treatment such as excision of a lesion or medication.

  As described above, since the imaging signal of the CMOS sensor 23 is thinned and read out during the illumination light extinguishing period, and the normal image and the special light are normally acquired in units of the accumulation period of the CMOS sensor 23, the special image is acquired. While using the sensor 23, a special image can usually be observed simultaneously. Further, by executing the thinning-out readout, it is possible to prevent the frame rate from being lowered, and usually it is possible to ensure the simultaneity of special images.

  In addition to the frame rate priority mode for thinning readout, the image quality priority mode for all pixel readout is provided, and these modes can be switched, so it is possible to switch to a more appropriate mode according to the mode of observation and treatment. The convenience is improved as compared with the case where the mode is provided independently.

  In the simultaneous imaging mode, the charge accumulation start timing is aligned for each horizontal line by all pixel batch reset. Further, by taking advantage of the fact that the electronic endoscope 10 is used in a dark place inside the subject, the light source 40 is turned off to substantially align the end timing of charge accumulation in each horizontal line. For this reason, while the imaging device of the electronic endoscope 10 is configured simply and inexpensively, the charge accumulation period does not shift in each horizontal line as in the rolling shutter system, and a subject with a large movement can be captured. The image is not distorted.

  In the above embodiment, every other horizontal line is selected in the frame rate priority mode to obtain an observation image in which the effective number of pixels is half the total number of pixels, but without skipping all the horizontal lines. It is also possible to adjust the clock signal to be scanned and input to the horizontal scanning circuit 55 to turn on the column selection transistor every other column and acquire the imaging signal every other column. In addition, the clock signal input to the vertical scanning circuit 51 and the horizontal scanning circuit 55 may be adjusted so that the pixel 56 that outputs the imaging signal may have a so-called checkered pattern.

  In the above-described embodiment, all pixel collective reset is exemplified. However, after the imaging signal is read while selecting every other horizontal line, the signal charge may be sequentially discharged to the drain for each horizontal line and reset. In this case, the reset of the nth line of the reset shift register has the same timing as the reading of the vertical scan shift register of the (n + 1) th line. Similarly, the horizontal line is reset immediately after the imaging signal is output, and the horizontal line that is skipped and the imaging signal is not read out is sequentially (or simultaneously in a lump) after the reading is completed. The signal charge may be discharged to the drain.

  In the above-described embodiment, an example in which imaging signals are read every other row has been described, but the present invention is not limited to this. By using the vertical scanning circuit 70 of FIG. 8, the reading start row and the ending row may be selected, and only a predetermined number of consecutive rows out of all the horizontal rows may be partially read out.

  When the reading method of frame rate priority mode / horizontal pixel selection is used, line selection is performed according to the color filter array so that there is no unread color among RGB pixels in the normal photographing mode. Depending on the arrangement of the color filters, reading may be performed every two horizontal lines instead of every other horizontal line. In this case, the number of readout pixels in the vertical direction is 1/3 of all pixels. The line selection is similarly performed not only in the RGB array but also in the complementary color array. For example, when color filters are arranged in two vertical line cycles, reading is performed every two horizontal lines.

  Note that pixel interpolation in the DSP 33 is also performed according to the color arrangement of the readout pixels and the readout method (every horizontal line or every two horizontal lines). In addition, when the frame rate priority mode is selected, it is selected to read out the pixel of the color filter that transmits the reflected light or fluorescence according to the special light to be irradiated.

  In FIG. 8, the vertical scanning shift register 71 of the vertical scanning circuit 70 is a serial input and parallel input-parallel output type having a plurality of D-type flip-flops 72 for horizontal lines. The flip-flop 72 is provided with an input terminal P for a preset signal and an input terminal C for a clear signal, similarly to the flip-flop 63 of the reset shift register 61 of FIG. A preset signal is individually input to each flip-flop 72. If “0” is input to the input terminal P and “1” is input to the input terminal C during the vertical scanning (shifting operation), data of “0” is held in all the flip-flops 72 of the vertical scanning shift register 71. The readout operation of the imaging signal is forcibly stopped. The configuration of the reset shift register is the same as that of the vertical scanning circuit 51 in FIG.

  When partial reading is performed, “0” is input to the input terminal P, “1” is input to the input terminal C, “0” is held in all the flip-flops 72, and then “0” is input to the input terminal C. Then, “1” is input to the input terminal P of the flip-flop 72 of the horizontal line from which reading is started. Then, after “0” is input to the input terminals P and C, the holding data is sequentially transferred between the flip-flops 72 in accordance with the clock signal as in the case of all pixel readout, and the horizontal line selected as the readout start row and thereafter. Are sequentially output. After completing the readout of the imaging signal of the horizontal line selected as the readout end row, “0” is input to the input terminal P, “1” is input to the input terminal C, and all the flip-flops 72 are held to be “0”. Is cleared, and the readout operation of the imaging signal is forcibly stopped.

  For example, when the first row is selected as the read start row and the center row is selected as the read end row, the upper half of the imaging area 50 of the CMOS sensor 23 is read, the center row is selected as the read start row, and the last row is selected as the read end row. If the lower half is read, the middle row between the first row and the middle row is selected as the reading start row, and the middle row between the middle row and the last row is selected as the reading end row, the middle portion reading is possible. If the horizontal line selected as the read start row is the first row, it is not necessary to input “1” to the input terminal P and hold “1” in the flip-flop 72 of the first row. The enable signal “1” may be input and a read operation may be performed according to the clock signal.

  Note that the pixels 56 to be thinned and read out are not limited to half of the imaging region 50. For example, in the case of using illumination light in a wavelength band where the exposure amount is insufficient, for example, the pixel 56 to be thinned and read is set to 1/3 of the whole to shorten the time taken to read the imaging signal, and increase the exposure time accordingly. May be. On the contrary, when a sufficient exposure amount can be obtained in a short time, the number of pixels 56 to be thinned and read out is increased to increase the time taken to read out the imaging signal, and instead the exposure time is shortened, An imaging signal may be obtained from many pixels to obtain an observation image with good image quality.

  In the above embodiment, the method of imaging with one CMOS sensor 23 in which the color filter is arranged in the imaging region 50 has been described as an example. However, illumination light in the wavelength band of each of RGB colors is sequentially irradiated to the site to be observed. A so-called frame sequential imaging method in which an image is captured in a time division manner by a monochrome CMOS sensor may be applied.

  In this case, for example, a disk-shaped rotary filter 80 shown in FIG. 9 is used. The rotary filter 80 has an R light irradiation region 81 for irradiating light in the wavelength bands of RGB colors, a G light irradiation region 82, a B light irradiation region 83, and a special light irradiation region 84 for irradiating special light. . Each of the regions 81 to 84 is provided in a section obtained by dividing the rotary filter 80 into six equal parts. A total of three special light irradiation regions 84 are arranged between the R light irradiation region 81, the G light irradiation region 82, and the B light irradiation region 83. The rotary filter 80 is disposed between the light source 40 and the condensing lens 44, and the rotary filter 80 is rotated so that illumination light and special light in the wavelength bands of RGB are sequentially emitted. Then, the CMOS sensor and the light source 40 are operated in the same manner as in the simultaneous shooting mode of the above embodiment.

  The normal image is generated based on the imaging signal of each image irradiated with the illumination light in the wavelength band of each RGB color. Specifically, a normal image is generated from each set continuous with RGB, GBR, and BRG among the output of imaging signals that follow RGBRGB. For this reason, the illumination light applied to the site to be observed changes as R light, special light, G light, special light, B light, special light,..., But the normal image and the special image are alternately output. .

  When acquiring only a special image, the special light irradiation region 84 of the rotary filter 80 is positioned between the light source 40 and the condenser lens 44. In the case of acquiring only a normal image, there is no special light irradiation region 84 that is arranged in a section obtained by dividing the R light irradiation region 81, the G light irradiation region 82, and the B light irradiation region 83 into three equal parts. A rotation filter is prepared, and this rotation filter is inserted between the light source 40 and the condenser lens 44 instead of the rotation filter 80 and rotated.

  In the above embodiment, an example in which a D-type flip-flop is used in the vertical scanning circuit has been described, but the present invention is not limited to this. For example, a clocked inverter, a combination of a clock train and a decoder, or the like may be used instead of the D-type flip-flop.

  Instead of the analog output type CMOS sensor 23 shown in FIG. 3, a CMOS sensor having another configuration may be used. For example, an A / D converter is mounted on the CDS circuit 52, the image pickup signal read from the pixel 56 is digitized and output in parallel, and the digitized image pickup signal is serialized and output through the parallel / serial conversion circuit. A thing may be used.

  Further, although one pixel 56 is composed of three transistors M1 to M3, one pixel may be composed of four transistors. In addition, the pixel selection transistor M2 is shared by the plurality of pixels 56, the signal of the photodiode D1 is transferred to the floating diffusion portion by the transistor, the transistors M1 and M2 are provided in the subsequent stage, and the photodiodes of the plurality of pixels 56 Although there are configurations such as a common floating diffusion unit transferred from D1, the present invention can be applied to any of the configurations.

  In the above embodiment, an example in which normal light and special light are used as illumination light to irradiate the observation site has been described. However, the normal light and special light used here are two types of illumination light having different wavelength bands. I just need it.

  In addition, illumination light of a specific wavelength band different from white visible light has been described as special light. Special light includes infrared light that highlights blood vessels and the intensity of autofluorescence of normal and diseased tissues. For emphasizing and displaying, the light of which the wavelength band is limited to a single color or several colors among white light corresponds. It is preferable that these various special lights have a configuration that can be freely selected according to the site or lesion to be observed, and the wavelength band and the components of the wavelength band to be mixed can be changed freely while observing the monitor. It is particularly preferable that the configuration be able to be made.

2 Electronic Endoscope System 10 Electronic Endoscope 11 Processor Device 12 Light Source Device 23 CMOS Image Sensor (CMOS Sensor)
26 Timing generator (TG)
27, 30, 42 CPU
40 Light source 41 Wavelength selection filter 51, 70 Vertical scanning circuit 56 Pixel 60, 71 Vertical scanning shift register 61 Reset shift register 62, 63, 72 Flip-flop 64a, 64b Changeover switch 65 Branch line

Claims (9)

A CMOS type image sensor that is provided in an insertion portion of an electronic endoscope that is inserted into a subject, and that images the inside of the subject;
Illumination means that repeatedly turns on and off the illumination light while switching the wavelength band of the illumination light to be irradiated in the subject,
A vertical scanning circuit for selectively reading an imaging signal from a pixel in an imaging region of the image sensor;
An electronic endoscope system comprising: control means for controlling an operation of the vertical scanning circuit so that an imaging signal is read while the illumination light is turned off.   2. The electronic endoscope according to claim 1, wherein the vertical scanning circuit is configured to selectively read out an imaging signal from a selected half or 1/3 of an imaging area of the image sensor. Mirror system.   The vertical scanning circuit is a shift register, and flip-flops for outputting a vertical scanning signal for reading an imaging signal to each horizontal line of pixels in the imaging region of the image sensor are connected every other horizontal line or every two horizontal lines. The electronic endoscope system according to claim 1, wherein an imaging signal is read out every other horizontal line or every two horizontal lines. The vertical scanning circuit is a shift register, and includes a flip-flop that outputs a vertical scanning signal for reading an imaging signal to each horizontal line of pixels in the imaging region of the image sensor,
3. The electronic endoscope system according to claim 1, wherein the flip-flop has a preset input terminal for forcibly holding data “1” or “0” and a clear input terminal. 4. The vertical scanning circuit is configured to read an imaging signal from all pixels in the imaging region of the image sensor,
5. An operation input unit that switches between selectively reading an imaging signal from a pixel in an imaging region of the image sensor or reading an imaging signal from all pixels in the imaging region of the image sensor. The electronic endoscope system according to any one of the above.   6. The electronic internal device according to claim 1, wherein the vertical scanning circuit is configured so that signal charges accumulated in all pixels of the image sensor can be collectively discharged to a drain. Endoscopic system. The vertical scanning circuit is a shift register, and has a flip-flop that outputs a reset signal for discharging a signal charge to a drain in each horizontal line of a pixel in an imaging region of the image sensor,
The electronic endoscope system according to claim 6, wherein the flip-flop has a preset input terminal for forcibly holding data “1” or “0” and a clear input terminal.   The electronic endoscope system according to any one of claims 1 to 7, wherein the illumination unit emits white light and light having a narrow wavelength band.   8. The electronic endoscope system according to claim 1, wherein the illuminating unit emits light in a wavelength band of each color of RGB and light in a narrow wavelength band.

Images