JP7069290B2 - Endoscope system and how to operate it - Google Patents

Description

The present invention relates to an endoscope system that switches and emits a plurality of types of illumination light and a method of operating the same.

In the medical field in recent years, an endoscope system including a light source device, an endoscope, and a processor device is widely used. In the endoscope system, the observation target is irradiated with illumination light from the endoscope, and the observation target being illuminated by the illumination light is imaged by the image sensor of the endoscope, and the observation target is based on the RGB image signal obtained. The image of is displayed on the monitor.

Further, in the endoscope system, a plurality of observation modes are provided so that the illumination light to be applied to the observation target can be switched and the image processing for the image to be observed can be switched according to the purpose of diagnosis. There is. Further, as shown in Patent Document 1 and Patent Document 2, a plurality of observation images obtained from each illumination light can be alternately observed by automatically switching and emitting light from a plurality of illumination lights. ing.

However, when a plurality of illumination lights are switched to emit light, an observation image corresponding to the switched illumination light may be displayed when the illumination light is switched. Therefore, in Patent Document 1, when the illumination light is switched, the observation image may be displayed. , The amount of illumination light after switching is minimized.

Patent No. 5677378 International Publication No. 2010/055938

As described above, even when a plurality of illumination lights are switched to emit light, it is necessary to control the amount of emission of each illumination light according to the brightness state of the observation target. For example, in Patent Document 2, the brightness of the observation target is calculated from the image obtained by the emission of the illumination light before switching, and the emission amount of the illumination light after switching is calculated based on the calculated brightness of the observation target. It is in control. In this case, since the calculated brightness of the observation target is not based on the illumination light after switching, it may not be possible to accurately control the amount of light emitted. Further, when a rolling shutter type image sensor is used as the image sensor, the illumination light before switching and the illumination light after switching may be mixed and exposed at the time of switching the illumination light. From the image obtained in this case, it may not be possible to accurately control the amount of light emitted.

INDUSTRIAL APPLICABILITY The present invention provides an endoscope system capable of accurately controlling the amount of emitted illumination light even during the switching period of the illuminated light when the plurality of illuminated lights are switched and emitted, and an operation method thereof. The purpose.

The endoscope system of the present invention includes a light source unit, a light source control unit, a rolling shutter type image pickup sensor, and a brightness information calculation unit. The light source unit emits the first illumination light and the second illumination light whose emission spectrum is different from that of the first illumination light. The light source control unit switches between the first illumination light and the second illumination light to control the light emission. The image pickup sensor has a plurality of lines for capturing an observation object illuminated by the first illumination light or the second illumination light. The image pickup sensor performs exposure at different exposure timings for each line, reads out charges at different read timings for each line, and outputs an image signal. The image signal is switched between the first image signal output during the illumination period of the first illumination light, the second image signal output during the illumination period of the second illumination light, and the first illumination light and the second illumination light. The image signal of the switching period to be output is included in the switching period to be performed. The brightness information calculation unit calculates the first brightness information representing the brightness of the observation target in the illumination period of the first illumination light by using the first image signal and the first brightness calculation coefficient, and calculates the second image. Using the signal and the second brightness calculation coefficient different from the first brightness calculation coefficient, the second brightness information indicating the brightness of the observation target in the illumination period of the second illumination light is calculated, and the image of the switching period is calculated. Using the signal, the first brightness calculation coefficient, and the second brightness calculation coefficient, the brightness information of the switching period representing the brightness of the observation target in the switching period is calculated. The light source control unit controls the amount of light emitted from the first illumination light or the second illumination light based on the first brightness information, the second brightness information, or the brightness information during the switching period, and the brightness information calculation unit , By multiplying the image signal for the switching period by the brightness calculation coefficient for the switching period calculated based on the first brightness calculation coefficient and the second brightness calculation coefficient, the brightness information for the switching period can be obtained. calculate.

The brightness information calculation unit uses the brightness calculation coefficient for the switching period, which is obtained by weighting and adding the first brightness calculation coefficient and the second brightness calculation coefficient with a specific weighting coefficient, and the brightness of the switching period. It is preferable to calculate the information. It is preferable that the brightness information calculation unit calculates the brightness information of the switching period by using the brightness calculation coefficient for the switching period, which is the average value of the first brightness calculation coefficient and the second brightness calculation coefficient. The brightness information calculation unit calculates the brightness for the switching period by weighting and adding the weighting coefficient for the line corresponding to a specific line in the image pickup sensor to the first brightness calculation coefficient and the second brightness calculation coefficient. It is preferable to calculate the brightness information of the switching period using the coefficient . The brightness information calculation unit is for the switching period in which the weighting coefficient for the area corresponding to a specific area in the image signal of the switching period is weighted and added to the first brightness calculation coefficient and the second brightness calculation coefficient. It is preferable to calculate the brightness information of the switching period by using the brightness calculation coefficient .

The light source control unit preferably illuminates the first illumination light and the second illumination light at intervals of at least two frames or more. During the switching period, it is preferable that the first illumination light and the second illumination light are exposed to the image pickup sensor. The light source unit can emit light in a plurality of wavelength bands, and the emission ratio of the light in each wavelength band can be changed. The first illumination light has the first emission ratio and the second illumination light. Preferably has a second light emission ratio different from the first light emission ratio. The first illumination light is purple light, the second illumination light is green light, and the light source control unit preferably switches between purple light and green light to emit light. The first illumination light and the second illumination light include purple light, blue light, green light, or red light, respectively, and the light intensity ratio Vs1: of purple light, blue light, green light, or red light in the first illumination light. The first emission ratio indicating Bs1: Gs1: Rs1 is different from the second emission ratio indicating the light intensity ratio Vs2: Bs2: Gs2: Rs2 of purple light, blue light, green light, or red light in the second illumination light. Therefore, it is preferable that the light source control unit switches between the first illumination light and the second illumination light by switching between the first emission ratio and the second emission ratio.

The method of operating the endoscope system of the present invention includes a light emission control step, an imaging step, and a brightness information calculation step. In the light emission control step, the light source control unit switches between the first illumination light and the second illumination light whose emission spectrum is different from that of the first illumination light to control the emission. In the imaging step, a plurality of lines for imaging an observation target illuminated by the first illumination light or the second illumination light are provided, exposure is performed at different exposure timings for each line, and different readout timings are used for each line. A rolling shutter type image sensor that reads out the charge and outputs an image signal outputs the first image signal as an image signal during the illumination period of the first illumination light, and outputs the second image signal during the illumination period of the second illumination light. It is output, and the image signal of the switching period is output during the switching period in which the first illumination light and the second illumination light are switched. In the brightness information calculation step, the brightness information calculation unit uses the first image signal and the first brightness calculation coefficient to represent the brightness of the observation target in the illumination period of the first illumination light. And using the second image signal and the second brightness calculation coefficient different from the first brightness calculation coefficient, the second brightness information representing the brightness of the observation target during the illumination period of the second illumination light is obtained. The brightness information of the switching period representing the brightness of the observation target in the switching period is calculated by using the image signal of the switching period, the first brightness calculation coefficient, and the second brightness calculation coefficient. In the light emission control step, the light source control unit controls the light emission amount of the first illumination light or the second illumination light based on the first brightness information, the second brightness information, or the brightness information of the switching period. The brightness information calculation unit multiplies the brightness calculation coefficient for the switching period, which is calculated based on the first brightness calculation coefficient and the second brightness calculation coefficient, to the image signal for the switching period, thereby performing the switching period. Calculate the brightness information for.

According to the present invention, when a plurality of illumination lights are switched and emitted, the amount of emitted illumination light can be accurately controlled even during the switching period of the illumination lights.

It is an external view of the endoscope system of 1st Embodiment. It is a block diagram which shows the function of the endoscope system of 1st Embodiment. It is a graph which shows the emission spectrum of purple light V, blue light B, green light G, and red light R. It is a graph which shows the emission spectrum of purple light V. It is a graph which shows the emission spectrum of green light G. It is explanatory drawing which shows the operation of the image pickup sensor of a rolling shutter type. It is explanatory drawing which shows the method of calculating the brightness information of a switching period using the brightness calculation coefficient (the average value of the 1st brightness calculation coefficient α and the 2nd brightness calculation coefficient) for a switching period. A method of calculating the brightness information of the switching period using the brightness calculation coefficient for the switching period (the coefficient obtained by weighting and adding the first brightness calculation coefficient α and the second brightness calculation coefficient by the weighting coefficient for the line). It is explanatory drawing which shows. A method of calculating the brightness information of the switching period using the brightness calculation coefficient for the switching period (the coefficient obtained by weighting and adding the first brightness calculation coefficient α and the second brightness calculation coefficient by the weighting coefficient for the area). It is explanatory drawing which shows. It is a flowchart which shows the series flow of a multi-observation mode. It is a graph which shows the emission spectrum of the 1st illumination light which has a 1st emission ratio. It is a graph which shows the emission spectrum of the 2nd illumination light which has a 2nd emission ratio. It is a block diagram which shows the function of the endoscope system of 2nd Embodiment. It is a graph which shows the emission spectrum of normal light. It is a graph which shows the emission spectrum of the 1st illumination light. It is a graph which shows the emission spectrum of the second illumination 100 million.

[First Embodiment]
As shown in FIG. 1, the endoscope system 10 of the first embodiment includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 12a to be inserted into the subject, an operation portion 12b provided at the base end portion of the insertion portion 12a, and a curved portion 12c and a tip portion 12d provided on the tip end side of the insertion portion 12a. have. By operating the angle knob 12e of the operating portion 12b, the curved portion 12c bends. With this bending motion, the tip portion 12d is directed in a desired direction. The user interface 19 includes a mouse and the like in addition to the illustrated keyboard.

Further, the operation unit 12b is provided with a mode switching SW13a in addition to the angle knob 12e. The mode switching SW13a is used for switching between the normal observation mode, the first special observation mode, the second special observation mode, and the multi-observation mode. The normal observation mode is a mode in which a normal image is displayed on the monitor 18. The first special observation mode is a mode in which a first special image emphasizing surface blood vessels and the like is displayed on the monitor 18 by illuminating the observation target with purple light V (first illumination light). In the second special observation mode, a second special image emphasizing deep blood vessels is displayed on the monitor 18 by illuminating the observation target with green light G (second illumination light having a emission spectrum different from that of the first illumination light). It is a mode to do. In the multi-observation mode, purple light V and green light G are alternately switched at intervals of a specific number of emission frames to illuminate the observation target, and the first special image and the second special image obtained by the illumination of those lights are specified. The display is displayed on the monitor 18 by alternately switching at intervals of the number of display frames. In the present embodiment, the frame is a unit for controlling the image pickup sensor 48 (see FIG. 2) that captures an image of an observation target. The illumination period of the illumination light is represented by the number of frames (number of light emitting frames (see FIGS. 6 to 9 (simply referred to as “frame”))), and the display period of the image is also represented by the number of frames (number of display frames).

As the mode switching unit for switching the mode, a foot switch may be used in addition to the mode switching SW13a. Further, the operation unit 12b is provided with a freeze button (not shown) for acquiring a still image. When the user detects a part that seems to be effective for diagnosis, the mode switching SW13a and the freeze button may be operated alternately.

The processor device 16 is electrically connected to the monitor 18 and the user interface 19. The monitor 18 outputs and displays image information and the like. The user interface 19 accepts input operations such as function settings. An external recording unit (not shown) for recording image information or the like may be connected to the processor device 16.

As shown in FIG. 2, the light source device 14 includes a light source unit 20, a light source control unit 21, and an optical path coupling unit 23. The light source unit 20 can emit light in a plurality of wavelength bands, and the emission ratio of the light in each wavelength band can be changed. The light source unit 20 emits light in a plurality of wavelength bands, so that the V-LED (Violet Light Emitting Diode) 20a, B-LED (Blue Light Emitting Diode) 20b, G-LED (Green Light Emitting Diode) 20c, R- It has an LED (Red Light Emitting Diode) 20d. An LD (Laser Diode) may be used instead of the LED.

The light source control unit 21 controls the driving of the LEDs 20a to 20d. The optical path coupling unit 23 couples the optical paths of the four colors of light emitted from the four colors of LEDs 20a to 20d. The light coupled by the optical path coupling portion 23 is irradiated into the subject through the light guide 41 inserted into the insertion portion 12a and the illumination lens 45.

As shown in FIG. 3, the V-LED 20a generates purple light V having a center wavelength of 405 ± 10 nm and a wavelength range of 380 to 420 nm. The B-LED 20b generates blue light B having a center wavelength of 460 ± 10 nm and a wavelength range of 420 to 500 nm. The G-LED 20c generates green light G having a wavelength range of 480 to 600 nm. The R-LED 20d generates red light R having a central wavelength of 620 to 630 nm and a wavelength range of 600 to 650 nm.

The light source control unit 21 controls to light the V-LED20a, the B-LED20b, the G-LED20c, and the R-LED20d in any of the observation modes. Further, the light source control unit 21 emits normal light having a emission ratio of Vc: Bc: Gc: Rc among the purple light V, the blue light B, the green light G, and the red light R in the normal observation mode. , Each LED 20a to 20d is controlled. In the present specification, the luminous ratio refers to the light intensity ratio of each semiconductor light source, and includes the case where the light intensity ratio is 0 (zero). Therefore, this includes the case where any one or more of the semiconductor light sources are not lit. For example, as in the case where the light intensity ratio between purple light V, blue light B, green light G, and red light R is 1: 0: 0: 0, only one of the semiconductor light sources is turned on, and the other three. Even if one does not light up, it shall have a luminous ratio.

Further, the light source control unit 21 controls each of the LEDs 20a to 20d so as to emit only purple light V as shown in FIG. 4 in the first special observation mode. Further, the light source control unit 21 controls each of the LEDs 20a to 20d so as to emit only the green light G as shown in FIG. 5 in the second special observation mode. Further, the light source control unit 21 controls each of the LEDs 20a to 20d so that the purple light V and the green light G are alternately emitted at intervals of a specific number of light emitting frames in the multi-observation mode. The specific number of light emitting frames is preferably at least 2 frames or more. This is because when the number of light emitting frames is one, only a mixed color image in which purple light V and green light G are mixed can be obtained. Further, in order to synchronize the light source device 14 and the processor device 16, it is necessary to switch the parameters used in the processor device 16 in accordance with the switching to the illumination light in the light source device 14, and the parameters are switched. This is because at least two frames are required for the time required for the process. In addition, since blinking may occur due to switching of the illumination light, the burden on the operator due to blinking can be reduced by setting the period to two or more frames.

Further, the light source control unit 21 controls the amount of light emitted from the illumination lights emitted from the LEDs 20a to 20d based on the brightness information sent from the brightness information calculation unit 54 of the processor device 16. Specifically, in the normal observation mode, the amount of light emitted from the normal light is controlled based on the brightness information Yc in the normal observation mode. In the first special observation mode, the amount of violet light V emitted is controlled based on the first brightness information Y1. In the second special observation mode, the amount of green light G emitted is controlled based on the second brightness information Y2. In the multi-observation mode, the amount of emitted purple light V is controlled based on the first brightness information Y1 when the purple light V is illuminated, and green based on the second brightness information Y2 when the green light G is illuminated. Controls the amount of light G emitted. In the switching period in which the purple light V and the green light G are switched, the illumination light after the switching (for example, the green light G when switching from the purple light V to the green light G) is based on the brightness information Yx of the switching period. Light emission amount) is controlled.

As shown in FIG. 2, the light guide 41 is built in the endoscope 12 and the universal cord (the cord connecting the endoscope 12, the light source device 14 and the processor device 16), and is formed in the optical path coupling portion 23. The combined light propagates to the tip portion 12d of the endoscope 12. A multimode fiber can be used as the light guide 41. As an example, a fine fiber cable having a core diameter of 105 μm, a clad diameter of 125 μm, and a diameter of φ0.3 to 0.5 mm including a protective layer serving as an outer skin can be used.

An illumination optical system 30a and an image pickup optical system 30b are provided at the tip end portion 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 45, and the light from the light guide 41 is irradiated to the observation target through the illumination lens 45. The image pickup optical system 30b has an objective lens 46 and an image pickup sensor 48. The reflected light from the observation target is incident on the image pickup sensor 48 via the objective lens 46. As a result, a reflected image to be observed is formed on the image pickup sensor 48.

The image pickup sensor 48 is a color image pickup sensor, which captures a reflected image of a subject and outputs an image signal. The image pickup sensor 48 is preferably a CMOS (Complementary Metal-Oxide Semiconductor) image pickup sensor or the like. Therefore, as shown in FIG. 6, the image pickup sensor 48 has a plurality of lines Line 0 to Line n, exposes each line at different exposure timings, and reads out charges at different read timings for each line to obtain an image signal. Is output. For example, when the purple light V is emitted at intervals of one frame, in Line 0, the exposure timing is started at the emission start timing of the purple light V1 to perform the exposure, and the exposure is performed at the emission end timing of the purple light V1. End the timing. At the end of this exposure timing, the read timing is started, the charge is read out on Line 0, and the image signal is output.

In the next Line 1, the exposure timing is started and the exposure is performed after the time for Line 1 from the start of emission of the purple light V1, and the exposure timing is ended after the time for Line 1 from the start of emission of the purple light V2. At the end of this exposure timing, the read timing is started, the charge is read out on Line 1, and the image signal is output. For Line 2 to Line n, exposure and readout are performed so that the exposure timing and readout timing are different for each line. When the output of the image signal of Line n is completed, the first special image having the image signals of Line 0 to Line n is obtained.

Further, the image pickup sensor 48 is a color image pickup sensor for obtaining RGB image signals of three colors of R (red), G (green) and B (blue), that is, an R pixel and a G filter provided with an R filter. It is a so-called RGB image pickup sensor provided with a G pixel provided with and a B pixel provided with a B filter. Therefore, in the normal observation mode, the Bc image signal is output from the B pixel, the Gc image signal is output from the G pixel, and the Rc image signal is output from the R pixel by imaging the observation target illuminated by the normal light with the image sensor 48. ..

Further, in the first special observation mode, the observation target illuminated by the purple light V is imaged by the image pickup sensor 48, so that the B pixel to the Bs1 image signal, the G pixel to the Gs1 image signal, and the R pixel to the Rs1 image signal. Is output. Further, in the second special observation mode, the observation target illuminated by the green light G is imaged by the image pickup sensor 48, so that the B pixel to the Bs2 image signal, the G pixel to the Gs2 image signal, and the R pixel to the Rs2 image signal. Is output. Further, in the multi-observation mode, the Bs1 image signal, the Gs1 image signal, and the Rs1 image signal are output from the B pixel, the G pixel, and the R pixel when the purple light V is illuminated, and the B pixel and the G pixel are output when the green light G is illuminated. , Bs2 image signal, Gs2 image signal, and Rs2 image signal are output from the R pixel. Further, in the switching period in which the purple light V and the green light G are switched, the Bsx image signal, the Gsx image signal, and the Rsx image signal are output from the B pixel, the G pixel, and the R pixel.

The image sensor 48 is a so-called complementary color image sensor equipped with C (cyan), M (magenta), Y (yellow), and G (green) complementary color filters instead of the RGB color image sensor. May be. When using a complementary color image pickup sensor, the image signal of four colors of CMYG is output, so it is necessary to convert the image signal of four colors of CMYG into the image signal of three colors of RGB by the complementary color-primary color conversion. .. Further, the image pickup sensor 48 may be a monochrome image pickup sensor without a color filter. In this case, the light source control unit 21 needs to turn on the blue light B, the green light G, and the red light R in a time-division manner, and add a simultaneous process in the processing of the image pickup signal.

The image signal output from the image pickup sensor 48 is transmitted to the CDS / AGC circuit 50. The CDS / AGC circuit 50 performs correlated double sampling (CDS (Correlated Double Sampling)) and automatic gain control (AGC (Auto Gain Control)) on an image signal which is an analog signal. The image signal that has passed through the CDS / AGC circuit 50 is converted into a digital image signal by the A / D converter (A / D (Analog / Digital) converter) 52. The A / D converted digital image signal is input to the processor device 16.

The processor device 16 includes an image acquisition unit 53, a brightness information calculation unit 54, a DSP (Digital Signal Processor) 56, a noise removal unit 58, an image processing unit 60, a parameter switching unit 62, and a video signal generation unit. It is equipped with 66. A digital color image signal from the endoscope 12 is input to the image acquisition unit 53. The color image signal is derived from an R image signal output from the R pixel of the image sensor 48, a G image signal output from the G pixel of the image sensor 48, and a B image signal output from the B pixel of the image sensor 48. It is a constituent RGB image signal.

The brightness information calculation unit 54 calculates brightness information indicating the brightness of the observation target based on the RGB image signal input from the image acquisition unit 53. The calculated brightness information is sent to the light source control unit 21 and used for controlling the amount of light emitted from the illumination light. In the normal observation mode, based on the Bc image signal, Gc image signal, and Rc image signal, using the normal calculation coefficients αc, βc, and γc, the brightness of the observation target during the light emission period of normal light is according to the following (Equation 1). The brightness information Yc of the normal observation mode indicating the above is calculated.
(Equation 1) Yc = αc × Bc image signal + βc × Gc image signal + γc × Rc image signal

Further, in the first special observation mode, based on the Bs1 image signal, the Gs1 image signal, and the Rs1 image signal (first image signal), the first brightness calculation coefficients α1, β1, and γ1 are used according to the following equation 2). The first brightness information indicating the brightness of the observation target during the illumination period of the purple light V is calculated.
(Equation 2) Y1 = α1 × Bc1 image signal + β1 × Gc1 image signal + γ1 × Rc1 image signal

Further, in the second special observation mode, based on the Bs2 image signal, the Gs2 image signal, and the Rs2 image signal (second image signal), the second brightness calculation coefficients α2, β2, and γ2 are used according to the following equation 3). The second brightness information indicating the brightness of the observation target during the illumination period of the green light G is calculated.
(Equation 3) Y2 = α2 × Bc2 image signal + β2 × Gc2 image signal + γ2 × Rc2 image signal

Further, in the multi-observation mode, in the illumination period of the purple light V, the first brightness information indicating the brightness of the observation target in the illumination period of the purple light V is calculated in the same manner as in the case of the first special observation mode. In the illumination period of the green light G, the second brightness information indicating the brightness of the observation target in the illumination period of the green light G is calculated in the same manner as in the case of the second special observation mode. As shown in FIGS. 7 to 9, in the illumination period of purple light V, the first brightness image representing the first brightness information is an image having brightness corresponding to the first brightness information. .. On the other hand, during the illumination period of purple light, the second brightness image representing the second brightness information is a black image having a pixel value of substantially "0". On the contrary, in the illumination period of the green light G, the second brightness image representing the second brightness information is an image having the brightness corresponding to the second brightness information. On the other hand, during the illumination period of the green light G, the first brightness image representing the first brightness information is a black image having a pixel value of substantially "0".

Further, when the observation target is imaged by the rolling shutter type image pickup sensor 48, the purple light V and the green light G are mixed and exposed to the image pickup sensor 48 during the switching period. A mixed color image in which the illumination lights of these two colors are mixed is displayed on the monitor 18. Further, when the brightness information of the switching period indicating the brightness of the observation target in the switching period is calculated by the brightness calculation coefficient of either the first brightness calculation coefficient or the second brightness calculation coefficient, rolling. By taking an image of the observation target with the shutter type image pickup sensor 48, the brightness image of the switching period representing the brightness information of the switching period is a gradation image having spatially different brightness. In the case of such a gradation image, the brightness information calculation unit 54 cannot accurately calculate the brightness information during the switching period. The gradation image based on the first brightness calculation coefficient α1, β1 and γ1 gradually darkens from top to bottom, whereas the gradation image based on the second brightness calculation coefficient α2, β2 and γ2 becomes darker. , It gradually becomes brighter from top to bottom. The switching period is based on the illumination light after switching after starting exposure and reading of the first line (Line 0) of the image sensor 48 based on the illumination light before switching before switching the illumination light. Therefore, it is preferable that the last line (Line n) is the period until the exposure and the reading are completed. For example, the switching period is preferably at least 2 frames or more.

Therefore, even when the image pickup sensor 48 is exposed to a mixture of purple light V and green light G during the switching period, it is possible to accurately calculate the brightness of the observation target during the switching period. As possible, the brightness calculation coefficients αx, βx, and γx for the switching period are calculated based on the first brightness calculation coefficients α1, β1, γ1 and the second brightness calculation coefficients α2, β2, and γ2. Since the brightness calculation coefficients αx, βx, and γx for this switching period are calculated using the first brightness calculation coefficient and the second brightness calculation coefficient stored in advance in the processor device 16, the switching period There is no need to store dedicated coefficients. Therefore, the memory capacity in the processor device 16 can be suppressed. Then, based on the Bsx image signal, Gsx image signal, and Rsx image signal (image signal of the switching period), the brightness calculation coefficients αx, βx, and γx for the switching period are used, and the following (Equation 4) is used in the switching period. The brightness information Yx of the switching period indicating the brightness of the observation target is calculated. The specific image based on the brightness information Yx during the switching period is an image in which the spatial variation in brightness is suppressed as compared with the above-mentioned gradation image.
(Equation 4) Yx = αx × Bsx image signal + βx × Gsx image signal + γx × Rsx image signal

Here, the brightness information calculation unit 54 weights and adds the first brightness calculation coefficient Y1 and the second brightness calculation coefficient Y2 with a specific weighting coefficient, so that the brightness calculation coefficient for the switching period is added. Calculate αx, βx, and γx. For example, as shown in FIG. 7, the brightness information calculation unit 54 averages the first brightness calculation coefficients α1, β1, γ1 and the second brightness calculation coefficients α2, β2, γ2 according to the following (Equation 5). May be calculated as the brightness calculation coefficients αx, βx, and γx for the switching period.
(Equation 5) αx = (α1 + α2) / 2,
βx = (β1 + β2) / 2,
γx = (γ1 + γ2) / 2

Further, as shown in FIG. 8, the brightness information calculation unit 54 weights for a line corresponding to a specific line i (any of “0” to “n”) in the image pickup sensor 48 by the following (Equation 6). By weighting and adding the coefficients to the first brightness calculation coefficients α1, β1, γ1 and the second brightness calculation coefficients α2, β2, γ2, the brightness calculation coefficients αx, βx, γx for the switching period are added. It may be calculated as. When i is "0", αx is α1, βx is β1, and γx is γ1. When i is "n", αx is α2, βx is β2, and γx is γ2.
(Equation 6) αx = α1 × (n−i) / n + α2 × i / n
βx = β1 × (n−i) / n + β2 × i / n
γx = γ1 × (n−i) / n + β2 × i / n

Further, as shown in FIG. 9, the brightness information calculation unit 54 sets the area weighting coefficients corresponding to specific areas in the Bsx image signal, the Gsx image signal, and the Rsx image signal to the first brightness calculation coefficients α1 and β1. , Γ1 and the second brightness calculation coefficients α2, β2, and γ2 may be weighted and added to calculate the brightness calculation coefficients αx, βx, and γx for the switching period. For example, in the Bsx image signal, when there are two specific areas, an area P1 in the central portion and an area P2 around the area P1, the first brightness calculation coefficients α1 and β1 for the area P1. , Γ1 is used, and the second brightness calculation coefficients α2, β2, and γ2 are used for the area P2. In this case, among the Bsx image signals having a total number of pixels m, the number of pixels calculated by using the first brightness calculation coefficients α1, β1 and γ1 in the area P1 is p, and the second brightness calculation is performed in the area P2. When the number of pixels calculated using the coefficients α2, β2, and γ2 is q, the brightness calculation coefficients αx, βx, and γx for the switching period are calculated by the following (Equation 7).
(Equation 7) αx = p / m × α1 + q / m × α2
βx = p / m × β1 + q / m × β2
γx = p / m × γ1 + q / m × γ2
In (Equation 7), "p / m" and "q / m" correspond to the area weighting coefficients, respectively.

The DSP 56 performs various signal processing such as defect correction processing, offset processing, gain processing, color adjustment processing, gamma conversion processing, and demosaic processing on the received image signal. In the defect correction process, the signal of the defective pixel of the image sensor 48 is corrected. In the offset processing, the dark current component is removed from the RGB image signal subjected to the defect correction processing, and an accurate zero level is set.

In the gain processing, the signal level is adjusted by multiplying the RGB image signal after the offset processing by the gain parameter. After that, the brightness and saturation are adjusted by the gamma conversion process. The RGB image signal after the linear matrix processing is subjected to demosaic processing (also referred to as isotropic processing and simultaneous processing), and a signal of a color lacking in each pixel is generated by interpolation. By this demosaic processing, all pixels have signals of each color of RGB.

The noise removing unit 58 removes noise from the RGB image signal by performing noise removing processing (for example, a moving average method, a median filter method, etc.) on the RGB image signal that has been gamma-corrected by the DSP 56. The RGB image signal from which noise has been removed is transmitted to the image processing unit 60.

The image processing unit 60 performs various image processing on the RGB image signal. Various types of image processing include image processing performed under the same conditions regardless of the observation mode, and image processing performed under different conditions for each observation mode. The image processing performed under different conditions for each observation mode includes a color adjustment process for enhancing color reproducibility and a structure enhancement process for emphasizing various structures such as blood vessels and irregularities. The color adjustment process and the structure enhancement process are processes using a two-dimensional LUT (Look Up Table), a three-dimensional LUT (Look Up Table), a matrix, or the like. In the image processing unit 60, when performing the color enhancement processing and the structure enhancement processing, the color enhancement processing parameters set for each observation mode and the structure enhancement processing parameters are used. Switching of these color enhancement processing parameters or structure enhancement processing parameters is performed by the parameter switching unit 62 according to the operation of the mode switching SW 13a. Further, in the multi-observation mode, the parameter switching unit 62 switches to a parameter corresponding to the purple light V (similar to the first special observation mode) when the purple light V is emitted, and corresponds to the green light G when the green light G is emitted. Switch to the parameter to be used (similar to the second special observation mode).

The video signal generation unit 66 converts the normal image, the first special image, or the second special image input from the image processing unit 60 into a video signal for displaying as an image that can be displayed on the monitor 18. Based on this video signal, the monitor 18 displays an observation image corresponding to each observation mode. In the normal observation mode, the normal image is displayed on the monitor 18. In the case of the first special observation mode, the first special image is displayed on the monitor 18. In the case of the second special observation mode, the second special image is displayed on the monitor 18. In the multi-observation mode, the purple-tinged first special image and the green-tinged second special image are switched at intervals of a specific number of display frames and displayed on the monitor 18 (see FIGS. 7 to 9). ..

Next, the light source control in the multi-observation mode will be described with reference to the flowchart of FIG. When set to the multi-observation mode, the purple light V is illuminated on the observation target. Purple light V is emitted at intervals of a specific number of emission frames. A Bs1 image signal, a Gs1 image signal, and an Rs1 image signal can be obtained by imaging an observation target illuminated by purple light V with a rolling shutter type image sensor 48. By multiplying these Bs1 image signal, Gs1 image signal, and Rs1 image signal by the first brightness calculation coefficients α1, β1, and γ1, respectively, the first brightness indicating the brightness of the observation target during the illumination period of purple light is shown. Information can be obtained. The light source control unit 21 controls the amount of violet light V emitted based on the first brightness information.

When the emission of purple light V for a specific number of emission frames is completed, the emission is switched to green light G. During the switching period in which the purple light V and the green light G are switched, the Bsx image signal, the Gsx image signal, and the Rsx image signal are obtained by taking an image of the observation target with the image sensor 48. By multiplying these Bsx image signal, Gsx image signal, and Rsx image signal by the brightness calculation coefficients αx, βx, and γx for the switching period, respectively, the brightness of the switching period indicating the brightness of the observation target in the switching period. Calculate the information. The brightness calculation coefficients αx, βx, and γx for the switching period are the first brightness calculation coefficients α1, β1, and γ1 used when illuminating the purple light V, and the second brightness calculation used when illuminating the green light other than the switching period. It is obtained based on the coefficients α2, β2 and γ2. As a result, the brightness information can be accurately calculated even when the purple light V and the green light G are mixed and exposed to the image pickup sensor 48 during the switching period. Therefore, the amount of green light G (or purple light V) emitted can be accurately controlled.

After the switching period, the green light G is emitted at intervals of a specific number of light emitting frames. A Bs2 image signal, a Gs2 image signal, and an Rs2 image signal can be obtained by imaging the observation target illuminated by the green light G with the image sensor 48. By multiplying these Bs2 image signal, Gs2 image signal, and Rs2 image signal by the second brightness calculation coefficients α2, β2, and γ2, respectively, the second brightness of the observation target during the illumination period of the green light G is shown. Brightness information can be obtained. The light source control unit 21 controls the amount of green light G emitted based on the second brightness information. The above series of steps is repeated as long as the multi-observation mode continues.

In the above embodiment, in the multi-observation mode, the purple light V and the green light G are switched to emit light, but the first illumination light having the first luminous ratio and the second luminous ratio different from the first luminous ratio are emitted. The second illumination light having the above may be switched to emit light. For example, the first illumination light preferably has a peak at 400 nm or more and 440 nm or less in order to emphasize the surface blood vessels. Therefore, as shown in FIG. 11, the first illumination light emits first light so that the light intensity of the purple light V is higher than the light intensities of the other blue light B, the green light G, and the red light R. The ratio Vs1: Bs1: Gs1: Rs1 is set (Vs1> Bs1, Gs1, Rs1). Further, since the first illumination light has a red band such as red light R, the color of the mucous membrane can be accurately reproduced. Further, since the first illumination light has a blue band and a green band such as purple light V, blue light B, and green light G, various types such as ductal structure and unevenness are obtained in addition to the above-mentioned surface blood vessels. The structure can also be emphasized.

Further, it is preferable that the intensity ratio of at least one of 540 nm, 600 nm, or 630 nm is increased in the second illumination light in order to emphasize the deep blood vessels. Therefore, as shown in FIG. 12, the second illumination light is the light of the green light G or the red light R as compared with the light intensity of the blue light B, the green light G, and the red light R in the first illumination light. The second light emission ratio Vs2: Bs2: Gs2: Rs2 is set so that the intensity becomes high. Further, since the second illumination light has a red band such as red light R, the color of the mucous membrane can be accurately reproduced. Further, since the second illumination light has a blue band and a green band like purple light V, blue light B, and green light G, it emphasizes various structures such as unevenness in addition to the deep blood vessels as described above. be able to.

[Second Embodiment]
In the second embodiment, instead of the four-color LEDs 20a to 20d shown in the first embodiment, a laser light source and a phosphor are used to illuminate the observation target. Other than that, it is the same as the first embodiment.

As shown in FIG. 13, in the endoscope system 100 of the second embodiment, the light source device 14 emits a blue laser light having a center wavelength of 445 ± 10 nm instead of the four-color LEDs 20a to 20d (FIG. 13). In 13, a bluish-purple laser light source (denoted as “405LD” in FIG. 13) 106 that emits a bluish-purple laser light having a center wavelength of 405 ± 10 nm is provided. The light emitted from the semiconductor light emitting elements of each of the light sources 104 and 106 is individually controlled by the light source control unit 108, and the light intensity ratio between the emitted light of the blue laser light source 104 and the emitted light of the blue-violet laser light source 106 can be freely changed. It has become.

The light source control unit 108 drives the blue laser light source 104 in the normal observation mode. In the case of the first special observation mode, both the blue laser light source 104 and the blue-violet laser light source 106 are driven, and the emission ratio Lv1 of the blue-violet laser light is made larger than the emission ratio of the blue laser light Lb1. Control. In the case of the second special observation mode, both the blue laser light source 104 and the blue-purple laser light source 106 are driven, and the emission ratio Lb2 of the blue laser light is made larger than the emission ratio Lv2 of the blue-purple laser light. Control. In the case of the multi-observation mode, both the blue laser light source 104 and the blue-violet laser light source 106 are driven, and the emission ratio Lv1 of the blue-violet laser light is set to be larger than the emission ratio of the blue laser light Lb1. Control is performed to switch the control of increasing the light emission ratio Lb2 of the laser light to be larger than the light emission ratio Lv2 of the blue-violet laser light at intervals of a specific number of light emission frames.

The half-value width of the blue laser light or the blue-violet laser light is preferably about ± 10 nm. Further, as the blue laser light source 104 and the blue-violet laser light source 106, a broad area type InGaN-based laser diode can be used, and an InGaN As-based laser diode or a GaN As-based laser diode can also be used. Further, as the light source, a light emitting body such as a light emitting diode may be used.

In addition to the illumination lens 45, the illumination optical system 30a is provided with a phosphor 110 to which a blue laser beam or a blue-violet laser beam from the light guide 41 is incident. When the phosphor 110 is irradiated with a blue laser beam, fluorescence is emitted from the phosphor 110. Further, some blue laser light passes through the phosphor 110 as it is. The bluish-purple laser beam passes through the phosphor 110 without exciting it. The light emitted from the phosphor 110 is irradiated into the sample through the illumination lens 45.

Here, in the normal observation mode, since the blue laser light is mainly incident on the phosphor 110, the blue laser light as shown in FIG. 15 and the fluorescence excited and emitted from the phosphor 110 by the blue laser light are combined. Normal light is applied to the observation target. In the first special observation mode, both the blue-violet laser light and the blue laser light are incident on the phosphor 110. Therefore, as shown in FIG. 16, the blue-purple laser light, the blue laser light, and the blue laser light are used as the phosphor. The sample is irradiated with the first illumination light, which is a combination of fluorescence excited and emitted from 110. In this first special light, the light intensity of the blue-violet laser light is higher than the light intensity of the blue laser light.

Even in the second special observation mode, both the blue-violet laser light and the blue laser light are incident on the phosphor 110, so that the blue-violet laser light, the blue laser light, and the blue laser light as shown in FIG. 17 make the phosphor. The sample is irradiated with the second illumination light, which is a combination of fluorescence excited and emitted from 110. In this second special light, the light intensity of the blue laser light is higher than the light intensity of the blue-purple laser light. Further, in the multi-observation mode, both the blue-violet laser light and the blue laser light are incident on the phosphor 110, while the magnitude relationship between the blue-purple laser light and the blue laser light is switched. The 1 illumination light and the 2nd illumination light shown in FIG. 17 are switched and emitted at intervals of a specific number of emission frames.

The phosphor 110 is a plurality of types of phosphors that absorb a part of the blue laser light and excite and emit light from green to yellow (for example, a YAG-based fluorescent substance or a fluorescent substance such as BAM (BaMgAl ₁₀ O ₁₇ )). It is preferable to use one composed of. If a semiconductor light emitting element is used as an excitation light source for the phosphor 110 as in this configuration example, high-intensity white light can be obtained with high luminous efficiency, the intensity of the white light can be easily adjusted, and the color of the white light can be adjusted. Changes in temperature and chromaticity can be suppressed to a small extent.

In the above embodiment, the hardware of the processing unit included in the processor device 16, such as the image acquisition unit 53, the brightness information calculation unit 54, the DSP 56, the noise removal unit 58, the image processing unit 60, and the parameter switching unit 62. Structure is various processors (processors) as shown below. For various processors, the circuit configuration is changed after manufacturing the CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), which is a general-purpose processor that executes software (program) and functions as various processing units. It includes a programmable logic device (PLD), which is a possible processor, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for performing various processes, and the like.

One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). May be done. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units. Second, as typified by System On Chip (SoC), there is a form that uses a processor that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. be. As described above, the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.

Further, the hardware-like structure of these various processors is, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

10 Endoscope system 12 Endoscope 12a Insertion part 12b Operation part 12c Curved part 12d Tip part 12e Angle knob 14 Light source device 16 Processor device 18 Monitor 19 User interface 20 Light source part 20a V-LED (Violet Light Lighting Diode)
20b B-LED (Blue Light Emitting Diode)
20c G-LED (Green Light Emitting Diode)
20d R-LED (Red Light Emitting Diode)
21 Light source control unit 23 Optical path coupling unit 30a Illumination optical system 30b Imaging optical system 41 Light guide 45 Illumination lens 46 Objective lens 48 Imaging sensor 50 CDS / AGC circuit 53 Image acquisition unit 54 Brightness information calculation unit 56 DSP (Digital Signal Processor)
58 Noise removal unit 60 Image processing unit 62 Parameter switching unit 66 Video signal generation unit 100 Endoscope system 104 Blue laser light source 106 Blue purple laser light source 108 Light source control unit 110 Phosphorus

Claims (11)

A light source unit that emits a first illumination light and a second illumination light having a emission spectrum different from that of the first illumination light.
A light source control unit that switches between the first illumination light and the second illumination light to control light emission,
It has a plurality of lines for imaging an observation target illuminated by the first illumination light or the second illumination light, exposure is performed at different exposure timings for each line, and charges are applied at different read timings for each line. It is an image pickup sensor that reads out and outputs an image signal. The image signal includes a first image signal that is output during the illumination period of the first illumination light and a second image signal that is output during the illumination period of the second illumination light. And a rolling shutter type image pickup sensor including an image signal of the switching period output during the switching period for switching between the first illumination light and the second illumination light.
Using the first image signal and the first brightness calculation coefficient, the first brightness information representing the brightness of the observation target in the illumination period of the first illumination light is calculated, and the second image signal and the first brightness are calculated. Using a second brightness calculation coefficient different from the first brightness calculation coefficient, the second brightness information representing the brightness of the observation target in the illumination period of the second illumination light is calculated, and the image of the switching period. A brightness information calculation unit that calculates the brightness information of the switching period representing the brightness of the observation target in the switching period by using the signal, the first brightness calculation coefficient, and the second brightness calculation coefficient. Prepare,
The light source control unit controls the amount of light emitted from the first illumination light or the second illumination light based on the first brightness information, the second brightness information, or the brightness information of the switching period .
The brightness information calculation unit multiplies the image signal for the switching period by multiplying the brightness calculation coefficient for the switching period calculated based on the first brightness calculation coefficient and the second brightness calculation coefficient. An endoscope system that calculates brightness information for the switching period . The brightness information calculation unit uses the brightness calculation coefficient for the switching period, which is obtained by weighting and adding the first brightness calculation coefficient and the second brightness calculation coefficient with a specific weighting coefficient. The endoscope system according to claim 1, which calculates the brightness information of the switching period. The brightness information calculation unit calculates the brightness information of the switching period by using the brightness calculation coefficient for the switching period, which is the average value of the first brightness calculation coefficient and the second brightness calculation coefficient. 2. The endoscope system according to claim 2. The brightness information calculation unit weights and adds the line weighting coefficient corresponding to a specific line in the image pickup sensor to the first brightness calculation coefficient and the second brightness calculation coefficient. The endoscope system according to claim 2, wherein the brightness information of the switching period is calculated by using the brightness calculation coefficient for the period. The brightness information calculation unit weights and adds the area weighting coefficient corresponding to a specific area in the image signal during the switching period to the first brightness calculation coefficient and the second brightness calculation coefficient. The endoscope system according to claim 2, wherein the brightness information for the switching period is calculated by using the brightness calculation coefficient for the switching period. The endoscope system according to any one of claims 1 to 5, wherein the light source control unit illuminates the first illumination light and the second illumination light at intervals of at least two frames or more. The endoscope system according to any one of claims 1 to 6, wherein the first illumination light and the second illumination light are exposed to the image pickup sensor during the switching period. The light source unit can emit light in a plurality of wavelength bands, and the emission ratio of the light in each wavelength band can be changed.
The endoscope system according to any one of claims 1 to 7, wherein the first illumination light has a first luminous ratio, and the second illumination light has a second luminous ratio different from the first luminous ratio. The first illumination light is purple light, and the second illumination light is green light.
The endoscope system according to any one of claims 1 to 8, wherein the light source control unit switches between the purple light and the green light to emit light. The first illumination light and the second illumination light include purple light, blue light, green light, or red light, respectively, and the purple light, the blue light, the green light, or the red color in the first illumination light. The first emission ratio indicating the light intensity ratio Vs1: Bs1: Gs1: Rs1 is the light intensity ratio Vs2: Bs2: of the purple light, the blue light, the green light, or the red light in the second illumination light. It is different from the second emission ratio indicating Gs2: Rs2,
The endoscope system according to claim 8, wherein the light source control unit switches between the first illumination light and the second illumination light by switching between the first luminous ratio and the second luminous ratio. A step in which the light source control unit switches between the first illumination light and the second illumination light whose emission spectrum is different from that of the first illumination light to control the emission.
It has a plurality of lines for imaging an observation target illuminated by the first illumination light or the second illumination light, exposure is performed at different exposure timings for each line, and charges are applied at different read timings for each line. A rolling shutter type image sensor that reads out and outputs an image signal outputs the first image signal as the image signal during the illumination period of the first illumination light, and outputs the second image signal during the illumination period of the second illumination light. And the step of outputting the image signal of the switching period during the switching period of switching between the first illumination light and the second illumination light.
The brightness information calculation unit calculates the first brightness information representing the brightness of the observation target in the illumination period of the first illumination light by using the first image signal and the first brightness calculation coefficient. Using the second image signal and the second brightness calculation coefficient different from the first brightness calculation coefficient, the second brightness information representing the brightness of the observation target during the illumination period of the second illumination light is calculated. Then, using the image signal of the switching period, the first brightness calculation coefficient, and the second brightness calculation coefficient, the brightness information of the switching period representing the brightness of the observation target in the switching period is calculated. Have steps and
A step in which the light source control unit controls the amount of light emitted from the first illumination light or the second illumination light based on the first brightness information, the second brightness information, or the brightness information of the switching period. Have ,
The brightness information calculation unit multiplies the image signal for the switching period by multiplying the brightness calculation coefficient for the switching period calculated based on the first brightness calculation coefficient and the second brightness calculation coefficient. A method of operating an endoscope system that calculates brightness information for the switching period .

Images