JP2013141474A - Color tone adjusting device and electronic endoscope device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a color tone adjusting device for keeping the color tone of an image to be picked up at a constant level, irrespective of a secular change of spectrum distribution of an illumination light.SOLUTION: The light from a light source 21 is irradiated via a light guide 22 and an image is picked up using an image pickup element 14. A white plate 24 is irradiated with the light from a branch fiber bundle 22A of the light guide 22 and a light reflected on the plate 24 is picked up using a second image pickup element 32. RGB signals obtained in advance by picking up the white plate 24 by the first and second image pickup elements 14, 32 under the same illumination condition are stored in a memory 34 as a first RGB signal and a second RGB signal, respectively. A first gain for correcting the difference in the spectrum sensitivity distribution between the first and second image pickup elements 14, 32 is calculated from the first and second RGB signals. A second gain for correcting a change in spectrum distribution of an illumination light is calculated from the first and second RGB signals detected by the second image pickup element 32. The gain of an RGB signal from the image pickup element 14 is adjusted on the basis of the first and second gains in a front stage video signal processing circuit 18.

Description

  The present invention relates to a color adjustment device for adjusting the color tone of a photographed image.

  In general, in an endoscope, light from a light source is guided to a distal end of an insertion portion through a light guide and used as illumination light. For this reason, when the spectral distribution of the light source changes, the color balance of the acquired image changes and adversely affects the diagnosis and the like. For example, in an electronic endoscope apparatus having two light sources that can be switched, the spectral distribution changes due to the switching of the light sources. As such an electronic endoscope, a configuration is known in which correction data for white balance adjustment is prepared in advance for each light source, and white balance processing is corrected according to the switched light source to maintain a constant color balance. (Patent Document 1).

  In addition, as a change in the spectral distribution of illumination light in the endoscope apparatus, the influence of a diaphragm for adjusting the amount of light is also known. That is, in an endoscope, a light beam is incident on the light guide through a general light amount adjusting diaphragm, but the spectral distribution of the light beam is not spatially uniform, and the color temperature is higher at the center of the light beam. Therefore, the spectral distribution of the light incident on the light guide changes depending on the size of the stop, and the color balance of the acquired image is not constant. For such a problem, a configuration is known in which a correction filter is inserted into and removed from the optical path in accordance with the size of the stop to suppress fluctuations in the spectral distribution of light incident on the light guide (Patent Document 2).

JP 2004-121549 A Japanese Patent Laid-Open No. 7-299028

  By the way, in addition to the above-mentioned changes in the spectral distribution of the light source, there are those over time. For example, the spectral distribution of the light source changes due to deterioration over time. In general, halogen lamps, metal halide lamps, xenon lamps, and the like are used as illumination light sources. However, the spectral distribution of these lamps does not stabilize immediately after lighting, and a certain amount of time is required for stabilization. It takes.

  An object of the present invention is to maintain a constant color tone of a photographed image regardless of a temporal change in the spectral distribution of illumination light.

  The color tone adjusting apparatus according to the present invention includes a first image sensor that performs imaging using illumination light, a second image sensor that branches a part of the illumination light and captures a reference color, and a first under the illumination light. The RGB signal when the reference color is imaged by the image sensor is used as the first reference RGB signal, and the RGB signal when the reference color is imaged by the second image sensor is recorded as the second reference RGB signal under illumination light. A first gain calculating means for calculating an RGB gain for absorbing a difference in characteristics between the first imaging element and the second imaging element from the recording means, the first reference RGB signal and the second reference RGB signal; Second gain calculating means for calculating an RGB gain for canceling the spectral distribution change of the illumination light from the RGB signal from the image sensor and the second reference RGB signal, and a gain for the RGB signal from the first image sensor. 1 and 2nd gain calculation It is characterized in that a gain adjustment means for adjusting, based on the RGB gain calculated using the stage.

  In order to correct the change including the change in the spectral distribution of the illumination light by the light guide, the illumination light is preferably light that has passed through the light guide. For example, a part of the light guide is branched, and light via the branched fiber bundle is received by the second image sensor. The reference color is preferably white. The gain adjustment unit may be configured to perform gain adjustment continuously only for a predetermined time from when the light source is turned on until the light source is stabilized. The second reference RGB signal is an RGB signal obtained by the second image sensor at the end of the previous use.

  An electronic endoscope apparatus according to the present invention includes the above-described color tone adjusting apparatus.

  The color tone adjustment method of the present invention performs imaging with the first image sensor using illumination light, performs imaging of the reference color with a part of the branched illumination light using the second image sensor, The RGB signal when the reference color is imaged with the first image sensor is used as the first reference RGB signal, and the RGB signal when the reference color is imaged with the second image sensor under illumination light is used as the second reference RGB signal. The RGB gain for absorbing the difference in characteristics between the first image sensor and the second image sensor is calculated from the first reference RGB signal and the second reference RGB signal, and the RGB signal from the second image sensor is calculated. The RGB gain for canceling the spectral distribution change of the illumination light is calculated from the second reference RGB signal, and the gain for the RGB signal from the first image sensor is calculated using the first and second gain calculating means. Adjust based on RGB gain It is characterized in Rukoto.

  According to the present invention, it is possible to maintain a constant color tone of a photographed image regardless of a temporal change in the spectral distribution of illumination light.

It is a block diagram which shows the structure of the electronic endoscope apparatus carrying the color tone adjustment apparatus of this embodiment. It is a flowchart of the color tone adjustment processing of the present embodiment. It is a flowchart of a 1st gain calculation process. It is a flowchart of a 2nd gain calculation process. It is a flowchart of the color tone adjustment process of a modification.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an electronic endoscope apparatus equipped with a color tone adjusting apparatus according to an embodiment of the present invention. In FIG. 1, only a part of the configuration relating to the illumination system and the imaging system is shown, and many other configurations are omitted.

  The electronic endoscope apparatus 10 mainly includes a scope 11 having a flexible tubular insertion portion, a processor device 12 to which the scope 11 is detachably connected, and a monitor 13 to be detachably connected to the processor device 12. Is done.

  At the distal end of the insertion portion of the scope 11, a first image sensor 14 such as a CCD or a CMOS for taking an endoscopic image is provided. The driving of the first image sensor 14 is controlled based on, for example, a drive signal from an image sensor drive circuit 15 provided in the scope 11, and the image sensor drive circuit 15 is supplied from a timing controller 16 provided in the processor device 12. A drive signal is generated based on the clock signal. A subject image is formed on the imaging surface of the first imaging element 14 via the imaging lens 14A, and an image signal of the subject image is generated.

  An image signal output from the first image sensor 14 is converted into a digital signal via an analog front end (AFE) 17 in the scope 11, for example, and sent to a pre-stage video signal processing circuit 18 in the processor device 12. In the pre-stage video signal processing circuit 18, predetermined well-known image processing including white balance processing of the present embodiment is performed on the video signal and is temporarily held in the image memory 19.

  The video signal held in the image memory 19 is output to the subsequent video signal processing circuit 20 at a predetermined timing. In the post-stage video signal processing circuit 20, the video signal is converted into a video signal of a predetermined standard and output to the monitor 13, for example. Note that a series of processing in the front-stage video signal processing circuit 18, the image memory 19, and the rear-stage video signal processing circuit 20 is performed based on a clock signal from the timing controller 16.

  Illumination light necessary for photographing a subject is incident on a light guide (optical fiber) 22 provided in the scope 11 via a condenser lens 25 from a light source 21 provided in the processor device 12, for example. 22 is transmitted to the distal end of the insertion portion of the scope 11. The light transmitted to the distal end of the insertion portion is irradiated toward the subject through the illumination lens 23. As the light source 21, for example, a halogen lamp, a metal halide lamp, a xenon lamp, or the like is used.

  In the light guide 22, a part of the fiber bundle 22A is branched in, for example, a connector portion 11A used for connection to the processor device 12 of the scope 11 to form a branched fiber bundle 22A. The distal end of the branch fiber bundle 22A is disposed in the connector portion 11A, and a white plate (white body) 24 serving as a reference color in white balance adjustment is provided at the tip of the branch fiber bundle 22A, for example, 45 ° with respect to the optical axis from the branch fiber bundle 22A. It is placed at an angle. That is, part of the light incident on the light guide 22 is irradiated by the branch fiber bundle 22A toward the white plate 24 inclined with respect to the optical axis of the branch fiber bundle 22A in the connector portion 11A.

  A second image sensor 32 for correction using a CCD, a CMOS, or the like is further provided in the connector portion 11A, and the second image sensor 32 reflects the light emitted from the branch fiber bundle 22A by the white plate 24. It is arranged in the direction. That is, the light irradiated from the branch fiber bundle 22 </ b> A is reflected by the white plate 24 and received by the second image sensor 32. In the present embodiment, the white plate 24 is made of a material (for example, the same material) having the same spectral reflectance as that of a white balance jig used for normal white balance adjustment.

  The second image sensor 32 is controlled based on, for example, a drive signal from the image sensor drive circuit 15, and an image signal output from the second image sensor 32 is, for example, via an analog front end (AFE) 33 in the scope 11. Are converted into digital signals and sent to the pre-stage video signal processing circuit 18 in the processor unit 12 via the connector. The values of the RGB signals detected by the first and second image sensors 14 and 32 can be recorded in a memory 34 provided in the processor device 12, and this embodiment, which will be described later, in the pre-stage video signal processing circuit 18. It is used for color tone adjustment processing.

  In the above configuration, in the present embodiment, the second image sensor 32 is used to monitor the temporal change in the spectral distribution of the illumination light, and based on this, the gain of the RGB signal output from the first image sensor 14 is determined as the illumination light. Is adjusted so as to cancel the temporal change in the spectral distribution (color tone adjustment processing), and the change in the color tone of the endoscope image due to the temporal change is prevented.

  That is, the change in the spectral distribution of the illumination light due to the change or deterioration of the state of the light source 21 or the light guide 22 over time is caused by irradiating the illumination light from the light guide 22 to the white plate 24 which is the reference color. Monitoring is performed by photographing with the image sensor 32. The change in the spectral distribution is, for example, when the white plate 24 is photographed in advance by the second image sensor 32 in a state where illumination light is irradiated, and the RGB signals at that time are used as second reference RGB signals (R2 ′, G2 ′, B2 ′). Recorded in the memory 34 and monitored by comparing the second reference RGB signals (R2 ′, G2 ′, B2 ′) with the RGB signals (R2, G2, B2) currently output from the second image sensor 32. Is done.

If there is no deterioration over time in the spectral sensitivity of the second image sensor 32, the change in the ratio between the components of the RGB signals is caused by the change in the spectral distribution of the illumination light. In the present embodiment, for the purpose of maintaining the brightness constant, the change of the spectral distribution is monitored from the value of the ratio R / G of the R signal to the G signal and the ratio B / G of the B signal to the G signal with the G signal as a reference.
R2 ′ / G2 ′ = β _R · (R2 / G2)
B2 ′ / G2 ′ = β _B · (B2 / G2)
Gains β _R and β _B (second gain) are obtained.

That under the current illumination, R signals obtained when photographing a white plate 24 by the second image sensor 32 R2, respectively beta _R times the B signal B2, when multiplied beta _B, second under current illumination The ratio of the RGB signals obtained by the image sensor 32 is equal to the ratio of the second reference RGB signals (R2 ′, G2 ′, B2 ′), and the change in the color tone of the image due to the change in the spectral distribution of the illumination light is canceled out. . When there is a change with time in the spectral sensitivity characteristic of the second image sensor 32, the gains β _R and β _B are offset including changes in the spectral distribution of illumination light and changes in the spectral sensitivity distribution of the second image sensor 32. The value to perform.

  On the other hand, since there are individual differences in the spectral sensitivity distribution of the image sensor, the ratios of the components of the RGB signals obtained by the first image sensor 14 and the second image sensor 32 are different even under the same illumination conditions. Therefore, in order to prevent a change in the color tone of the endoscopic image captured by the first image sensor 14 due to a change in the spectral distribution of the illumination light, not only the gain correction due to the change in the spectral distribution of the illumination light but also the first image sensor. 14, it is necessary to perform gain correction based on the difference in spectral sensitivity characteristics between the second imaging elements 32.

  The difference in spectral sensitivity distribution can be detected by photographing the reference color with the first image sensor and the second image sensor 32 under the same illumination light, and comparing the RGB signals of both at this time. In the present embodiment, when detecting the second reference RGB signals (R2 ′, G2 ′, B2 ′) of the second image sensor 32 (substantially at the same time), under the illumination light emitted from the light guide 22, The RGB signal obtained by the white balance adjustment process of the first image sensor 14 performed using the white balance jig made of the same material as the white plate 24 is recorded in, for example, the memory 34 as the first reference RGB signal (R1, G1, B1). Then, by comparing the first and second RGB signals, the difference in spectral sensitivity distribution between the first and second imaging elements 14 and 32 is quantified.

That is, if the spectral sensitivity distributions of the first and second image sensors 14 and 32 have not changed over time since the first and second reference RGB signals were detected, the RGB signals of the first image sensor 14 and the second Between the RGB signals of the image sensor 32, for example, using the gains α _R and α _B (first gain) with reference to the G signal, R1 / G1 = α _R · (R2 ′ / G2 ′)
B1 / G1 = α _B · (B2 ′ / G2 ′)
There is a relationship. That is, when the R signal and B signal obtained by the second image sensor 32 are multiplied by α _R and α _B respectively, the ratio of the RGB signals obtained by the second image sensor 32 is the RGB signal obtained by the first image sensor 14. Is converted to a value equal to the ratio of.

Therefore, in order to prevent a change in color tone of an image captured by the first image sensor 14 based on a change in the spectral distribution of illumination light, for example, when the G signal is kept constant, the R signal of the first image sensor 14 is expressed as α. _R · β _R times and B signal may be multiplied by α _B · β _B times.

  Next, the color tone adjustment processing of the present embodiment will be described with reference to the flowchart of FIG. 2 (and FIG. 1).

  The color tone adjustment processing of the present embodiment is executed in the pre-stage video signal processing circuit 18 when the light source 21 is turned on and imaging of an endoscopic image by the first imaging element 14 is started. In step S100, it is determined whether or not the combination of the scope 11 and the processor device 12 is the first time. That is, the values of the first reference RGB signal (R1, G1, B1) and the second reference RGB signal (R2 ′, G2 ′, B2 ′) recorded in the memory 34 are the scope 11 currently attached to the processor device 12. It is determined whether or not it is. The processing of step S100 is performed by comparing the scope model number recorded in the scope 11 with the scope model number stored in the memory 34 of the processor device 12 when the processor device 12 was used last time, for example. .

  If it is determined in step S100 that the combination of the scope 11 and the processor device 12 is the first combination, in step S102, the first reference RGB signal (R1, G1, R1) is stored from the memory 34 recorded as described above. B 1), the second reference RGB signals (R 2 ′, G 2 ′, B 2 ′) are read out and input to the pre-stage video signal processing circuit 18. At this time, the model number of the scope 11 attached to the processor device 12 is read from the scope 11 side, and the scope model number recorded in the memory 34 is updated and recorded.

In step S104, the gain α _R for absorbing the difference between the imaging elements described above based on the first reference RGB signal (R1, G1, B1) and the second reference RGB signal (R2 ′, G2 ′, B2 ′), α _B is calculated in the pre-stage video signal processing circuit 18 and recorded in, for example, the memory 34 (first gain calculation process). When the process of step S104 ends, the process returns to step S100, and it is determined again whether or not the combination of the scope 11 and the processor device 12 is the first time.

If it is determined in step S100 that the combination is not the first time, RGB signals (R2, G2, B2) are acquired from the second image sensor 32 in step S106. In step S108, the gain β for canceling the above-described change in the spectral distribution of the illumination light from the RGB signals (R2, G2, B2) and the second reference RGB signals (R2, G2, B2) of the second image sensor 32 acquired. _{R 1} and β _B are calculated by the pre-stage signal processing circuit 18 and recorded in, for example, the memory 34 (second gain calculation process). When the second gain calculation process S108 ends, the process returns to step S106, and the same process is repeated for the newly acquired RGB signals (R2, G2, B2) of the second image sensor 32.

  Next, details of the first gain calculation processing in step S104 will be described with reference to the flowchart of FIG.

In step S200, it is determined whether or not the value of R1 / G1 is equal to R2 ′ / G2 ′. If it is determined that they are not equal, the value of the gain α _R = (R1 / G1) · (G2 ′ / R2 ′) for the R signal is calculated and recorded in the memory 34 in step S202, and step S204 is executed. . On the other hand, when it is determined that the value of R1 / G1 is equal to R2 ′ / G2 ′, step S204 is immediately executed.

In step S204, it is determined whether or not the value of B1 / G1 is equal to B2 ′ / G2 ′. If it is determined that they are not equal, the value of the gain α _B = (B1 / G1) · (G2 ′ / B2 ′) with respect to the B signal is calculated and recorded in the memory 34 in step S206, and this process ends. Further, when it is determined that the value of B1 / G1 is equal to B2 ′ / G2 ′, this processing is immediately terminated. For example, 1 is given as a default value for the gains α _R and α _B.

  Next, details of the second gain calculation processing in step S108 will be described with reference to the flowchart of FIG.

At step S300, the using the value of the recorded alpha _R in the memory 34, the value of R1 / G1 whether equal to α _R · (R2 / G2) is determined. If it is determined that they are not equal, the value of the gain β _R = (R2 ′ / G2 ′) · (G2 / R2) for the R signal is calculated and recorded in the memory 34 in step S302, and step S304 is executed. . On the other hand, when the value of R1 / G1 is determined to be equal to _β R · (R2 / G2) immediately step S304 is executed.

In step S304, it is determined whether the value of B1 / G1 is equal to α _B · (B2 / G2) using the value of α _B recorded in the memory 34. If it is determined that they are not equal, the value of the gain β _B = (B2 ′ / G2 ′) · (G2 / B2) for the B signal is calculated and recorded in the memory 34 in step S306. The gain for the R signal of the first image sensor 14 in the signal processing circuit 18 is changed to α _R · β _R , and the gain for the B signal is changed to α _B · β _B , and this processing ends. Further, when it is determined that the value of B1 / G1 is equal to α _B · (B2 / G2), step S308 is immediately executed, and this process is terminated.

  As described above, according to the present embodiment, even during image shooting, the gain of the RGB signal is adjusted according to the change in the spectral distribution of the illumination light, and the color tone change of the captured image due to the change in the spectral distribution of the illumination light is adjusted. Can be prevented. In addition, since the illumination light passing through the light guide is detected, it is possible to correct not only the change in the spectral distribution of the light source but also the change in the spectral distribution due to the deterioration of the light guide over time.

  Next, a first modification of the present embodiment will be described with reference to FIG. In the present embodiment, it is assumed that the spectral sensitivity distribution of the image sensor does not change with time, but in the modification, a color tone adjustment process is performed assuming a spectral sensitivity distribution change with time that occurs in the image sensor. The electromechanical configuration is the same as that of the embodiment shown in FIG.

Considering changes in the spectral sensitivity distribution of the first image sensor 14 and the second image sensor 32, it is necessary to change the first gains α _R and α _B over time. Therefore, as shown in the flowchart of FIG. 5, the first gain calculation process is executed regardless of whether or not the combination of the scope 11 and the processor device 12 is the first time.

That is, the first reference RGB signal (R1, G1, B1) and the second reference RGB signal (R2 ′, G2 ′, B2 ′) are sequentially read from the memory 34 in steps S400 and S402, and in step S406, step S104 is read. A first gain calculation process similar to that of the first gain α _R and α _B is calculated.

In step S408, the RGB signals (R2, G2, B2) from the second image sensor 32 are acquired in the same manner as in step S106, and in step S410, the second gain calculation process is executed in the same manner as in step S108, and the second gain β _{R is obtained.} , Β _B are calculated. Then, the gain for the R signal of the first image sensor 14 is changed to α _R · β _R and the gain for the B signal is changed to α _B · β _B , and the process returns to step S408 again. In the modification, the second reference RGB signal (R2 ′, G2 ′, B2 ′) uses the RGB signal obtained by the second image sensor 32 when the previous use of the electronic endoscope is finished.

  As described above, in the modified example, the same effect as that of the embodiment can be obtained, and the secular change of the second image sensor can be dealt with.

  Note that the color tone adjustment process of the present embodiment may be performed only for a predetermined time from the start of lighting until the light source is stabilized. Further, it may be configured so that it is continuously executed only within the predetermined time, and thereafter adjusted at regular intervals such as 5 to 10 minutes in order to correct fluctuation due to deterioration of the light source and the light guide with time. In this case, the frequency (interval) at which correction is performed may be adjustable.

  In addition, the branch fiber bundle and the second image sensor of this embodiment can be arranged in the processor device.

  In this embodiment, a memory that stores basic data of the imaging system and the spectral reflectance of the white plate is provided in the processor device, but the memory may be provided on the scope side (for example, a connector unit). Even when the secular change of the spectral sensitivity distribution of the image sensor is not taken into account, the second reference RGB signal may be the RGB signal value obtained by the second image sensor at the previous use. What is measured in advance before shipment is stored in the memory as a default value. In addition, it is possible to use a reference color other than white.

DESCRIPTION OF SYMBOLS 10 Electronic endoscope apparatus 11 Scope 12 Processor apparatus 13 Monitor 14 1st image pick-up element 18 Previous stage video signal processing circuit 21 Light source 22 Light guide 22A Branch fiber 24 White plate 32 2nd image pick-up element 34 Memory

Claims (8)

A first image sensor that performs imaging using illumination light;
A second imaging element for imaging a reference color by branching a part of the illumination light;
The RGB signal when the reference image is captured by the first image sensor under the illumination light is used as the first reference RGB signal, and the reference color is imaged by the second image sensor under the illumination light. Recording means for recording the current RGB signal as a second reference RGB signal;
First gain calculating means for calculating an RGB gain for absorbing a difference in characteristics between the first image sensor and the second image sensor from the first reference RGB signal and the second reference RGB signal;
Second gain calculating means for calculating an RGB gain for canceling a change in spectral distribution of the illumination light from the RGB signal from the second image sensor and the second reference RGB signal;
And a gain adjusting unit that adjusts a gain for the RGB signal from the first image sensor based on the RGB gain calculated by using the first and second gain calculating units.   The color adjustment device according to claim 1, wherein the illumination light is light through a light guide.   The color tone adjusting apparatus according to claim 2, wherein the reference color is white.   4. The color tone adjustment device according to claim 3, wherein a part of the light guide is branched, and light through the branched fiber bundle is received by the second image sensor.   2. The color tone adjustment apparatus according to claim 1, wherein the gain adjustment unit performs gain adjustment continuously only for a predetermined time from when the light source is turned on until the light source is stabilized.   The color tone adjustment apparatus according to claim 1, wherein the second reference RGB signal is an RGB signal obtained by the second image sensor at the end of the previous use.   An electronic endoscope apparatus comprising the color tone adjusting apparatus according to claim 1. Imaging with the first image sensor using illumination light,
Using a part of the branched illumination light, a reference color is imaged by the second image sensor,
The RGB signal when the reference image is captured by the first image sensor under the illumination light is used as the first reference RGB signal, and the reference color is imaged by the second image sensor under the illumination light. When the RGB signal is recorded as the second reference RGB signal,
From the first reference RGB signal and the second reference RGB signal, an RGB gain for absorbing a difference in characteristics between the first image sensor and the second image sensor is calculated,
From the RGB signal from the second image sensor and the second reference RGB signal, an RGB gain for canceling the spectral distribution change of the illumination light is calculated,
A color tone adjustment method, comprising: adjusting gains for RGB signals from the first image sensor based on RGB gains calculated using the first and second gain calculating means.

Images