JP2009297133A - Apparatus and method for processing endoscope image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent deterioration in the quality of a spectrum estimation image by means of matrix arithmetic. <P>SOLUTION: When generating a spectrum estimation image SP by subjecting an endoscope image captured by an endoscope to the matrix arithmetic using a matrix parameter, an apparatus for processing endoscope image sets a matrix parameter M to be used for the matrix arithmetic, adjusts the gradation of the endoscope image P3 (P4) according to the set matrix parameter M, and thereafter, subjects the adjusted endoscope image P5 to the matrix arithmetic to generate the spectrum estimation image SP. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an endoscopic image processing apparatus and method for generating a spectral estimation image from an endoscopic image photographed using an endoscope.

  In recent years, in an electronic endoscope apparatus using a solid-state image sensor, spectral imaging combined with a narrow band-pass filter based on a spectral reflectance in a digestive organ (such as gastric mucosa), that is, an electronic endoscope apparatus incorporating a narrow band filter. (Narrow Band Imaging-NBl) is drawing attention. This device is provided with three narrow (wavelength) band-pass filters instead of the surface sequential R (red), G (green), and B (blue) rotary filters. By sequentially performing the same processing as in the case of the R, G, B (RGB) signals while changing the respective weights for the three signals obtained with these illumination lights, the spectral image is output. Is formed. According to such a spectral image, in the digestive organs such as the stomach and the large intestine, a fine structure or the like that has not been obtained conventionally is extracted.

On the other hand, it was obtained with white light in a simultaneous method in which a fine mosaic color filter is arranged in a solid-state imaging device, as shown in Patent Document 1, instead of a plane sequential type using the above-described narrowband bandpass filter. It has been proposed to form a spectral image by arithmetic processing based on an image signal. This is obtained as a matrix data (coefficient set) between the RGB color sensitivity characteristics converted into numerical data and the spectral characteristics of a specific narrowband bandpass converted into numerical data. A spectral image signal obtained by estimating a spectral image obtained through a narrow-band bandpass filter by calculation with an RGB signal is obtained. When a spectral image is formed by such an operation, it is not necessary to prepare a plurality of filters corresponding to a desired wavelength region, and replacement arrangement of these is unnecessary, so that the apparatus can be prevented from being enlarged and reduced in size. Cost can be reduced.
JP 2002-291682 A

  However, depending on the signal value of each RGB component of the normal observation image, performing the above-described matrix calculation may cause overexposure, blackout, or the like in the spectrally estimated image, thereby degrading the image quality of the spectrally estimated image. For example, even when the signal value of each RGB component in the normal observation image is within the maximum gradation expression number (for example, 255), the maximum gradation expression number of the spectral estimation image is exceeded when the matrix operation is performed. May end up. Since the signal values of the pixels exceeding the maximum number of gradation representations are all 255, the image quality deteriorates as a result.

  Further, in a normal observation image, there is a case where the whiteout has already occurred and exceeds 255 due to the halation of the light source, etc. However, when the spectral estimation image is used, the whiteout region increases and the image quality further deteriorates. There is.

  Therefore, an object of the present invention is to provide an endoscopic image processing apparatus and method that can prevent deterioration in image quality of a spectrally estimated image due to matrix calculation.

  An endoscopic image processing apparatus according to the present invention is an endoscopic image processing apparatus that generates a spectral estimation image by performing matrix calculation using matrix parameters on an endoscopic image acquired by an endoscope. A parameter setting means for setting matrix parameters used for matrix calculation, an image adjusting means for adjusting the gradation of the endoscopic image in accordance with the matrix parameters set by the parameter setting means, and an inner part adjusted by the image adjusting means Spectral image generation means for generating a spectral estimation image by performing a matrix operation on the endoscopic image is provided.

  An endoscopic image processing method according to the present invention is an endoscopic image processing method for generating a spectral estimation image by performing a matrix operation on an endoscopic image acquired by an endoscope using matrix parameters. Setting a matrix parameter used for matrix calculation, adjusting the gradation of the endoscopic image according to the set matrix parameter, and generating a spectral estimation image by performing matrix calculation on the adjusted endoscopic image It is characterized by.

  Here, the endoscopic image may be acquired using these component images after acquiring each RGB image separately by the frame sequential method, or acquired by the simultaneous method. Also good.

  In addition, the parameter setting means can set the matrix parameter used for the matrix calculation, regardless of the setting method. For example, when a predetermined wavelength is selected for each of the RGB components of the spectral estimation image, the parameter setting unit corresponds to the wavelength. A matrix parameter may be set, or a matrix parameter may be set based on an input from an input unit.

  The image adjusting means may be any method as long as it adjusts the gradation of the endoscopic image according to the matrix parameter. For example, the endoscope image processing apparatus further includes reference value detection means for detecting a reference pixel value used for gradation adjustment from the endoscope image, and the image adjustment means uses the reference pixel value detected by the reference value detection means. When the reference calculation value when matrix calculation is performed using the matrix parameter set in the parameter setting means is saturated with the set value, and the number of saturated pixels is larger than before the spectral estimation image is generated, You may adjust the gradation of a mirror image.

  In particular, the reference value detection means detects the maximum pixel value in the endoscopic image as the reference pixel value, and the maximum calculation value when the image adjustment means performs matrix calculation on the maximum pixel value using the matrix parameter is When the maximum gradation expression number set as the set value is saturated and the number of saturated pixels is larger than that of the endoscope image, the gradation of the endoscope image may be adjusted. Note that although the case where the gradation is adjusted based on the maximum pixel value of the endoscopic image is mentioned, the gradation may be adjusted based on the minimum pixel value or the average value of the endoscope image. . Note that “increased from the endoscopic image” means that it has increased compared to the normal observation image that was formed before the generation of the spectral estimation image.

  Furthermore, the image adjustment means has a gradation conversion table in which the adjustment amount of the gradation is set regardless of the method as long as it adjusts the gradation of the endoscope image, and uses the adjustment coefficient. In this case, a gradation adjustment unit that adjusts gradation conversion characteristics of the gradation conversion table may be provided. At this time, the adjustment of the gradation of the endoscopic image is performed by changing the change amount of the gradation conversion characteristic by changing the adjustment coefficient in accordance with the matrix parameter.

  Alternatively, the image adjustment unit may include a compression processing unit that generates a blur mask image from the endoscopic image, performs a dynamic range compression that is added to the endoscopic image after multiplying the blur mask image by a weighting coefficient. Good. At this time, the adjustment of the gradation of the endoscopic image is performed by changing the weighting coefficient according to the matrix parameter.

  Further, the endoscope image processing apparatus of the present invention includes a spectral image generation unit that generates a spectral estimation image by performing matrix calculation using a matrix parameter on a normal observation image acquired by an endoscope, and a spectral image An image composition unit that generates a composite image obtained by weighted addition of the spectral estimation image generated by the generation unit and the endoscopic image, and a composite image that is generated by the image composition unit by changing the weight according to the matrix parameter And image adjusting means for adjusting the gradation of the image.

  According to the endoscopic image processing apparatus and method of the present invention, when generating a spectral estimation image by performing matrix calculation on an endoscopic image acquired by an endoscope using matrix parameters, matrix calculation is performed. The matrix parameter to be used is set, the gradation of the endoscopic image is adjusted according to the set matrix parameter, and a matrix calculation is performed on the adjusted endoscopic image to generate a spectral estimation image. Changes in pixel values due to matrix computation using matrix parameters Predicted in advance and adjusted the gradation of the endoscopic image, the spectral estimated image is generated, so what endoscopic image is acquired and what Even if the matrix parameter is set, it is possible to prevent the deterioration of the image quality of the spectrally estimated image.

  The endoscopic image processing apparatus further includes reference value detection means for detecting a reference pixel value used for gradation adjustment from the endoscope image, and the image adjustment means detects the reference pixel value detected by the reference value detection means. When the matrix calculation using the matrix parameter set in the parameter setting means is saturated with the set value, and the number of saturated pixels is larger than that of the endoscopic image, the level of the endoscopic image When adjusting the tone, it is possible to prevent the deterioration of the image quality of the spectrally estimated image by adjusting the gradation in accordance with the characteristics of the endoscopic image.

  Further, the image adjusting means has a gradation conversion table in which the gradation change amount is set, and has gradation adjustment means for adjusting the gradation conversion characteristics of the gradation conversion table using the adjustment coefficient. The adjustment coefficient is changed according to the matrix parameter to adjust the amount of change in the gradation conversion characteristics of the endoscopic image, or the blur mask image is generated from the endoscopic image, and the weighting coefficient is applied to the blur mask image. It is provided with compression processing means for performing dynamic range compression to be added to the endoscopic image after multiplication, and to adjust the gradation of the endoscopic image by changing the weighting coefficient according to the matrix parameter. For example, by adjusting various coefficients in gradation processing or compression processing applied to an endoscopic image in a conventional endoscopic image processing apparatus, a new configuration It is possible to prevent deterioration in the image quality of the efficient spectral estimation image without adding.

  Further, the endoscope image processing apparatus of the present invention includes a spectral image generation unit that generates a spectral estimation image by performing matrix calculation using a matrix parameter on a normal observation image acquired by an endoscope, and a spectral image An image composition unit that generates a composite image obtained by weighting and adding the spectral estimation image generated by the generation unit and the endoscopic image, and a composite image generated by the image composition unit by changing the weight according to the matrix parameter Since the image adjustment means for adjusting the gradation of the image is provided, the pixel value of the composite image that changes according to the size of the matrix parameter can be predicted in advance, and the weight can be determined. Can be prevented from occurring.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows a basic configuration of an endoscope system 1 in which an endoscope image processing apparatus of the present invention is used. The endoscope system 1 includes a light source unit 10, a scope 20, and an endoscope image processing device 30.

  The light source unit 10 emits light for performing observation with an endoscope, and includes a xenon lamp that emits light L and the like. The light source unit 10 is optically connected to the light guide 15 of the scope 20, and the light L emitted from the light source unit 10 enters the light guide 15 and is irradiated onto the subject from the observation window 16.

  The scope 20 includes an imaging optical system 21, an image sensor 22, a CDS / AGC circuit 23, an A / D converter 24, a CCD driving unit 25, a lens driving unit 26, and the like. It is controlled by. The imaging element 22 is made of, for example, a CCD or a CMOS, and obtains an endoscope image P0 by photoelectrically converting the subject image formed by the imaging optical system 21. For example, a complementary color type having Mg (magenta), Ye (yellow), Cy (cyan), and G (green) color filters or a primary color type having RGB color filters is used as the image pickup element 22. . The operation of the image sensor 22 is controlled by the CCD drive unit 25. When the image pickup device 22 acquires an image (video) signal, a CDS / AGC (correlated double sampling / automatic gain control) circuit 23 samples and amplifies, and an A / D converter 24 outputs from the CDS / AGC circuit 17. The obtained endoscopic image is A / D converted and output to the endoscopic image processing apparatus 30.

  The endoscopic image processing apparatus 30 performs image processing of the endoscopic image P0 output from the scope 20, and includes an image acquisition unit 31, a gamma correction unit 32, an image adjustment unit 40, and a spectral image generation unit 50. The display controller 60 and the like are provided, and the operation of each component is controlled by the apparatus controller 70.

  The image acquisition means 31 is made up of, for example, a DSP (digital signal processor) or the like, and acquires an endoscopic image P imaged by the imaging element 22 of the scope 20. Note that when the image pickup device 22 acquires an endoscope image P0 composed of Mg (magenta), Ye (yellow), Cy (cyan), and G (green), a function of performing color conversion to an endoscope image P0 composed of RGB. have. When the imaging element 22 acquires an endoscopic image P composed of RGB components, the color conversion process is not necessary.

  The gamma correction unit 32 corrects each pixel value of the endoscopic image P0 based on the gamma curve with respect to the endoscopic image P0 acquired by the image acquisition unit 31, and outputs an endoscopic image P1. . The synchronization processing unit 33 generates one endoscopic image P2 using the color component images of each RGB component in the endoscopic image P1 acquired by the frame sequential method. In addition, when the endoscope image P is acquired by a method called a so-called frame sequential method that irradiates light of RGB colors at different timings, the above-described synchronization processing is unnecessary, but the so-called simultaneous method using white light When the endoscopic image P0 is acquired by a method called, a synchronization process is necessary.

  The image adjustment means 40 has a function of performing inverse gamma correction and returning to a linear signal P3 ′, a function of performing dynamic range compression processing on P3 ′, and a function of performing gradation adjustment processing, and gradation adjustment is performed. A normal observation image P5 is output (see FIG. 2 described later).

  The spectral image generation means 50 generates a spectral estimation image SP by performing matrix calculation on the normal observation image P5 using matrix parameters. Note that details of an operation example of the spectral image generation means 50 are described in JP-A-2003-93336.

なお、式（１）において、ＳＰｒ、ＳＰｇ、ＳＰｂは分光推定画像ＳＰの各ＲＧＢ成分、Ｐｒ、Ｐｇ、Ｐｂは通常観察画像Ｐ５の各ＲＧＢ成分、Ｍ_００〜Ｍ_２２からなる３×３行の行列はマトリクス演算を行うためのマトリクスパラメータＭをそれぞれ示している。 Specifically, the spectral image generation means 50 generates a spectral estimation image SP by performing a matrix calculation represented by the following formula (1).

In Expression (1), SPr, SPg, and SPb are RGB components of the spectral estimated image SP, and Pr, Pg, and Pb are RGB components of the normal observation image P5, and 3 × 3 rows including M _{00 to} M ₂₂ . The matrix indicates a matrix parameter M for performing a matrix operation.

  The display control unit 60 displays the spectral estimation image SP generated by the spectral image generation unit 50 on the display device 3 including a liquid crystal display device, a CRT, or the like. The display control unit 37 performs, for example, inverse gamma conversion, mirror image processing, processing for generating and displaying a mask image from various images, and processing for displaying information on the various images as character information, and performing these processing on the spectral estimation image SP. It has a function to display at the same time. Further, the display control means 60 has a function of displaying the normal observation P5 and the spectral estimation image SP on the display device 3.

  FIG. 2 is a block diagram showing an example of the configuration of the image adjusting means 40 and the apparatus controller 70. The image adjusting means 40 and the apparatus controller 70 will be described with reference to FIGS. As described above, the image adjustment unit 40 includes the inverse gamma conversion unit 40a, the compression processing unit 41, and the gradation adjustment unit 42. The inverse gamma conversion unit 40a performs inverse gamma conversion on the endoscope image P2 that has been gamma converted by the gamma correction unit 32.

The compression processing means 41 performs dynamic range compression processing on the endoscope image P. Specifically, the compression processing unit 41 generates a blurred image YL of the endoscopic image P3 ′ and a mask image generation that generates a blurred mask image (YL−Y0) using the blurred image YL. Means 41b and image addition means 41c for adding the blurred mask image (YL-Y0) and the endoscope image P3 ′ are provided. Here, the image addition means 41c multiplies the blurred mask image (YL-Y0) by the weighting coefficient k and adds the result to the endoscope image P3 as shown in the following equation (2).
P4 = P3 ′ + k (YL−Y0) (2)
Here, Y0 is a signal value serving as a reference for dynamic range compression, and when P3 ′ is designed to be 10 bits wide, Y0 = 512 is exemplified.

  Therefore, the compression processing means 41 performs a process of lifting the signal of the high density portion or the low density portion area in the endoscopic image P2 to an observable density area. In FIG. 2, the weighting coefficient k is set to a predetermined value set in advance. When k is set to a negative value, a dynamic range compression effect is obtained.

  The gradation adjusting unit 42 multiplies the RGB components of the endoscopic image P4 that has been subjected to the dynamic range compression processing by the compression processing unit 41 by the adjustment coefficients α, β, and γ, respectively. An endoscope image (normal observation image) P5 in which the gradation of each RGB component is adjusted is generated. Then, the spectral image generation unit 50 generates a spectral estimation image SP using the normal observation image P5 whose gradation is adjusted by the gradation adjustment unit 42.

  Here, the bit widths of P0 to P5, P3 ′, and SP are exemplified. For example, P0: 10 bits (number of bits after A / D conversion), P1: 10 bits, P2: 8 bits, P3: 8 bits, P3 ': 10 bits, P4: 10 bits, P5: 8 bits, SP: 8 bits. Here, the gradation adjusting unit 42 converts the endoscope image P4 (10 bits) to the normal observation image P5 (8 bits), and gradation conversion in which a gradation adjustment amount prepared in advance is set. The gradation is adjusted using a table. FIG. 3 is a graph showing an example of a preset gradation conversion table. FIG. 3 shows that black collapse occurs in the normal observation image P5 when a G signal smaller than the lower limit threshold GCmin of the endoscopic image P4 is input to the gradation adjusting means 42. In addition, when a G signal larger than the upper limit threshold GCmax of the endoscopic image P4 is input, whiteout occurs in the normal observation image P5. The gradation characteristics are corrected with adjustment coefficients α, β, and γ described later. Although FIG. 3 illustrates the G signal, the same applies to the R signal and the B signal. Note that the gradation adjusting unit 42 outputs a normal observation image P5 obtained by performing gamma correction on the endoscope image P4 subjected to dynamic range compression.

  On the other hand, the device controller 70 includes channel acquisition means 71, parameter setting means 72, reference value detection means 73, change amount calculation means 74, and the like. The channel acquisition means 71 is based on the channels CH1 to CH8 input from the input unit 2, and the wavelengths λ1, λ2, and λ3 when the spectral estimation image SP is generated and the set gain coefficient Rg used in the gradation adjustment means 42, Gg and Bg are acquired from the channel database CDB.

  FIG. 4 is a table showing an example of the channel database CDB. In the channel database CDB, as a wavelength set, for example, a standard set CH1 of (λ1, λ2, λ3) = (400, 500, 600), (λ1, λ2, λ3) = (470,500, 670) or (475, 510, 685) blood vessel set CH2, CH3, (λ1, λ2, λ3) = (440, 480, 520) or (480, 510, 580) tissue set for rendering a specific tissue CH5, hemoglobin set CH6 of (λ1, λ2, λ3) = (400, 430, 475) for depicting the difference between oxyhemoglobin and deoxyhemoglobin, (λ1, λ2) for depicting the difference between blood and carotene , Λ3) = (415,450,500) blood-carotene set CH7, (λ1, λ2) for depicting the difference between blood and cytoplasm , Λ3) = (420,550,600), eight wavelength sets of blood-cytoplasm set CH8 are stored. Further, the channel database CDB stores gain coefficients Rg, Gg, and Bg corresponding to the channels CH1 to CH8. Then, the channel acquisition means 71 acquires wavelength information λ1 to λ3 and set gain coefficients Rg, Gg, and Bg associated with the channels CH1 to CH8 input from the channel database CDB.

  The parameter setting unit 72 in FIG. 2 sets the matrix parameter M used when generating the spectral estimation image SP based on the wavelength information (λ1, λ2, λ3) acquired by the channel acquisition unit 71. . Specifically, the parameter setting unit 72 acquires the matrix parameter M0 corresponding to the wavelength set (λ1, λ2, λ3) from the parameter database PDB.

FIG. 5 is a table showing an example of the parameter database PDB. In the parameter database PDB, for example, parameters P _i = (M0 _j0 , M0 _j1 , M0 _j2 ) (i = 1 to 61, j = 0 to 2) are provided for each wavelength range obtained by dividing the wavelength range from 400 nm to 700 nm at 5 nm intervals. It is remembered. Then, the parameter setting unit 72 _extracts matrix parameters M0 _i = (M0 _j0 , M0 _j1 , M0 _j2 ) from the parameter database PDB for each of the wavelength information λ1, λ2, and λ3 selected by the channel acquisition unit 71. Then, the matrix parameter M0 of the above equation (1) is set.

2 detects the maximum pixel values R _HL , G _HL , and B _HL as reference pixel values among the RGB components of the endoscopic image P0. For example, as shown in FIG. The histogram Rh, Gh, Bh is calculated for each RGB component of the endoscope image P0, and the maximum pixel values R _HL , G _HL , B _HL are detected from the histograms Rh, Gh, Bh. .

The change amount calculation unit 74 in FIG. 2 adjusts the gradation based on the matrix parameter M0 set by the parameter setting unit 72 and the maximum pixel values R _HL , G _HL , B _HL detected by the reference value detection unit 73. The adjustment coefficients α, β, and γ used in the means 42 are calculated. Here, when predetermined channels CH1 to CH8 are designated in the input unit 2, the set gain coefficients Rg, Gg, and Bg of each RGB component and the matrix parameter M0 are determined. The gain coefficients Rg, Gg, Bg and the matrix parameter M0 are integrated as in the following expression (1 ′), and a new matrix M is generated.

Therefore, the pixel values SPr, SPg, and SPb of the spectral estimation image SP are expressed by the following equations (3-1) to (3-3) when the pixel values of the RGB components of the endoscope image P4 are Pr, Pg, and Pb. Can be expressed as:
SPr = M ₀₀ Pr + M ₀₁ Pg + M ₀₂ Pb ... (3-1)
SPg = M ₁₀ Pr + M ₁₁ Pg + M ₁₂ Pb (3-2)
SPb = M ₂₀ Pr + M ₂₁ Pg + M ₂₂ Pb ... (3-3)

When the pixel values SPr, SPg, and SPb of the spectral estimation image SP are calculated by the above formulas (3-1) to (3-3), the maximum number of gradation representations (for example, 255) of the spectral estimation image SP is exceeded. In some cases, this causes overexposure. Therefore, first, the reference value detection unit 73 detects the maximum pixel values R _HL , G _HL , and B _HL . Here, the maximum value can be detected more stably by using the cumulative histogram 99% in order to eliminate the influence of noise. Thereafter, R _HL ′, G _HL ′, and B _HL ′ are obtained as the maximum pixel values when gradation adjustment is performed using an initial gradation conversion table prepared in advance. These are substituted into the above equations (3-1) to (3-3), and the maximum of each RGB component when the maximum pixel values R _HL ′, G _HL ′, B _HL ′ are converted into the spectral estimation image SP. Calculation values SPr _max , SPg _max , and SPb _max are calculated and set as reference calculation values.
SPr _max = M ₀₀ R _HL '+ M ₀₁ G _HL ' + M ₀₂ B _HL '(4-1)
SPg _max = M ₁₀ R _HL '+ M ₁₁ G _HL ' + M ₁₂ B _HL '(4-2)
SPb _max = M ₂₀ R _HL '+ M ₂₁ G _HL ' + M ₂₂ B _HL '(4-3)

  6A and 7A are histograms of the G component in the endoscopic image P0, FIGS. 6B and 7B are histograms of the normal observation image P5, and FIGS. 6C and 6D and FIGS. 7C and 7D are matrix calculations with different parameters, respectively. The histograms of the G component SPg of the spectral estimation image SP are respectively shown. As shown in FIGS. 6A and 6B, even when whiteout does not occur in the normal observation image P5, when whiteout does not occur in the spectral estimated image SPg as shown in FIG. 6C, As described above, whiteout may occur in the spectral estimation image SPg. In particular, in FIG. 6D, it can be said that the number of whiteout pixels is increased compared to the normal observation image P5, and it can be said that the image quality is deteriorated when the spectral image is generated.

  Further, as shown in FIGS. 7A and 7B, when whiteout has already occurred in the endoscopic image P0 and the normal observation image P5, the number of whiteout pixels in the spectral estimation image SPg is large as shown in FIG. 7C. In some cases, as shown in FIG. 7D, the number of whiteout pixels in the spectral estimated image SPg is reduced. In particular, in FIG. 7C, since the number of whiteout pixels is increased compared to the normal observation image P5, it can be said that the image quality is deteriorated when the spectral image is generated.

That is, as shown in FIGS. 6 and 7, one condition is that “the spectrally estimated image is whiteout”. This corresponds to the fact that the reference calculation values SPr _max , SPg _max and SPb _max are saturated to the set value Cmax = 255. Another condition is that by calculating the number of saturated pixels from the histogram, the number of saturated pixels has increased from before the spectral formation. When both of the two conditions are satisfied, the adjustment coefficients α, β, and γ are calculated.

When it is determined that the state is that of FIGS. 6D and 7C, the gradation adjustment unit 42 calculates the adjustment coefficients α, β, γ by the following formulas (5-1) to (5-3) in the change amount calculation unit 74. Then, the gradation of each RGB component is adjusted using the calculated adjustment coefficients α, β, γ.
α = C _max / (M ₀₀ R _HL '+ M ₀₁ G _HL ' + M ₀₂ B _HL ') (5-1)
β = C _max / (M ₁₀ R _HL '+ M ₁₁ G _HL ' + M ₁₂ B _HL ') (5-2)
γ = C _max / (M ₂₀ R _HL '+ M ₂₁ G _HL ' + M ₂₂ B _HL ') (5-3)
Since P3 ′ is obtained by performing gamma conversion on P0 and then inverse gamma conversion, it can be considered that the gradation characteristics of P3 ′ and P0 are equivalent, and P0 is supplied to the reference value detection means 73. . Further, different adjustment values of α, β, and γ are obtained for each RGB, and instead of performing independent adjustment, an average value of α, β, γ, or a weighted average value such as 1: 2: 1 is obtained, and RGB Common adjustments may be made.

Next, an example of the endoscopic image processing method of the present invention will be described with reference to FIGS. First, when channels CH1 to CH8 are designated by the input unit 2, set gain coefficients Rg, Gg, and Bg corresponding to the channels CH1 to CH8 selected by the channel acquisition means 71 are determined, and wavelength information λ1, λ2 , Λ3 is determined. Then, the parameter setting means 72 extracts parameters (M0 _j0 , M0 _j1 , M0 _j2 ) suitable for the wavelength information λ1, λ2, and λ3 from the parameter database PDB, and generates a matrix parameter M. Furthermore, the gain coefficients Rg, Gg, Bg and the matrix parameter M0 are integrated to generate a new matrix parameter M.

On the other hand, an endoscopic image P0 photographed by the image sensor 22 is acquired by the image acquisition means 31, and the maximum pixel values R _HL , G _{HL of the} endoscopic image P0 from the histograms Rh, Gh, Bh by the reference value detection means 73. , B _HL is detected. Then, after the change amount calculation means 74 obtains values R _HL ′, G _HL ′, B _HL ′ obtained by passing R _HL , G _HL , B _HL through the gradation adjustment means 40, the above equation (4-1) (4-3) are calculated, and it is determined whether or not there is a pixel saturated in the maximum gradation expression number C _max in the spectral estimation images SPr, SPg, and SPb.

When it is determined that there is no pixel saturated in the maximum gradation expression number C _max in the spectral estimation images SPr, SPg, SPb, the gradation adjustment unit 42 performs gradation adjustment using default gradation conversion characteristics. Is called. On the other hand, even if there is a pixel that is saturated at the maximum gradation expression number C _max , the default gradation that is prepared in advance in the gradation adjusting means 42 even when the number of saturated pixels is not increased from before the spectral estimation. The gradation is adjusted using the conversion table.

On the other hand, if there is a pixel that is saturated at the maximum gradation expression number C _max and the number of saturated pixels is larger than before the spectral estimation, the change amount calculation means 74 uses the equations (5-1) to (5-3). ) To calculate the coefficients α, β, γ. Then, the gradation adjusting unit 42 uses the characteristics obtained by correcting the default gradation characteristics with the adjustment coefficients α, β, and γ (specifically, the inclination is α times, β times, and γ times). Adjustments are made.

As described above, when the spectral estimation image SP is calculated by matrix calculation, whether or not there is a pixel that is saturated at the maximum gradation expression number C _max and the number of saturated pixels is increased from before the spectral estimation. By adjusting the gradation of the endoscopic image P4 by predicting whether or not the spectroscopic estimated image SP is always overexposed, regardless of the pixel value of the endoscopic image P0 or the value of the matrix parameter M. It is possible to prevent image quality deterioration such as an increase. That is, for the endoscope image P3 having the G component of the histogram Gh as shown in FIGS. 6 and 7, the set gain coefficients Rg, Gg, and Bg are used without calculating the above-described coefficients α, β, and γ. When the gradation is adjusted, a portion where the pixel value of the G component spectral estimation image SPg is large is saturated to the maximum gradation expression number _Cmax . All the pixels saturated to the maximum gradation expression number _Cmax are overexposed because the pixel value = 255. On the other hand, by adjusting the gradation by changing the adjustment coefficients α, β, and γ according to the matrix parameter M as described above, image quality degradation such as an increase in whiteout in the spectral estimated image SP occurs. Can be prevented.

  FIG. 8 is a block diagram showing a second embodiment of the endoscopic image processing apparatus of the present invention. The endoscopic image processing apparatus will be described with reference to FIG. In the endoscopic image processing apparatus of FIG. 8, parts having the same configuration as those of the endoscopic image processing apparatus of FIG. The endoscope image processing apparatus in FIG. 8 is different from the endoscope image processing apparatus in FIG. 2 in that the gradation is adjusted by changing the weighting coefficient k when the dynamic range compression process is performed. In FIG. 8, the gradation adjusting unit 42 changes the gradation using the set gain coefficients Rg, Gg, and Bg corresponding to the channels CH1 to CH8 selected by the channel acquisition unit 71.

The change amount calculation unit 174 has a function of changing the weighting coefficient k in the above equation (2) using the maximum pixel values (reference pixel values) R _HL , G _HL , B _HL and the matrix parameter M. Here, as described above, when the predetermined channels CH1 to CH8 are designated, the pixel values SPr, SPg, and SPb of the spectral estimation image SP are calculated by the above formulas (3-1) to (3-3).

  The processes up to calculation of adjustment coefficients α, β, and γ for correcting gradation of RGB are the same as in the first embodiment. In the illustrated dynamic range compression, since common compression processing is performed for RGB, the average value of α, β, and γ is taken as η (η = (α + β + γ) / 3). It is not limited to this, Arbitrary weighting addition etc. may be sufficient. To obtain a dynamic range compression weighting factor k from η, set k = 1−η. However, dynamic range compression is a linear region, and spectral image estimation is a region after gamma correction, and the processing space is different, so a desired effect may not be obtained with common sense. In that case, the function of k = f (η) may be realized by the LUT format or function approximation.

Here, the endoscopic image P that has been subjected to the dynamic range compression processing by the compression processing means 41 can be expressed as in the above equation (1). Therefore, when Expression (1) is substituted into Expressions (3-1) to (3-3), they can be expressed as the following Expressions (6-1) to (6-3).
SPr = (M ₀₀ + M ₀₁ + M ₀₂ ) Rg ・ (Pr + kr (Pr-Y0r)) ... (6-1)
SPg = (M ₁₀ + M ₁₁ + M ₁₂ ) Gg ・ (Pg + kg (Pg-Y0g)) ・ ・ ・ (6-2)
SPb = (M ₂₀ + M ₂₁ + M ₂₂ ) Bg ・ (Pb + kb (Pb-Y0b)) ... (6-3)

Here, when the pixel values SPr, SPg, and SPb of the spectral estimation image SP are calculated, the range of the number of gradation bits (for example, 0 to 255) may be exceeded, which causes image quality degradation. Therefore, the change amount calculation unit 174 substitutes the maximum pixel values R _HL , G _HL , and B _HL detected by the reference value detection unit 73 into the above formulas (6-1) to (6-3), thereby obtaining the maximum value. Calculated values SPr _max , SPg _max and SPb _max are calculated.
SPr _max = (M ₀₀ + M ₀₁ + M ₀₂ ) Rg ・ (R _HL + kr (R _HL -Y0r)) (7-1)
SPg _max = (M ₁₀ + M ₁₁ + M ₁₂ ) Gg ・ (G _HL + kg (G _HL -Y0g)) (7-2)
SPb _max = (M ₂₀ + M ₂₁ + M ₂₂ ) Bg ・ (B _HL + kb (B _HL -Y0b)) (7-3)

When SPr _max, SPg _max, the value of SPb _max exceeds 255, the correction amount calculation unit 174 SPr _max, SPg _max, SPb _max value maximum gradation number _{C max} (eg 255,1023) The weighting coefficient k in Expression (2) is changed so as to perform such dynamic range compression. Here, when η = C _max / (M ₀₀ + M ₀₁ + M ₀₂ ), the above formulas (7-1) to (7-3) are
kr = (η / Rg-R _HL ) / (R _HL -Y0r) (8-1)
kg = (η / Gg -G _HL ) / (G _HL -Y0g) (8-2)
kb = (η / Bg -B _HL ) / (B _HL -Y0b) (8-3)
It becomes. This means that the slopes kr, kg, kb are changed according to the matrix parameter M as shown in FIG. The compression processing means 41 performs dynamic range compression on the endoscope image P3 using the weighting coefficient k changed by the change amount calculation means 174. In this way, by adjusting the degree of gradation change in the gradation adjusting means 42 in accordance with the matrix parameter M selected for each channel CH1 to CH8, any spectral estimated image SP of any channel CH1 to CH8 is adjusted. Degradation of image quality such as increased whiteout can be prevented.

  FIG. 10 is a block diagram showing a third embodiment of the endoscopic image processing apparatus of the present invention. The endoscopic image processing apparatus will be described with reference to FIG. In the endoscopic image processing apparatus of FIG. 10, parts having the same configurations as those of the endoscopic image processing apparatus of FIG. The endoscope image processing apparatus in FIG. 10 is different from the endoscope image processing apparatus in FIG. 2 in that a composite image P10 obtained by combining the spectral estimation image SP and the endoscope image P4 is output. The apparatus controller 270 in FIG. 10 has substantially the same configuration as that in FIGS. 2 and 8, but sets a predetermined weighting coefficient k for the output signal of the image adjustment means 40, and the amount of change The calculating means 74 and 174 calculate the weighting coefficient w.

Specifically, the endoscopic image processing apparatus 200 combines a composite image obtained by combining an image obtained by multiplying the spectral estimated image SP weighting coefficient w and an image obtained by multiplying the endoscopic image by the weighting coefficient (1-w). An image synthesizing unit 210 is provided. Then, the device controller 270 changes the weighting coefficient w, (1-w) according to the matrix parameter M. Here, when the pixel value of the endoscopic image after gradation adjustment is P and the pixel value of the spectral estimation image is SP, a composite image P10 output from the image composition unit 210 is:
P10 = SP ・ w + P (1-w) (9)
It can be expressed.

Here, when the composite image P10 is generated, the pixel value of the composite image P10 may exceed the range of the maximum gradation expression number C _max , which causes whiteout. Therefore, the change amount calculation means 74 uses the maximum pixel values R _HL , G _HL , B _{HL of the} endoscopic image P 0 and the above formula (9) and formulas (3-1) to (3-3) to generate a composite image P 10. The maximum pixel value P10 _max at is calculated for each RGB component. The device controller 270 determines whether or not the maximum pixel value P10 _max is saturated at the maximum gradation expression number C _max , and determines that the maximum pixel value is saturated at the maximum gradation expression number, The weighting coefficient w is calculated so that becomes the maximum gradation expression number _Cmax . Then, the image synthesis unit 210 generates a synthesized image using the calculated weighting coefficient w. Thereby, deterioration of the image quality of the composite image P10 can be prevented regardless of the type of the endoscopic image and the value of the matrix parameter M.

10 illustrates the case where the weighting coefficient w is adjusted according to the matrix parameter M, the embodiment of FIG. 2 and the embodiment of FIG. 8 are considered in consideration of the weighting coefficient w set to a predetermined value. The adjustment coefficients α, β, γ or the weighting coefficient k may be determined. At this time, the expression (9) is substituted into the maximum pixel values R _HL , G _HL , and B _HL of the expressions (3-1) to (3-3) or (8-1) to (8-3). Then, the adjustment coefficients α, β, γ or the weighting coefficient k are adjusted so that the pixels of the composite image P10 fall within the maximum gradation expression number C _max .

  According to each of the above embodiments, the matrix parameter used for the matrix calculation when generating the spectral estimated image SP by performing the matrix calculation on the endoscopic image acquired by the endoscope using the matrix parameter M. M is set, the gradation of the endoscopic image is adjusted according to the set matrix parameter M, and the spectral estimated image SP is generated by performing a matrix operation on the adjusted endoscopic image. Change in pixel value due to matrix calculation using matrix parameters is performed in advance, and after adjusting the gradation of the endoscopic image, the spectral estimated image SP is generated, so what endoscopic image is acquired and which Even if such matrix parameters are set, it is possible to prevent the degradation of the image quality of the spectrally estimated image SP.

Reference value detecting means 73 for detecting a reference pixel value used for gradation adjustment from the endoscopic image is further provided, and the image adjusting means 40 has a reference pixel value (maximum pixel value) detected by the reference value detecting means 73. ) Reference calculation values (maximum calculation values) SPr _max , SPg _max , SPb _max when matrix calculation is performed using the matrix parameter M set in the parameter setting means 72 for R _HL , G _HL , B _HL are set values (maximum) Number of gradation representations) When saturated with C _max and the number of saturated pixels is larger than that of the endoscopic image, when adjusting the gradation of the endoscopic image, the characteristics of the endoscopic image Therefore, it is possible to prevent the degradation of the image quality of the spectrally estimated image SP by adjusting the gradation in accordance with.

  The image adjusting unit 40 has a gradation conversion table in which the adjustment amount of gradation is set, and includes a gradation adjusting unit 42 that adjusts the gradation conversion characteristics of the gradation conversion table using the adjustment coefficient. The amount of change in the gradation conversion characteristics is adjusted by changing the adjustment coefficients α, β, and γ according to the matrix parameter M, or the blur mask image is generated from the endoscopic image, and the blur mask image is generated. Compression processing means 41 for performing dynamic range compression that is added to the endoscopic image after being multiplied by the weighting coefficient k is provided, and the level of the endoscopic image is changed by changing the weighting coefficient k according to the matrix parameter M. If the tone is to be adjusted, various coefficients α, β, γ, k in tone processing or compression processing applied to the endoscopic image in the conventional endoscopic image processing apparatus are adjusted. It is possible to prevent deterioration in the image quality of the efficient spectral estimation image SP by.

  Also, as shown in FIG. 10, a spectral image generation means 50 that generates a spectral estimated image SP by performing matrix calculation using a matrix parameter M on a normal observation image acquired by an endoscope, and spectral image generation The image synthesizing unit 210 that generates a synthesized image obtained by weighted addition of the spectral estimation image SP and the endoscopic image generated by the unit 50, and the image synthesizing unit 210 that changes the weighting according to the matrix parameter M. Provided with the image adjustment means 40 for adjusting the gradation of the synthesized image, the pixel value of the synthesized image that changes according to the size of the matrix parameter M can be predicted in advance, and weighting can be determined. It is possible to prevent the image quality from deteriorating in the image P10.

  The embodiment of the present invention is not limited to the above embodiment. In each of the above embodiments, the case of overexposure is described as an example. For example, in the image processing diagram, when the order of the gradation adjustment unit and the spectral image generation unit is reversed, that is, the matrix calculation for spectral image generation is performed in a linear region (region where gamma correction is not performed). There may be. In that case, it is also conceivable that black collapse increases when a spectral estimation image is generated. When applied to blackout, various coefficients are set so that the minimum signal value becomes a predetermined value instead of the maximum pixel value described above. Alternatively, the image adjustment unit 40 is not limited to the case where the maximum pixel value or the minimum pixel value is used, and may perform the gradation adjustment so that the pixel average value becomes a predetermined value.

  Moreover, in the said embodiment, although the case where the endoscopic image processing apparatus 30 was comprised using hardware, such as DSP, illustrated, the endoscopic image processing apparatus consists of computers, such as a personal computer. There may be. At this time, the configuration of the endoscopic image processing device 30 in FIG. 1 is realized by executing an endoscopic image processing program read into the auxiliary storage device on a computer (for example, a personal computer).

  Further, in the above embodiment, the case where various parameters change according to the channels CH1 to CH8 is illustrated, but each parameter may be set according to the input from the input unit 2.

1 is a block diagram showing a preferred embodiment of an endoscope system in which an endoscope image processing apparatus of the present invention is used. The block diagram which shows an example of the image adjustment means and apparatus controller in the endoscope image processing apparatus of FIG. Table showing an example of the channel database of FIG. Table showing an example of the parameter database in FIG. Histogram showing an example of the G component of the endoscopic image input to the gradation adjusting means of FIG. The histogram which shows an example of the G component of the endoscopic image which is acquired from the endoscope of FIG. Histogram showing an example of the G component of the normal observation image that is not whiteout and is output from the gradation adjusting means of FIG. Histogram showing an example of an unexposed G component of the spectral estimation image generated by the spectral image generation means of FIG. Histogram showing an example of an overgrown G component of the spectral estimation image generated by the spectral image generation means of FIG. Histogram showing an example of the G component of a whiteout endoscope image acquired from the endoscope of FIG. Histogram showing an example of the G component of the normal observation image that is out of focus output from the gradation adjusting means of FIG. Histogram showing an example of an overgrown G component of the spectral estimation image generated by the spectral image generation means of FIG. Histogram showing an example of an overgrown G component of the spectral estimation image generated by the spectral image generation means of FIG. The block diagram which shows 2nd Embodiment of the endoscopic image processing apparatus of this invention. The graph which shows a mode that the weighting coefficient k in the compression process means of FIG. 7 changes. The block diagram which shows 3rd Embodiment of the endoscopic image processing apparatus of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,200 Endoscope system 30 Endoscope image processing apparatus 40 Image adjustment means 41 Compression processing means 42 Gradation adjustment means 50 Spectral image generation means 70 Device controller 71 Channel acquisition means 72 Parameter setting means 73 Reference value detection means 74, 174 Change amount calculation means 210 Image composition means 270 Device controller CDB Channel database C _max Maximum gradation expression number (setting value)
G _HL , G _HL , B _HL maximum pixel value (reference pixel value)
k Weighting coefficient M Matrix parameter PDB Parameter database Rg, Gg, Bg Set gain coefficient α, β, γ Gain coefficient SP Spectral estimation image
SPr _max , SPg _max , SPb _maxx Maximum calculation value (reference calculation value)
w Weighting factor

Claims (7)

An endoscopic image processing apparatus that generates a spectral estimation image by performing a matrix operation using a matrix parameter on an endoscopic image acquired by an endoscope,
Parameter setting means for setting the matrix parameters used in the matrix calculation;
Image adjusting means for adjusting the gradation of the endoscopic image in accordance with the matrix parameter set by the parameter setting means;
An endoscopic image processing apparatus comprising: a spectral image generation unit configured to generate the spectral estimation image by performing the matrix operation on the endoscopic image adjusted by the image adjustment unit. A reference value detecting means for detecting a reference pixel value used for gradation adjustment from the endoscopic image;
When the image adjustment unit performs a matrix calculation on the reference pixel value detected by the reference value detection unit using the matrix parameter set by the parameter setting unit, a reference calculation value is saturated to a set value, The endoscope image processing apparatus according to claim 1, wherein when the number of pixels being increased is greater than that of the endoscope image, the gradation of the endoscope image is adjusted.   The reference value detection unit detects a maximum pixel value in the endoscopic image as the reference pixel value, and the image adjustment unit performs a matrix operation on the maximum pixel value using the matrix parameter. When the maximum calculated value is saturated to the maximum gradation expression number set as the set value and the number of saturated pixels is larger than that of the endoscope image, the gradation of the endoscope image is adjusted. The endoscopic image processing device according to claim 2, wherein the endoscope image processing device is a device.   The image adjustment means has a gradation conversion table in which a gradation adjustment amount is set, and includes a gradation adjustment means for adjusting a gradation conversion characteristic of the gradation conversion table using an adjustment coefficient. 4. The method according to claim 1, wherein the amount of change in gradation conversion characteristics of the endoscopic image is adjusted by changing the adjustment coefficient in accordance with the matrix parameter. The endoscope image processing apparatus according to claim 1.   The image adjusting unit includes a compression processing unit that generates a blur mask image from the endoscopic image, performs a dynamic range compression to be added to the endoscopic image after multiplying the blur mask image by a weighting coefficient. The endoscopic image according to any one of claims 1 to 3, wherein gradation of the endoscopic image is adjusted by changing the weighting coefficient in accordance with the matrix parameter. Processing equipment. An endoscopic image processing method for generating a spectral estimation image by performing a matrix operation using a matrix parameter on an endoscopic image acquired by an endoscope,
Set the matrix parameters used for the matrix calculation,
Adjust the tone of the endoscopic image according to the set matrix parameter,
An endoscope image processing method, wherein the spectral estimated image is generated by performing the matrix operation on the adjusted endoscope image. Spectral image generation means for generating a spectral estimation image by performing a matrix operation on a normal observation image acquired by an endoscope using a matrix parameter;
Image combining means for generating a combined image obtained by weighted addition of the spectral estimated image generated by the spectral image generating means and the endoscopic image;
An endoscopic image processing apparatus comprising: an image adjusting unit that adjusts a gradation of the synthesized image generated by the image synthesizing unit by changing the weighting according to the matrix parameter.

Images