JP4875578B2 - Color conversion definition creation method, color conversion definition creation device, and endoscope system - Google Patents

Description

  The present invention relates to a color conversion definition creating method, a color conversion definition creating apparatus, and an endoscope system for creating a color conversion definition for converting the color of an image.

  Conventionally, in the medical field, an endoscope in which an elongate tube having an optical member or an image sensor attached at the tip is inserted into the body of a subject, and the inside of the subject is photographed to observe a tumor, a thrombus, etc. Mirror devices are widely used. By directly photographing the body of the subject, it is possible to grasp the color and shape of the lesion, which is difficult to understand with radiographic images, without damaging the subject externally, and it is necessary to determine the treatment policy etc. Easy information.

  Also, in recent years, it is provided with a plurality of narrow (wavelength) band-pass filters based on spectral reflectances of digestive organs or the like, and an internal view for obtaining a spectral image based on subject light passing through these narrow-band band-pass filters. Mirror devices have been widely used. According to the endoscope apparatus using this narrow band-pass filter, it is possible to observe the fine structure of the digestive organs such as the stomach and the large intestine, and to perform more accurate diagnosis.

  On the other hand, a color filter of each color of R (red), G (green), and B (blue) is attached to the image sensor without using a narrow band-pass filter, and obtained based on subject light that has passed through the color filter. Endoscopic apparatuses have also been used that perform predetermined arithmetic processing on a color image and generate a pseudo spectral image obtained through a narrow band-pass filter. Usually, a spectral matrix indicating the correspondence between R, G, and B color sensitivity values in an endoscope apparatus and spectral characteristic values in a predetermined narrow band band pass is prepared in advance and obtained by an imaging device. Generally, a color image is converted into a spectral image using a spectral matrix. Endoscopic devices that use color filters do not require replacement of narrow-band bandpass filters in accordance with the spectral reflectance of the stomach or intestine being observed, which reduces the size and cost of the device. There is an advantage that you can.

  Here, the endoscope device includes a light source that emits light, an optical probe that is inserted into the body of the subject, an image processing device that performs image processing on an image signal obtained by imaging the inside of the subject, In general, the display device includes a display device that displays an image. This optical probe has a light guide line that guides light emitted from a light source, an optical member that condenses light guided by the light guide line into the body of the subject, and light irradiated by the optical member is reflected by the subject. An imaging device that receives reflected light and generates an image signal is mounted inside, and at the time of imaging, it may be replaced with an optical probe that matches the position and shape of the imaging location (for example, stomach and intestines). It is common. However, if the optical probe is replaced, the spectral sensitivity characteristic changes, so that even if the same part of the same subject is imaged, the color of the obtained color image may be slightly different. As described above, in an endoscope apparatus, a color image is often used after being converted into a spectral image, and when the color of the color image changes, the wavelength band is completely different from the wavelength band corresponding to the target observation object. There is a possibility that a peak occurs in the band and correct diagnosis cannot be performed. For this reason, it is necessary to perform color conversion processing on the image signal so that the color of the color image approaches the target color. Examples of methods for performing color conversion processing on color images include a method that uses a matrix that linearly converts image colors for each color space, and a method that uses a look-up table that directly associates image colors with target colors. It has been known. In color conversion processing using a matrix, the color of the image is corrected using a simple linear equation, so the effort to design the matrix, the processing time required for color correction, and the storage capacity for storing the matrix However, since the nonlinear distortion of the color space cannot be corrected, the color of the image cannot be corrected sufficiently. On the other hand, color conversion processing using a lookup table has the advantage that color conversion with a high degree of freedom is possible. However, in order to improve the accuracy of the color conversion processing, when trying to associate many colors with the target color in the lookup table, the storage capacity for storing the lookup table is greatly increased. If you try to reduce the number of colors defined in the lookup table, the interpolation process for colors other than the defined colors will occur. Therefore, there is a problem that the color correction accuracy deteriorates. In an endoscope apparatus, it is necessary to prepare a plurality of look-up tables corresponding to the spectral sensitivity characteristics of each of a plurality of optical probes and to display images in real time. Development of a technique capable of correcting the color of an image with high accuracy and high speed while reducing the design effort is desired.

In this regard, Patent Document 1 discloses that color conversion using a linear matrix for adjusting luminance is performed before gamma processing for correcting the color gradation of an image, and hue and saturation are adjusted after gamma processing. An example of performing color conversion using a color difference matrix is described. Patent Document 2 describes a matrix and a look for a color image obtained by selecting one of a plurality of light sources. A technology that makes the look-up table common and simplified by adjusting matrix coefficients in accordance with the color temperatures of each of a plurality of light sources in an image processing apparatus that performs color conversion processing using both the up-table and the table Is described. According to the techniques described in Patent Document 1 and Patent Document 2, an increase in storage capacity can be suppressed, and the accuracy of color conversion can be improved.
JP 2001-86519 A Japanese Patent Laid-Open No. 2004-40544

  However, in the technique described in Patent Document 1, linear transformation of the color space is executed twice, but the nonlinear distortion of the color space cannot be corrected, and the color can be applied to the endoscope apparatus. The accuracy of the conversion is insufficient. Moreover, as shown in Patent Document 2, correction can be made using a matrix if the color temperatures of the plurality of light sources are different, but the optical characteristics of the imaging element, optical member, light source, and light guide line are complicated. The difference in spectral sensitivity between the entangled optical probes cannot be absorbed by the matrix alone. For this reason, when the technique described in Patent Document 2 is applied to an endoscope apparatus, a color conversion process using a common lookup table is performed without correcting differences in spectral sensitivity characteristics among a plurality of optical probes. As a result, there is a problem that the color of the image cannot be corrected sufficiently.

  In view of the above circumstances, it is an object of the present invention to provide a color conversion definition creation device and an endoscope system that can correct an image color with high accuracy while suppressing an increase in storage capacity.

The color conversion definition creating method of the present invention that achieves the above object includes a plurality of colors obtained by reading each of a plurality of color patches having different colors by an input device that reads the image and generates image data representing the image. A patch color data acquisition process for acquiring a plurality of patch color data expressing the color of each patch by coordinates on a predetermined color space;
Based on a difference between a plurality of patch color data and a plurality of target color data in which target colors for the plurality of color patches are expressed by coordinates in a color space, the plurality of patch color data are approximated to a plurality of target color data. A matrix generation process for generating a correction matrix representing a linear transformation of the color space;
Converts a plurality of conversion patch color data obtained by converting a plurality of patch color data by a correction matrix into a plurality of target color data, and a plurality of conversion patch colors for each color data representing coordinates in a color space. Generates a color conversion definition that defines color conversion that performs color correction of the correction amount obtained by linear interpolation based on some of the differences between the data and multiple target color data. And a color conversion definition generation process.

  According to the color conversion definition creating method of the present invention, a correction matrix for linearly converting color data in which colors are expressed in coordinates in a predetermined color space for each color space, and a plurality of color patches prepared in advance are read. The obtained plurality of patch color data is converted into a plurality of target color data, and the color data excluding the patch color data is obtained by linear interpolation based on the difference between the patch color data and the target color data. A color conversion definition for performing color correction of the correction amount is generated. The interpolation used here is linear, but the difference used for interpolation is a partial difference, so the linearity is local, and in all color spaces it is a non-linear that accurately reproduces the colors of all patches. Color conversion is realized. After correcting the linear distortion of the color space by the correction matrix, the color conversion definition is stored by correcting each color of the image by the local linear color conversion by the color conversion definition. Therefore, it is possible to suppress an increase in storage capacity and to realize highly accurate color correction processing.

  In the color conversion definition creating method of the present invention, the color conversion definition generation process includes color data in which a color is expressed by coordinates in a color space by using a coordinate axis, an angle around the coordinate axis, and a distance from the coordinate axis. It is preferable that the data is converted into color data in which a color is expressed by coordinates in a cylindrical coordinate color space representing lightness, hue, and saturation, and interpolation is performed in the cylindrical coordinate color space.

  In the color conversion definition creating method of the present invention, the cylindrical coordinate color space is preferably a color space whose brightness and saturation are linear with respect to the luminance of the color.

  By using a cylindrical coordinate color space whose lightness and saturation are linear with respect to the luminance of the color, linear interpolation processing can be easily executed.

In the color conversion definition creation method of the present invention, the color conversion definition generation process includes
For the color data to be interpolated, a selection process for selecting two conversion patch color data sandwiching the hue of the color data to be interpolated from among a plurality of conversion patch color data,
Reference data is generated by linearly interpolating the two conversion patch color data selected in the selection process with respect to each of lightness and saturation at a hue ratio between the two conversion patch color data and the color data to be interpolated. The reference generation process;
Each of two conversion patch color data and two target color data corresponding to the two conversion patch color data in a hue ratio between the two conversion patch color data selected in the selection process and the color data to be interpolated A reference correction process for generating reference correction data representing a correction amount for the reference data by linearly interpolating the difference;
When the saturation is 0, the saturation correction amount is 0, and when the saturation is the same as the saturation of the reference data, the saturation correction amount is equal to the saturation correction amount represented by the reference correction data. A saturation amount calculation process for calculating a saturation correction amount by applying the saturation of the color data to be interpolated to a function;
When the saturation is 0, the lightness correction amount is 0, and when the saturation is the same as the saturation of the reference data, the lightness correction amount is equal to the lightness correction amount represented by the reference correction data. It is preferable to have a lightness amount calculation process for calculating a lightness correction amount by applying the saturation of the target color data.

  By calculating the saturation correction amount and the lightness correction amount according to a linear function related to saturation, it is possible to suppress an increase in the load and processing speed required for the interpolation processing and execute the color conversion processing with sufficiently high accuracy.

  In the color conversion definition creating method of the present invention, the color conversion definition is preferably a lookup table in which color data before color conversion and color data after color conversion are associated with each other.

  Here, the color conversion definition according to the present invention may be a function other than a lookup table. Since the lookup table has a simple format but a high degree of freedom, it can be preferably applied to the present invention.

In the color conversion definition creating method of the present invention,
The correction matrix and the color conversion definition generation are used for a series of color conversion processing in which other color processing is performed between color correction using the correction matrix and color conversion according to the color conversion definition. ,
The color conversion definition generation process is as follows:
An inverse conversion process of performing an inverse conversion process of the color processing on the grid point data representing the grid points of the color space;
A color correction process for calculating the correction amount for the lattice point data subjected to the inverse conversion process by linear interpolation, and performing correction amount color correction on the lattice point data to generate correction lattice point data;
A forward conversion process of performing color processing between the correction grid point data and color conversion;
It is preferable to have a corresponding process of associating the corrected grid point data that has been subjected to color processing in the forward conversion process with the grid point data before the color conversion process.

  According to this preferred color conversion definition creation method, it is possible to generate a color conversion definition in which color data representing grid points in a color space and correction grid point data that is target color data in the color data are associated with each other, The accuracy of color interpolation processing using them can be improved.

The color conversion definition creating apparatus of the present invention that achieves the above object is obtained by reading each of a plurality of color patches having different colors by an input device that reads an image and generates image data representing the image. A patch color data acquisition unit for acquiring a plurality of patch color data expressing the colors of a plurality of color patches by coordinates on a predetermined color space;
Based on a difference between a plurality of patch color data and a plurality of target color data in which target colors for the plurality of color patches are expressed by coordinates in a color space, the plurality of patch color data are approximated to a plurality of target color data. A matrix generation unit that generates a correction matrix that represents linear transformation of the color space;
Converts a plurality of conversion patch color data obtained by converting a plurality of patch color data by a correction matrix into a plurality of target color data, and a plurality of conversion patch colors for each color data representing coordinates in a color space. Generates a color conversion definition that defines color conversion that performs color correction of the correction amount obtained by linear interpolation based on some of the differences between the data and multiple target color data. And a color conversion definition generation unit.

  According to the color conversion definition creating apparatus of the present invention, it is possible to generate a color conversion definition capable of accurately correcting the color of an image while suppressing an increase in storage capacity.

  Note that the color conversion definition creating apparatus is only shown here for its basic form, but this is merely to avoid duplication, and the color conversion definition creating apparatus referred to in the present invention includes the above basic form. In addition, various forms corresponding to the respective forms of the color conversion definition creating method described above are included.

An endoscope system of the present invention that achieves the above object includes a light source that emits light, an optical probe that includes an input device, and
The color conversion definition creation device;
A color conversion device that performs color conversion processing on image data obtained by an input device using the correction matrix and the color conversion definition generated by the color conversion definition creation device;
And a display device that displays an image represented by the image data that has been subjected to color conversion processing by the color conversion device.

  According to the endoscope system of the present invention, even when the optical probe is replaced, the color of the image can be corrected with high accuracy, and a high-quality medical image can be displayed on the display device.

  According to the present invention, the color of an image can be accurately adjusted even in an endoscope apparatus in which an optical probe is replaced.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of an endoscope system to which an embodiment of the present invention is applied.

  An endoscope system 1 shown in FIG. 1 guides and irradiates light into the body of a subject P, generates an image signal based on the reflected light, a light source device 20 that emits light, and an optical probe. The image signal obtained in 10 is subjected to predetermined image processing to generate a medical image obtained by photographing the inside of the subject P, and the medical image generated by the image processing device 30 is displayed on the display monitor 41. And a display device 40 for displaying. In the endoscope system 1, the optical probe 10 is detachably attached to the light source 20 and the image processing device 30. The optical probe 10 corresponds to an example of the optical probe according to the present invention, and the display device 40 corresponds to an example of the display device according to the present invention. The light source device 20 corresponds to an example of a light source according to the present invention.

  The optical probe 10 includes a flexible elongated probe unit 11, an operation unit 12 that operates the probe unit 11, and a light / signal guide 13 that connects the light source device 20, the image processing device 30, and the optical probe 10. It is configured. Hereinafter, the side of the optical probe 10 that is inserted into the body of the subject P is referred to as the front end, and the opposite side of the front end is referred to as the rear end.

  The operation unit 12 includes a bending operation lever 121 for bending the probe unit 11, a shooting button 122 for shooting a still image, a color adjustment button 123 for adjusting the color of the displayed image, and the like. Is provided.

  The light / signal guide 13 includes a light guide 131 that transmits light and a signal line 132 that transmits signals. The light guide 131 has a rear end connected to the light source device 20, guides the light emitted from the light source device 20 to the probe unit 11, and guides the light from the irradiation window 11 a provided at the tip of the probe unit 11. Irradiate toward P. The signal line 132 has a CCD 133 attached to the front end, and the rear end side is connected to the image processing apparatus 30. The reflected light reflected from the inside of the subject P by the light irradiated from the irradiation window 11a of the light guide 131 is collected by the optical member 134 provided at the tip of the probe unit 11, received by the CCD 133, and reflected light. Is generated. The CCD 133 has a plurality of light receiving sensors arranged side by side, and light is received by each of the plurality of light receiving sensors to generate image data in which an image is expressed by a plurality of pixels. In the present embodiment, color filters (see FIG. 2) in which R, G, B colors are arranged in a regular color pattern are attached at positions corresponding to the plurality of light receiving sensors constituting the CCD 133, When the light that has passed through the color filter is received by the CCD 133, image data representing a color mosaic image in which R, G, and B pixels are arranged in the same color pattern as the color pattern of the color filter is generated. The CCD 133 corresponds to an example of an input device referred to in the present invention.

  The generated image data is transmitted to the image processing device 30 through the signal line 132, and after being subjected to predetermined image processing in the image processing device 30, is transmitted to the display device 40.

  FIG. 2 is a schematic functional block diagram of the endoscope system 1.

  In FIG. 2, the display monitor 41 and the operation unit 12 of the optical probe 10 are not shown, and only main elements related to the generation of the image signal are shown.

  The light source device 20 also shown in FIG. 1 emits white light and is controlled by the overall control unit 360 of the image processing device 30.

  In addition to the CCD 133 shown in FIG. 1, the optical probe 10 also includes a color filter 140 in which each color of R, G, and B is arranged in a mosaic pattern with a regular color pattern, and an image signal generated by the CCD 133 as a digital image. An A / D converter 150 that converts data, an imaging controller 160 that controls processing in various elements in the optical probe 10, and the like are provided. In the present embodiment, the optical probe 10 generates RGB color space image data in which the color of the image is represented by R, G, and B colors and the components of the R, G, and B colors are linear in luminance. It will be explained as a thing.

  The image processing apparatus 30 includes a dark correction unit 310 that corrects black spots of image data sent from the optical probe 10 through the signal line 132, an AWB correction unit 320 that corrects white balance of the image data, and image data. By performing image correction processing using the linear matrix 331, a matrix correction unit 330 that corrects linear components in the spectral characteristics of the optical probe 10 including the CCD 133, a matrix generation unit 340 that generates the linear matrix 331, and an image A gamma correction unit 350 that performs gradation correction processing on the data, and a 3D-LUT correction unit that adjusts the color of the image by performing color correction processing using a three-dimensional lookup table (3D-LUT) 371 on the image data. 370, a 3D-LUT generation unit 380 that generates a 3D-LUT 371, and image data In addition, noise in the image such as synchronization processing for generating a color image in which each pixel is expressed by a mixed color of R, G, and B colors based on the mosaic image, sharpness processing for adjusting color sharpness, false color, etc. An image processing unit 390 that performs noise reduction processing for reducing image quality, extraction processing for extracting blood vessels in the image, and the like, converts image data into display image data for the display monitor 41 using the display LUT 391, and an optical probe 10 and an overall control unit 360 for controlling the overall processing of the image processing apparatus 30.

  FIG. 3 shows a flow of a series of processes until the optical probe 10 is attached to the image processing device 30 and the light source device 20 to image the subject P, and the captured medical image is displayed on the display monitor 41. FIG.

  When the optical probe 10 is attached to the image processing device 30 and the light source device 20 (step S1 in FIG. 3), the linearity corresponding to the spectral sensitivity characteristic of the optical probe 10 is taken before the subject P is actually imaged. A matrix 331 and a 3D-LUT 371 are generated. First, a chart image to be described later is read by the optical probe 10 (step S2 in FIG. 3), and a linear matrix 331 for correcting the spectral characteristics of the optical probe 10 is generated based on the reading result (step in FIG. 3). S3).

  The ideal RGB data C (R, G, B) imaged in a state where the influence of the light source or the like is excluded and depending only on the spectral reflectance of the subject P, and the wavelength of the spectral data extracted from the RGB data The relationship with the band F (λ1, λ2, λ3) is shown as follows.

  In addition, the storage unit 300 includes three wavelength bands F (λ1, λ2, λ3) based on the spectral reflectance of the imaging location for each imaging location of the subject P, and the basic spectral matrix shown in Equation (1). The spectroscopic parameters constituting A are stored in association with each other.

  For example, when imaging the stomach with the endoscope apparatus 1, three wavelength bands λ1, λ2, and λ3 based on the spectral reflectance in the stomach are acquired, and further, the three wavelength bands λ1, λ2, and λ3 are obtained. Based on the associated spectral parameters a1, a2,..., A9, a basic spectral matrix A shown in Expression (1) is generated.

  Subsequently, a chart image in which a plurality of color patches whose target color data T (r, g, b) for each of R, G, and B is previously known is arranged is photographed using the endoscope apparatus 1, Each patch color data D (dr, dg, db) generated by the probe 10 is transmitted to the matrix generation unit 330 via the dark correction unit 310 and the AWB correction unit 320 of the image processing apparatus 20. Patch color data D (dr, dg, db) is acquired. The process of step S2 in which the patch color data D is acquired corresponds to an example of the patch color data acquisition process according to the present invention.

  FIG. 4 is a diagram illustrating an example of the chart image.

  The chart image 500 shown in FIG. 4 has colors C (cyan), M (magenta), Y (yellow), R (red), G (green), B (blue), white, and skin color on a black background. A patch 501 is arranged. The plurality of color patches 501 correspond to an example of a plurality of color patches according to the present invention. In the storage unit 300 of FIG. 2, target color data for each patch color data obtained by photographing each of the plurality of color patches 501 is prepared. This target color data T is ideal color data obtained in a state where the influence of the light source or the like is eliminated and only the spectral reflection characteristics of each color patch 501 are dependent. The target color data T (r, g, b) and the patch color data D (dr, dg, db) are converted into the target spectral data Ft and patch spectral data Fd shown below by the basic spectral matrix A.

  Here, each patch color data D (dr, dg, db) includes the spectral sensitivity characteristic S (λ) of the light source device 20 shown in FIG. 2 and the spectral sensitivity characteristics R (λ), G (λ), In general, the target color data T (r, g, b) does not coincide with each other due to the influence of B (λ) or the like. The relationship between the patch color data D (dr, dg, db) generated by the optical probe 10 and the target color data T (r, g, b) is related to the spectral sensitivity characteristic S (λ) of the light source device 20 and the CCD 133. Using correction coefficients K (kr, kg, kb) determined by spectral sensitivity characteristics R (λ), G (λ), B (λ), etc.

Can be expressed as

  For each of the plurality of color patches 501 shown in FIG. 4, the correction coefficient K (kr, kg, kb) using the patch color data D (dr, dg, db) and the target color data T (r, g, b). ) Is determined, and the average value of the plurality of correction coefficients K (kr, kg, kb) is determined as the correction coefficient K (Kr, Kg, Kb) of the optical probe 10. A sensitivity correction matrix K representing the correction coefficient K (Kr, Kg, Kb) is expressed by Equation (6).

  By substituting equation (5) into equation (3), the patch color data D is converted into spectral data, and the corrected patch spectral data Ft ′ subjected to color correction is

Thus, when Expressions (1) and (6) are substituted into Expression (7), the following Expression (8) is generated for the corrected patch spectral data Ft ′ (λ1 ′, λ2 ′, λ3 ′).

  The matrix generation unit 340 in FIG. 2 generates a linear matrix 331 by combining the sensitivity correction matrix K and the basic spectral matrix A. The linear matrix 331 is an example of a correction matrix referred to in the present invention, and the processing in step S3 for generating the linear matrix 331 corresponds to an example of a matrix generation process referred to in the present invention.

  The linear matrix 331 converts color data in the RGB color space into spectral data and performs linear conversion in the luminance linear RGB color space. Originally, the target spectral data Ft (λ1, λ2, λ3) obtained by the equation (3) and the corrected patch spectral data Ft ′ (λ1 ′, λ2 ′, λ3 ′) obtained by the equation (7). Is ideal, but since the linear matrix 331 cannot correct the distortion of the non-linear color space, they do not match.

  The generated linear matrix 331 is transmitted to the matrix correction unit 330. The matrix correction unit 330 sequentially assigns the three wavelength components λ1, λ2, and λ3 constituting the target spectral data Ft (λ1, λ2, λ3) shown in Equation (3) to the R, G, and B color channels, The three wavelength components λ1 ′, λ2 ′, λ3 ′ constituting the corrected patch spectral data Ft ′ (λ1 ′, λ2 ′, λ3 ′) obtained in 7) are also sequentially assigned to the R, G, B color channels.

  In the following description, the target color data Tf shown in Equation (9), in which the target spectral data Ft is assigned to each of the R, G, and B color channels, is the target color data in each color patch 501, and the corrected patch spectral data Ft ′ is R, The corrected patch color data Daf shown in Expression (10) assigned to the G and B color channels will be described as patch color data in each color patch 501.

  When the linear matrix 331 is generated as described above, the 3D-LUT generation unit 380 generates the 3D-LUT 371 (step S4 in FIG. 3).

  FIG. 5 is a flowchart showing a flow of a series of processes for generating the 3D-LUT 371.

  Here, the description of the flowchart of FIG. 3 is interrupted, and a series of processes for generating the 3D-LUT 371 will be described according to the flowchart of FIG.

  In the 3D-LUT 371, first, a plurality of patch color data Daf and a plurality of target color data Tf subjected to color correction processing using the linear matrix 331 represent brightness (Y) whose coordinate axis is luminance linear, and around the coordinate axis. The angle represents the hue (H), and the distance from the coordinate axis is converted into data on the YHC color space representing the saturation (C) (step S11 in FIG. 5). The YHC color space corresponds to an example of a cylindrical coordinate color space referred to in the present invention.

  Subsequently, grid point data N (Rn, Bn, Gn) representing grid points in the RGB color space as input values in the 3D-LUT 371 is acquired (step S12 in FIG. 5), and the grid point data N (Rn, Bn, Gn) is subjected to inverse color processing conversion from the subsequent stage of the matrix correction unit 330 shown in FIG. 2 to the previous stage of the 3D-LUT correction unit 370 (step S13 in FIG. 5). In this example, since the gamma correction unit 350 exists between the matrix correction unit 330 and the 3D-LUT correction unit 370, the lattice point data N (Rn, Bn, Gn) is the inverse of the gamma correction in the gamma correction unit 350. Conversion processing is performed. The process of step S13 which performs the inverse transformation process of the color process on the grid point data N (Rn, Bn, Gn) corresponds to an example of the inverse transformation process referred to in the present invention.

  Subsequently, the grid point data N (Rn_r, Bn_r, Gn_r) that has been subjected to the inverse conversion process is converted into grid point data N (Yn, Hn, Cn) in the YHC color space (step S14 in FIG. 5).

  Next, among the plurality of patch color data Daf on the YHC color space converted at step S11, the hue “Hn” of the grid point data N (Yn, Hn, Cn) on the YHC color space converted at step S14. Two pieces of patch color data Dad are searched for (step S15 in FIG. 5). The process of step S15 for searching for two patch color data Dad sandwiching the hue of the grid point data N corresponds to an example of a selection process according to the present invention.

  FIG. 6 is a diagram showing patch color data Daf, target color data Tf, and grid point data N in the YHC color space.

  FIG. 6 shows a YHC color space in which brightness is represented by a luminance axis extending in the front-rear direction of the drawing, hue is represented by an angle around the luminance axis, and saturation is represented by a distance from the luminance axis. The patch color data Daf is indicated by a black circle, the target color data Tf is indicated by a white circle, and the lattice point data N is indicated by an asterisk.

  FIG. 6 representatively shows one grid point data N among the plurality of grid point data N. The grid point data N is sandwiched in the hue direction by the patch color data Daf_y obtained by reading the Y (yellow) color patch 501 and the patch color data Daf_r obtained by reading the R (red) color patch 501. These two patch color data Daf_y and Dad_r are searched.

  Subsequently, using the searched two patch color data Daf and their target color data Tf, an interpolation process is performed on the grid point data N (step S16 in FIG. 5). In the present embodiment, interpolation data N_H in which the brightness and saturation of the patch color data Daf are interpolated by the ratio of the hue difference between the grid point data N and each of the two patch color data Daf, and the grid point data N Difference interpolation data N_all is generated by interpolating the brightness, saturation, and hue differences between the patch color data Daf and the target color data Tf based on the ratio of the hue difference between each of the two patch color data Daf. The interpolation data N_H corresponds to an example of reference data according to the present invention, and the differential interpolation data N_all corresponds to an example of reference correction data according to the present invention. The process of generating the interpolation data N_H corresponds to an example of the reference generation process according to the present invention, and the process of generating the differential interpolation data N_all corresponds to an example of the reference correction process according to the present invention. In the example of FIG. 6, the hue differences between the grid point data N and the two patch color data Dad_y and Dad_r are θ1 and θ2, respectively, and the brightness and saturation of the patch color data Dad_y and Dad_r are expressed by the following formulas (11 ), The brightness and saturation obtained by substituting for the interpolation data N_H having the hue “Hn” of the grid point data N, the brightness values of the two patch color data Dad_y and Dad_r, and their target color data Tf_y and Tf_r, Difference interpolation data N_all obtained by substituting the difference between the saturation and the hue into Expression (11) is generated.

  Further, a linear function for correcting the color of the grid point data N is generated based on the differential interpolation data N_all and the interpolation data N_H (step S17 in FIG. 5).

  FIG. 7 is a diagram illustrating an example of the generated linear function.

  In FIG. 7, when the saturation is “0”, the correction amount is “0”, and when the saturation is the saturation value of the interpolation data N_H, the correction amount is the saturation difference value of the differential interpolation data N_all. When the saturation amount is “0” when the saturation is “0” and the saturation value is the saturation value of the interpolation data N_H, the correction amount is the brightness difference value of the differential interpolation data N_all. A linear function q_2 is shown.

  When the linear function is generated, the grid point data N is color-corrected using the linear functions q_1 and q_2 (step S18 in FIG. 5). First, as shown in FIG. 7, the saturation value of the grid point data N is acquired, and the saturation value is calculated by substituting the acquired saturation value into the upper linear function q_1. The brightness correction amount is calculated by substituting the acquired saturation value into the function q_2. The process for calculating the saturation correction amount corresponds to an example of the saturation amount calculation process according to the present invention, and the process for calculating the brightness correction amount corresponds to an example of the brightness amount calculation process according to the present invention. Further, the corrected hue amount of the differential interpolation data N_all is acquired as the hue correction amount. Subsequently, the calculated saturation correction amount, lightness correction amount, and hue correction amount are added to the saturation, lightness, and hue of the lattice point data N to generate corrected lattice point data N. The process of generating the corrected grid point data N corresponds to an example of a color correction process according to the present invention.

  When the corrected grid point data N is generated, the corrected grid point data N is converted into the corrected grid point data N in the RGB color space (step S19 in FIG. 5). Further, processing from the subsequent stage of the matrix correction unit 330 shown in FIG. 2 to the previous stage of the 3D-LUT correction unit 370 is sequentially performed on the corrected grid point data N (step S20 in FIG. 5). In the present embodiment, since the gamma correction unit 350 exists between the matrix correction unit 330 and the 3D-LUT correction unit 370, the gamma correction processing in the gamma correction unit 350 is performed on the corrected grid point data N. . The process of step S20 in which the forward conversion process is performed on the corrected grid point data N corresponds to an example of the forward conversion process according to the present invention.

  Further, the patch color data Daf and the target color data Tf are associated with each other, and the lattice point data N acquired in step S12 in FIG. 5 and the corrected lattice point data N that has been subjected to the gamma correction processing in step S20. Correspondingly, a 3D-LUT 371 is generated (step S21 in FIG. 5). The process of step S21 in which the lattice point data N and the corrected lattice point data N are associated with each other corresponds to an example of a correspondence process according to the present invention. A series of processes from acquiring this grid point data N to being associated with the corrected grid point data N is executed for all grid points (step S22 in FIG. 5: Yes).

  Through the processing as described above, the patch color data Daf is color-corrected to the target color data Tf, and the grid point data is in accordance with the linear functions q_1 and q_2 based on the difference between the patch color data Daf and the target color data Tf. A 3D-LUT 371 to be color corrected is generated. The 3D-LUT 371 is an example of the color conversion definition referred to in the present invention, and a series of processes for generating the 3D-LUT 371 shown in FIG. 5 corresponds to an example of the color conversion definition generation process referred to in the present invention. The generated 3D-LUT 371 is transmitted to the 3D-LUT correction unit 370 in FIG.

  The description of the flowchart of FIG. 5 is ended, and the description will be returned to the flowchart of FIG.

  When the optical probe 10 is attached and the 3D-LUT 371 is generated (step S4 in FIG. 5), imaging of the subject P is started (step S5 in FIG. 5).

  The light emitted from the light source 20 is irradiated from the tip of the light guide 131, and the reflected light is received by the CCD 133 to generate image data. The image data is transmitted to the image processing apparatus 30 through the signal line 132, and after black point correction processing is performed in the dark correction unit 310, white balance correction processing is performed in the AWB correction unit 320, and the matrix correction unit 330 is processed. Communicated.

  The matrix correction unit 330 uses the linear matrix 331 to convert the image data into spectral data and performs linear conversion of the color space on the image data. Image data that has been subjected to color correction processing in the matrix correction unit is subjected to gradation correction processing in the gamma correction unit 350 and is transmitted to the 3D-LUT correction unit 370.

  In the 3D-LUT correction unit 370, color correction processing is performed using the 3D-LUT 371 (step S6 in FIG. 3). In the present embodiment, after the color space of the image data is linearly converted in the matrix correction unit 330, the color of the image is individually corrected by the linear color conversion locally in the 3D-LUT correction unit 370. The size of the 3D-LUT 371 can be reduced, and color correction processing can be executed at high speed and with high accuracy.

  The image data subjected to the color correction processing in the 3D-LUT correction unit 370 is subjected to synchronization processing, noise reduction processing, blood vessel extraction processing, and the like in the image processing unit 390 using a display LUT 391 prepared in advance. Then, it is sent to the display device 40 and a medical image based on the image data is displayed on the display monitor 41.

  The above processing is continued until the photographing is finished.

  Thus, according to the endoscope system 1 of the present embodiment, the color of the image represented by the image data can be accurately corrected even when the optical probe 10 is replaced, and the abnormal shadow in the image can be accurately projected. Can do.

  Above, description of 1st Embodiment of this invention is complete | finished and 2nd Embodiment of this invention is described. The second embodiment of the present invention has substantially the same configuration as the endoscope system 1 of the first embodiment shown in FIG. For this reason, the same elements as those in the first embodiment are denoted by the same reference numerals, description thereof is omitted, and only differences from the first embodiment will be described.

  FIG. 8 is a schematic functional block diagram of the endoscope system 1 ′ in the present embodiment.

  The endoscope system 1 'according to the present embodiment has substantially the same configuration as the endoscope system 1 according to the first embodiment shown in FIG. 2, but the endoscope system 1' according to the present embodiment is 3D. The difference from the first embodiment is that the -LUT 371 and the display LUT 391 are combined.

  In the endoscope system 1 ′ of the present embodiment, when the 3D-LUT 371 shown in FIG. 2 is generated in the 3D-LUT 380, the 3D-LUT 371 is combined with the display LUT 391. The combined LUT 372 is transmitted to the 3D-LUT correction unit 370 ′.

  The 3D-LUT correction unit 370 ′ performs color processing using the synthesis LUT 372 on the image data obtained by the optical probe 10, thereby correcting the color of the image, and the image processing unit of FIG. The process of converting to image data for the display monitor 41 performed in 390 is performed at the same time.

  As described above, when a display LUT or the like is also present in the subsequent stage of the 3D-LUT, the storage capacity for storing the lookup table can be reduced by generating a combined LUT by combining them. , Processing can be speeded up.

  Here, the example in which the linear matrix 331 and the 3D-LUT are generated each time the optical probe is mounted has been described. For example, the identification number for identifying the optical probe is associated with the linear matrix 331 and the 3D-LUT. The identification number may be detected by attaching the optical probe, and the linear matrix 331 and the 3D-LUT associated with the identification number may be used.

1 is a schematic configuration diagram of an endoscope system according to a first embodiment of the present invention. It is a schematic functional block diagram of an endoscope system. It is a flowchart figure which shows the flow of a series of processes until an optical probe is mounted | worn with an image processing apparatus and a light source device, the test object P is image | photographed, and the image | photographed medical image is displayed on a display screen. It is a figure which shows an example of a chart image. It is a flowchart figure which shows the flow of a series of processes which produce | generate 3D-LUT. FIG. 5 is a diagram showing patch color data Daf, target color data Tf, and grid point data N in the YHC color space. It is a figure which shows an example of the produced | generated primary function. It is a schematic functional block diagram of the endoscope system in a 2nd embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,1 'Endoscope system 10 Optical probe 20 Light source device 30 Image processing apparatus 40 Display apparatus 11 Probe part 12 Operation part 13 Light / signal guide 121 Bending operation lever 122 Shooting button 123 Selection button 131 Light guide 132 Signal line 133 CCD
140 Color Filter 150 A / D Conversion Unit 160 Imaging Control Unit 300 Storage Unit 310 Dark Correction Unit 320 AWB Correction Unit 330 Matrix Correction Unit 331 Linear Matrix 340 Matrix Generation Unit 350 Gamma Correction Unit 360 Overall Control Unit 370 3D-LUT Correction Unit 371 3D-LUT
380 3D-LUT generation unit 390 image processing unit 391 processing LUT

Claims (7)

Each of the plurality of color patches obtained by reading each of the plurality of color patches having different colors by an input device that reads the image and generates image data representing the image is expressed in coordinates in a predetermined color space. A patch color data acquisition process for acquiring a plurality of expressed patch color data;
Based on the difference between the plurality of patch color data and a plurality of target color data expressing the target color for each of the plurality of color patches by coordinates in the color space, the plurality of patch color data is converted into the plurality of target color data. A matrix generation process for generating a correction matrix representing a linear transformation of the color space to be approximated to color data;
The plurality of patch color data converted from the plurality of patch color data by the correction matrix is converted into the plurality of target color data, and for each color data representing each coordinate in the color space, Of a plurality of differences between the plurality of conversion patch color data and the plurality of target color data, color correction of a correction amount obtained by linear interpolation based on a part of the difference corresponding to the local region including the coordinates is performed. A color conversion definition generation process for generating a color conversion definition that defines the color conversion to be performed ,
In the color conversion definition generation process, color data in which a color is expressed by coordinates in the color space is represented by brightness, hue, and saturation with a coordinate axis, an angle around the coordinate axis, and a distance from the coordinate axis. A color conversion definition creating method, characterized in that color data expressed in coordinates in a cylindrical coordinate color space is converted into color data and the interpolation is performed in the cylindrical coordinate color space . 2. The color conversion definition creation method according to claim 1, wherein the cylindrical coordinate color space is a color space in which brightness and saturation are linear with respect to the luminance of the color.   The color conversion definition generation process includes:
  For the color data to be interpolated, a selection process for selecting two conversion patch color data sandwiching the hue of the color data to be interpolated from among a plurality of conversion patch color data,
  Reference data is obtained by linearly interpolating the two conversion patch color data selected in the selection process with respect to each of lightness and saturation with a hue ratio between the two conversion patch color data and the color data to be interpolated. A reference generation process for generating
  The two conversion patch color data and the two target color data corresponding to the two conversion patch color data in a hue ratio between the two conversion patch color data selected in the selection process and the color data to be interpolated A reference correction process for generating reference correction data representing a correction amount with respect to the reference data by linearly interpolating each difference with
  When the saturation is 0, the saturation correction amount is 0, and when the saturation is the same as the saturation of the reference data, the saturation correction amount is equal to the saturation correction amount represented by the reference correction data 1 A saturation amount calculation process for calculating a saturation correction amount by applying the saturation of the color data to be interpolated to a next function;
  When the saturation is 0, the lightness correction amount is 0, and when the saturation is the same as the saturation of the reference data, the lightness correction amount is equal to the lightness correction amount represented by the reference correction data. The color conversion definition creating method according to claim 1, further comprising a lightness amount calculation step of calculating a lightness correction amount by applying a saturation of the color data to be interpolated.   4. The color conversion definition according to claim 1, wherein the color conversion definition is a lookup table in which color data before the color conversion and color data after the color conversion are associated with each other. Color conversion definition creation method.   The correction matrix and the color conversion definition are used for a series of color conversion processes in which another color process is performed between color correction using the correction matrix and color conversion according to the color conversion definition. Yes,
  The color conversion definition generation process includes:
  An inverse conversion process of performing an inverse conversion process of the color processing on the grid point data representing the grid points of the color space;
  A color correction step of calculating the correction amount for the lattice point data subjected to the inverse transformation process by linear interpolation, and performing color correction of the correction amount on the lattice point data to generate corrected lattice point data;
  A forward conversion process for performing color processing between the correction grid point data and the color conversion;
  5. The method according to claim 1, further comprising: a correspondence process of associating the corrected grid point data subjected to the color processing in the forward conversion process with the grid point data before the color conversion process. Color conversion definition creation method.   Each of the plurality of color patches obtained by reading each of the plurality of color patches having different colors by an input device that reads the image and generates image data representing the image is expressed in coordinates in a predetermined color space. A patch color data acquisition unit for acquiring a plurality of expressed patch color data;
  Based on the difference between the plurality of patch color data and a plurality of target color data expressing the target color for each of the plurality of color patches by coordinates in the color space, the plurality of patch color data is converted into the plurality of target color data. A matrix generation unit that generates a correction matrix that represents linear transformation of the color space, and approximates color data;
  The plurality of patch color data converted from the plurality of patch color data by the correction matrix is converted into the plurality of target color data, and for each color data representing each coordinate in the color space, Of a plurality of differences between the plurality of conversion patch color data and the plurality of target color data, color correction of a correction amount obtained by linear interpolation based on a part of the difference corresponding to the local region including the coordinates is performed. A color conversion definition generation unit that generates a color conversion definition that defines the color conversion to be performed,
  The color conversion definition generation unit represents color data in which a color is expressed by coordinates in the color space, and represents brightness, hue, and saturation with a coordinate axis, an angle around the coordinate axis, and a distance from the coordinate axis. An apparatus for creating a color conversion definition, which converts color data expressed in coordinates in a cylindrical coordinate color space and performs the interpolation in the cylindrical coordinate color space.   An optical probe comprising a light source that emits light and the input device;
  A color conversion definition creation device according to claim 6;
  A color conversion device that performs color conversion processing on image data obtained by the input device using the correction matrix and color conversion definition generated by the color conversion definition creation device;
  An endoscope system comprising: a display device that displays an image represented by image data subjected to color conversion processing by the color conversion device.

Images