JP2009077950A - Color conversion calculating method, color conversion calculating device, color converter and endoscope system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color conversion calculating method, a color conversion calculating device, a color converter and an endoscope system, performing color conversion processing with high accuracy to the color of an image even when an image before color conversion processing or in the process of color conversion cannot be obtained. <P>SOLUTION: This color conversion calculating method includes: an image obtaining process of obtaining a color patch image output by photographing a color patch using an imaging unit adapted to photograph an object to obtain an object image and an image processor for performing image processing for the object image and outputting the same; an image estimating process of performing inverse transformation processing to image processing to the color patch image obtained in the image obtaining process to thereby estimate the color patch image before image processing is performed; and a color conversion calculating process of calculating color conversion for converting the color of the color patch image estimated by the image estimating process to an original color of the color patch. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a color conversion calculation method for calculating a color conversion for converting the color of an image, a color conversion calculation device, a color conversion device that performs color conversion processing on an image according to the calculated color conversion, and an endoscope system.

  Conventionally, in the medical field, an endoscope in which an elongate tube having an optical member or an image sensor attached at the tip is inserted into the body of a subject, and the inside of the subject is photographed to observe a tumor, a thrombus, etc. Mirror systems are widely used. By directly photographing the body of the subject, it is possible to grasp the color and shape of the lesion, which is difficult to understand with radiographic images, without damaging the subject externally, and it is necessary to determine the treatment policy etc. Easy information.

  The endoscope system includes a light source that emits light, an optical probe that is inserted into the body of the subject, an image processing device that performs various image processing on an image obtained by imaging the inside of the subject, and a display that displays the image Generally, a device or the like is provided. This optical probe has a light guide that guides light emitted from a light source, an optical member that collects light guided by the light guide into the body of the subject, and light irradiated by the optical member is reflected by the subject. An imaging device that receives reflected light and generates an image signal is mounted inside, and at the time of imaging, it may be replaced with an optical probe that matches the position or shape of the imaging location (for example, stomach or intestine). It is common. However, if the optical probe equipped with an image sensor is replaced, the color sensitivity characteristics of the entire endoscope system will change, so even if the same part of the same subject is imaged, the color image obtained will differ. End up. Even when the same type of optical probe is mounted, the color of the color image is often slightly different due to the difference between the image sensor and the optical member. Endoscopic systems are required to observe subtle irregularities and discoloration of the stomach wall and the like, and if the color of the image changes due to the replacement of the optical probe, there is a risk that correct diagnosis cannot be performed. For this reason, it is necessary to perform a color conversion process on the image so that the color of the image approaches a target color that can diagnose the state of the observation location.

  The image processing apparatus mounted on the endoscope system has a gain correction unit that corrects the gain of the image, a gamma correction unit that corrects the gradation, and the like, like a normal color copier that inputs an image, It is common to have multiple color conversion units, such as a white balance correction unit that corrects white spots, a shading correction unit that corrects density unevenness, and a color correction unit that matches the color of an image with your favorite color. It is. In a color copier or the like, color data obtained by reading a patch image is sequentially subjected to color conversion processing in each of a plurality of color conversion units, and color data is extracted each time the color conversion processing is performed. Generally, color conversion parameters used for each color conversion process are adjusted by comparing with target color data. However, since endoscope systems are required to display moving images on the screen in real time, a plurality of color conversion units are built on a dedicated circuit to speed up processing, and images can be displayed. In many cases, it cannot be taken out in the middle of processing. For this reason, there is a problem that the color conversion parameters of the endoscope system cannot be adjusted by the same method as a color copying machine that performs color conversion processing on a still image.

With respect to this point, Patent Document 1 is obtained by sequentially capturing a white balance image for correcting white balance, a gray scale image for correcting gray scale, and the like using an endoscope system. A technique for adjusting a color conversion parameter using target color data and target color data is described. Patent Document 2 captures a patch image with an optical probe to generate color data. The color data is directly acquired from the light source and the light source and the electronic shutter are adjusted, and then the color data is sent to the display device via the image processing device to display the patch image on the display screen. Further, a technique for adjusting a display screen and color conversion parameters using color data obtained by further capturing a patch image of the above with a light probe is described.
JP-A-5-137893 Japanese Patent Laid-Open No. 8-152656

  However, since an optical probe is usually connected to an image processing apparatus as a set with a light source and is integrally controlled by a CPU or the like in the image processing apparatus, in a general endoscope system, an image captured by an optical probe is used. Often images cannot be retrieved directly. For this reason, in order to apply the technique described in Patent Document 2, there is a problem that a special mechanism must be provided in the optical probe and the image processing apparatus. In addition, in the techniques described in Patent Document 1 and Patent Document 2, color conversion parameters that can be adjusted are limited to white balance, gray scale, and the like, and therefore, a plurality of color conversion parameters must be finely adjusted individually. There is also a problem that the color of the image cannot be corrected sufficiently.

  Such a problem is not limited to the endoscope apparatus alone, but is a problem that occurs in the general field of color conversion processing apparatuses that perform color conversion processing on images.

  In view of the above circumstances, the present invention is a color conversion calculation capable of performing high-accuracy color conversion processing on the colors of an image even when an image before color conversion processing or in the middle of color conversion processing cannot be acquired. It is an object to provide a method, a color conversion calculation device, a color conversion device, and an endoscope system.

According to the color conversion calculation method of the present invention that achieves the above object, a color patch is captured and output by an imaging device that captures a subject and obtains a subject image and an image processing device that performs image processing on the subject image and outputs the processed subject image. An image acquisition process for acquiring a color patch image;
An image estimation process for estimating the color patch image before the image processing is performed by performing an inverse transformation process of the image processing on the color patch image acquired in the image acquisition process;
And a color conversion calculation process for calculating a color conversion for converting the color of the color patch image estimated by the image estimation process into the original color of the color patch.

  According to the color conversion calculation method of the present invention, the color patch image output from the image processing apparatus is subjected to the inverse conversion process of the image processing to estimate the color patch image before the image processing, and the estimated color patch image A color conversion is calculated such that the color is converted to the original color. Therefore, even in an endoscope system or the like in which the imaging device and the image processing apparatus are integrally connected and the subject image before the image processing is not performed can be converted with high accuracy. it can.

  In the color conversion calculation method of the present invention, it is preferable that the color conversion calculation process is a process of creating a color conversion definition that defines color conversion.

  By creating the color conversion definition, it is possible to easily calculate the color conversion for converting the color of the image.

In the color conversion calculation method of the present invention, the image processing apparatus is configured to set a processing parameter used for image processing, and execute the image processing using the set processing parameter.
Prior to the image acquisition process, it is preferable to have a parameter setting process for setting a processing parameter to be non-converted for a processing part that can be non-converted during image processing as a processing parameter for the image processing apparatus.

  In the process of image processing executed by the image processing apparatus, data bit loss occurs, so the color of the color patch image estimated in the image estimation process is completely different from the original color patch color. Often does not match. By setting the processing parameters to be converted as little as possible before the image acquisition process, it is possible to reduce the influence of bit dropping or the like and improve the color conversion accuracy.

Also, in the color conversion calculation method of the present invention, the image acquisition process acquires a plurality of color patch images obtained by photographing a plurality of color patches having different color systems from each other by an imaging device and an image processing apparatus. Is the process of
The image estimation process is a process of estimating the color of the color patch image before image processing is performed for each of the plurality of color patch images.
In the color conversion calculation process, as the color conversion, a correction conversion that corrects the distortion of the color of the subject image caused by the imaging device is performed, and the color estimated for each of the plurality of color patch images in the image estimation process and each of the plurality of color patches Preferably, the calculation process is based on the original color.

  By obtaining a plurality of color patch images obtained by photographing a plurality of color patches having different color systems, it is possible to calculate color conversion that accurately corrects the color distortion caused by the imaging device.

Also, in the color conversion calculation method of the present invention, the image acquisition process acquires a plurality of color patch images obtained by photographing a plurality of color patches having the same color and different densities, respectively, by the imaging device and the image processing apparatus. Is the process of
The image estimation process is a process of estimating the density of the color patch image before image processing is performed for each of the plurality of color patch images.
In the color conversion calculation process, as the color conversion, the gradation correction conversion for correcting the distortion of the gradation of the subject image caused by the image pickup device is performed, and the density and the plurality of colors estimated for each of the plurality of color patch images in the image estimation process. It is also preferable that the calculation is based on the original density of each patch.

  By obtaining a plurality of color patch images obtained by photographing a plurality of color patches having similar colors and different densities, it is possible to calculate color conversion that accurately corrects distortion of the gradation of the image.

In the color conversion creation method according to the present invention, the image acquisition process includes a plurality of first colors obtained by photographing and outputting each of a plurality of first-type color patches having the same color and different densities. A first acquisition process of acquiring a seed color patch image, and a plurality of second type color patch images obtained by capturing and outputting a plurality of second type color patches having different color systems from each other. 2 acquisition process,
An image estimation process includes a first estimation process for estimating a density of a first type color patch image before image processing is performed on each of a plurality of first type color patch images, and a plurality of second type color patch images. A second estimation step of estimating the color of the second type color patch image before image processing is performed,
In the color conversion calculation process, the gradation correction conversion for correcting the distortion of the gradation of the subject image caused by the image pickup device is performed. The density estimated for each of the plurality of first-type color patch images in the first estimation process and the plurality of first colors are calculated. A gradation correction calculation process that is calculated based on the original density of each of the one type of color patch and a correction conversion that corrects the distortion of the color of the subject image caused by the image pickup device are performed in the second estimation process. A process including a color correction calculation process for calculating based on a color obtained by performing gradation correction conversion on the color estimated for each seed color patch image and an original color of each of the plurality of second kind color patches. Further preferred.

  First, the gradation correction conversion is calculated based on the first type color patch image, and then the correction conversion is calculated based on the second type color patch image after the gradation correction conversion is performed. It is possible to calculate color conversion that brings the color of the image closer to the original subject color with higher accuracy.

Further, in the color conversion calculation method of the present invention, the image acquisition process includes one or more color patches such that the image capturing process and the image processing apparatus have similar and the same density of colors at a plurality of locations within the shooting angle of view. Is a process of acquiring one or more color patch images captured and output,
The image estimation process is a process of estimating the density of a color patch image before image processing is performed for each of a plurality of locations,
In the color conversion calculation process, as the color conversion, the shading correction conversion for correcting the sensitivity difference of the image pickup device within the shooting angle of view is performed. It is preferable that the process is based on the calculation.

  Here, one color patch spreading over the entire area within the shooting angle of view may be taken at a time to obtain one color patch image, and a plurality of color patches arranged at each location within the shooting angle of view may be obtained. A plurality of color patch images may be obtained by photographing each time.

  According to this preferred color conversion calculation method, it is possible to calculate color conversion that accurately corrects the sensitivity difference of the image pickup device within the shooting angle of view.

In the color conversion creation method according to the present invention, the image acquisition process includes a plurality of first colors obtained by photographing and outputting each of a plurality of first-type color patches having the same color and different densities. A first acquisition process of acquiring a seed color patch image, and a second acquisition of a plurality of second type color patch images that are captured and output by the imaging device, each of a plurality of second type color patches having different color systems. The acquisition process and one or more third-type colors that are captured and output by one or more third-type color patches so that there are colors of the same system and the same density at a plurality of locations within the shooting angle of view by the imaging device A third acquisition step of acquiring a patch image,
An image estimation process includes a first estimation process for estimating a density of a first type color patch image before image processing is performed on each of a plurality of first type color patch images, and a plurality of second type color patch images. The second estimation process for estimating the color of the second type color patch image before the image processing is performed, and the density of the third type color patch image before the image processing is estimated for each of a plurality of locations. And a third estimation process to
In the color conversion calculation process, the gradation correction conversion for correcting the distortion of the gradation of the subject image caused by the image pickup device is performed. The density estimated for each of the plurality of first type color patch images in the first estimation process and the plurality of A gradation correction calculation process that is calculated based on the original density of each type of color patch and a shading correction conversion that corrects the sensitivity difference of the imaging device within the shooting angle of view are estimated for each of a plurality of locations in the third estimation process. Shading correction calculation process based on the density obtained by performing gradation correction conversion on the obtained density and the original density of the third type color patch, and correction conversion for correcting the color distortion of the subject image caused by the image pickup device For the colors estimated for each of the plurality of second-type color patch images in the second estimation process and the original colors of the plurality of second-type color patches respectively. It is further preferable that a process and a color correction calculation step of calculating, based on and.

  First, tone correction conversion is calculated based on the first type color patch image, and shading correction conversion is calculated based on the third type color patch image on which the tone correction conversion has been performed. By calculating the correction conversion based on the second-type color patch image subjected to the shading correction conversion, the color of the image can be converted with higher accuracy.

In addition, the color conversion calculation apparatus of the present invention that achieves the above-described object has a color patch photographed by an imaging device that captures a subject to obtain a subject image and an image processing device that performs image processing on the subject image and outputs the subject image. An image acquisition unit for acquiring the output color patch image;
An image estimation unit that performs an inverse transformation process of the image processing on the color patch image acquired by the image acquisition unit and estimates a color patch image before the image processing is performed;
And a color conversion calculation unit that calculates a color conversion for converting the color of the color patch image estimated by the image estimation unit into the original color of the color patch.

  According to the color conversion calculation apparatus of the present invention, it is possible to calculate color conversion that accurately converts the color of an image even in an endoscope system or the like in which an image taken by an imaging device cannot be directly extracted before image processing. it can.

Further, in the color conversion device of the present invention, a color patch image obtained by photographing a color patch and output by an imaging device that captures a subject and obtains a subject image and an image processing device that performs image processing on the subject image and outputs the subject image An image acquisition unit for acquiring
An image estimation unit that performs an inverse transformation process of the image processing on the color patch image acquired by the image acquisition unit and estimates a color patch image before the image processing is performed;
A color conversion calculation unit for calculating a color conversion for converting the color of the color patch image estimated by the image estimation unit into the original color of the color patch;
And a color conversion unit that performs color conversion processing on the subject image in accordance with the color conversion calculated by the color conversion calculation unit.

  According to the color conversion device of the present invention, the color of an image can be accurately converted to the original subject color.

  Here, only the basic form of the color conversion device is shown here, but this is only for avoiding duplication, and the color conversion device according to the present invention is not limited to the basic form described above. Various forms corresponding to each form of the color conversion method are included.

In addition, the endoscope system of the present invention that achieves the above object includes a light source that emits light;
An optical probe including a light guide path for guiding light emitted from a light source to a subject and an imaging device for photographing the subject to obtain a subject image;
An image processing device that performs image processing on a subject image obtained by an imaging device and outputs the image; and an image acquisition unit that acquires a color patch image obtained by photographing a color patch by the imaging device and the image processing device;
An image estimation unit that performs an inverse transformation process of the image processing on the color patch image acquired by the image acquisition unit and estimates a color patch image before the image processing is performed;
A color conversion calculation device including a color conversion calculation unit that calculates a color conversion for converting the color of the color patch image estimated by the image estimation unit into the original color of the color patch;
The image processing apparatus is characterized in that the subject image is subjected to color conversion processing according to the color conversion calculated by the color conversion calculation device as at least a part of the image processing and is output.

  According to the endoscope system of the present invention, the color of an image can be converted with high accuracy, and a medical image useful for diagnosis can be generated.

  Here, the endoscope system is only shown in its basic form here, but this is merely for avoiding duplication, and the endoscope system referred to in the present invention has only the above basic form. Instead, various forms corresponding to the respective forms of the color conversion method described above are included.

  According to the present invention, even when an image before the color conversion process or during the color conversion process cannot be acquired, a high-precision color conversion process can be performed on the color of the image.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of an endoscope system to which an embodiment of the present invention is applied.

  An endoscope system 1 shown in FIG. 1 guides and irradiates light into the body of a subject P, generates an image signal based on reflected light from the subject P, and a light source device that emits light. 20, an image processing device 30 that performs predetermined image processing on an image signal obtained by the optical probe 10 to generate a medical image obtained by photographing the inside of the subject P, and a medical image generated by the image processing device 30 Is displayed on the display monitor 41. In the endoscope system 1, the optical probe 10 is detachably attached to the light source 20 and the image processing device 30. The optical probe 10 is an example of an optical probe according to the present invention, and the image processing apparatus 30 corresponds to an example of an image processing apparatus according to the present invention and also corresponds to an example of a color conversion apparatus according to the present invention.

  The optical probe 10 includes a flexible elongated probe unit 11, an operation unit 12 that operates the probe unit 11, and a light / signal guide 13 that connects the light source device 20, the image processing device 30, and the optical probe 10. It is configured. Hereinafter, the side of the optical probe 10 that is inserted into the body of the subject P is referred to as the front end, and the opposite side of the front end is referred to as the rear end.

  The operation unit 12 includes a bending operation lever 121 for bending the probe unit 11, a shooting button 122 for shooting a still image, a color adjustment button 123 for adjusting the color of the displayed image, and the like. Is provided.

  The light / signal guide 13 includes a light guide 131 that transmits light and a signal line 132 that transmits signals. The light guide 131 has a rear end connected to the light source device 20, guides the light emitted from the light source device 20 to the probe unit 11, and guides the light from the irradiation window 11 a provided at the tip of the probe unit 11. Irradiate toward P. The light guide 131 corresponds to an example of a light guide path according to the present invention. The signal line 132 has a CCD 133 attached to the front end, and the rear end side is connected to the image processing apparatus 30. The reflected light reflected from the inside of the subject P by the light irradiated from the irradiation window 11a of the light guide 131 is collected by the optical member 134 provided at the tip of the probe unit 11, received by the CCD 133, and reflected light. A photographed image representing is generated. The CCD 133 corresponds to an example of an imaging device according to the present invention. The CCD 133 has a plurality of light receiving portions arranged side by side, and light is received by each of the plurality of light receiving portions, whereby image data in which an image is expressed by a plurality of pixels is generated. In the present embodiment, a color filter (see FIG. 2) in which R, G, and B colors are arranged in a regular color pattern is attached to the CCD 133 at positions corresponding to the plurality of light receiving units. When the light that has passed through is received by the CCD 133, a color mosaic image in which pixels of R, G, and B colors are arranged in the same color pattern as the color pattern of the color filter is generated.

  The generated color mosaic image is transmitted to the image processing device 30 through the signal line 132, and the image processing device 30 performs predetermined image processing.

  FIG. 2 is a schematic functional block diagram of the endoscope system 1.

  In FIG. 2, the display monitor 41 and the operation unit 12 of the optical probe 10 are not shown, and only main elements related to the generation of the image signal are shown.

  The light source device 20 also shown in FIG. 1 emits white light and is controlled by the overall control unit 360 of the image processing device 30.

  In addition to the CCD 133 also shown in FIG. 1, the optical probe 10 digitally receives analog image signals generated by the color filter 140 in which each color of R, G, B is arranged in a mosaic pattern with a regular color pattern, and the CCD 133. Are provided with an A / D conversion unit 150 that converts the image signal into an image signal, an imaging control unit 160 that controls processing in various elements in the optical probe 10, and the like.

  In the endoscope apparatus 1, in order to display a moving image on the display monitor 41 in real time, various elements constituting the image processing apparatus 30 are mounted on the same circuit board, and the processing speed of the image processing apparatus 30 is increased. Is planned. In the image processing apparatus 30, linearity correction processing for correcting gradation distortion on an image sent from the optical probe 10 through the signal line 132, and unevenness in light receiving sensitivity within the photographing field angle of the optical probe 10 are performed. The preprocessing unit 300 that performs shading correction processing to be corrected, the color correction unit 31 that corrects the color of the image that has been preprocessed by the preprocessing unit 300, and the overall processing of the optical probe 10 and the image processing apparatus 30 are controlled. And various parameters (linearity parameter 301, shading parameter 302, gain parameter 311, spectral correction matrix 321, gamma parameter 341, synchronization parameter 351, YCC conversion parameter 361, sharpness used by the image processing apparatus 30. Parameter 371A, low-pass parameter 371 A parameter calculation unit 330 that calculates the display LUT 381) and a storage unit 400 that stores a patch image used for parameter calculation, and the color correction unit 31 corrects the gain of the image. Each pixel of the color mosaic image generated by the gain correction unit 310, the spectral correction unit 330 that corrects the spectral characteristics of the optical probe 10, the gamma correction unit 350 that performs gradation correction processing on the image, and the optical probe 10 is provided. Each pixel is expressed as a mixed color of R, G, and B colors by interpolating other color components (for example, B and G colors) except the color component (for example, R color) using surrounding pixels. A synchronization processing unit 350 that generates a color image, a YCC conversion unit 360 that decomposes the image into a luminance component Y and color difference components Cr and Cb, and a sharpness processing that performs sharpness processing on the luminance component Y. The display device 40 displays a YCC image including the luminance component Y and the color difference components Cr and Cb, a low-pass processing unit 370B that reduces the false color by removing the spatial high frequency components in the color difference components Cr and Cb The display adjustment unit 380 converts the display image into a display image that can be displayed on the monitor 41 and transmits the display image to the display device 40 and the parameter calculation unit 330. The display adjustment unit 380 is an example of an image acquisition unit according to the present invention, the color correction unit 31 is an example of an image estimation unit according to the present invention, and the parameter calculation unit 330 is an example of a color conversion calculation unit according to the present invention. It corresponds to.

  An image generated by the optical probe 10 is transmitted to the image processing apparatus 30 through the signal line 132. In the image processing apparatus 30, the linearity correction process using the linearity parameter 301 is performed in the pre-processing unit 300, whereby the linearity of the color density of the image is corrected, and the shading correction process using the shading parameter 302 is performed. As a result, the unevenness of the color density of the image is corrected. The image that has been preprocessed by the preprocessing unit 300 is transmitted to the color correction unit 31. The image transmitted to the color correction unit 31 is subjected to spectral correction processing using the spectral correction matrix 321 in the spectral correction unit 320, whereby the spectral characteristics of the optical probe 10 are corrected, and the gamma correction unit 340 corrects the gamma parameter. By executing gamma correction processing using 341, the color gradation is adjusted, and by performing synchronization processing using the synchronization parameter 351 in the synchronization processing unit 350, each pixel has R, A color mosaic image having one color component of G and B is converted into a color image in which each pixel is expressed by R, G and B color components. Further, the color conversion process using the YCC conversion parameter 361 is executed in the YCC conversion unit 360, whereby the image is color-separated into a luminance component and a color difference component, and the sharpness processing unit 370A performs the sharpness parameter on the luminance component. By performing the sharpness processing using 371A, the sharpness of the image is adjusted, and the low-pass filter processing using the low-pass parameter 371B is performed by the low-pass processing unit 370B on the color difference component, so that False color is removed. The luminance component subjected to the sharpness processing and the color difference component subjected to the low-pass filter processing are transmitted to the display adjustment unit 380 and synthesized, and display correction processing using the display LUT 381 is performed on the image. A display image for the display monitor 41 is generated.

  The generated display image is sent to the display device 40, and a medical image based on the display image is displayed on the display monitor 41.

  Here, in the endoscope apparatus 1 of the present embodiment, when the optical probe 10 is attached to the light source device 20 and the image processing device 30, color conversion parameters (linearity parameter 301, shading) that match the characteristics of the optical probe 10 are used. Parameter 302, gain parameter 311, spectral correction matrix 321, gamma parameter 341, synchronization parameter 351, YCC conversion parameter 361, sharpness parameter 371A, low-pass parameter 371B, and display LUT 381) are calculated.

  FIG. 3 is a flowchart showing a flow of a series of processes for calculating the color conversion parameter.

  When the optical probe 10 is attached to the image processing apparatus 30 and the light source apparatus 20, the parameter calculation unit 330 displays the color conversion parameters (linearity parameter 301, shading parameter 302, gain parameter 311, spectral correction matrix 321, gamma) shown in FIG. Adjustment parameters prepared in advance are set in the parameter 341, the synchronization parameter 351, the YCC conversion parameter 361, the sharpness parameter 371A, the low-pass parameter 371B, and the display LUT 381) (step S11 in FIG. 3). This adjustment parameter is a non-conversion parameter for preventing the image from being converted as much as possible. For example, the linearity parameter 301 that defines a gradation curve with a slope of “1” or the density of all pixels on the image is set to 1. A shading parameter 302 to be multiplied, a gain parameter 311 with a gain amount “1”, a spectral correction matrix 321 with a diagonal term “1” and a non-diagonal term “0” are applied. For the synchronization parameter 351 and the like for which it is difficult to make the process non-converted, a parameter prepared in advance is used as it is.

  Subsequently, the gray probe chart in which a plurality of gray patches having different color densities are arranged is read by the optical probe 10, and a plurality of gray patch images are generated (step S12 in FIG. 3). In this embodiment, a gray step chart is attached to a sealed box from which the entry of outside light is blocked, and the gray step chart is inserted with the optical probe 10 inserted through an insertion hole provided in the sealed box. Taken. A plurality of gray patches having different color densities are examples of the first type color patch referred to in the present invention, and the process of generating a plurality of gray patch images corresponds to an example of a first acquisition process referred to in the present invention. .

  FIG. 4 is a diagram illustrating an example of the gray step chart 500.

  In the gray step chart 500 shown in FIG. 4, eight gray patches 501 whose color levels change linearly are arranged close to the central portion of the region Q within the photographing field angle of the optical probe 10. Since there is little color density unevenness of the image in the central portion of the photographing region Q, color conversion with high color correction accuracy is achieved by using the gray step chart 500 in which the gray patch 501 is arranged near the central portion of the photographing region Q. Parameters can be calculated.

  Eight gray patch images obtained by reading the eight gray patches 501 on the gray step chart 500 are transmitted from the optical probe 10 to the image processing apparatus 30. In the image processing apparatus 30, a plurality of image processes are sequentially performed on the gray patch image using the adjustment parameters. The processed gray patch image that has been subjected to image processing using the display LUT 381 in the display adjustment unit 380 at the final stage is transmitted to the parameter calculation unit 330 before being transmitted to the display device 40.

  In the parameter calculation unit 330, the processed gray patch image is subjected to inverse conversion processing of each of the plurality of image processes using the adjustment parameter, and an approximate gray patch image that approximates the gray patch image input to the image processing device 30 is obtained. It is generated (step S13 in FIG. 3). The process for generating the approximate gray patch image corresponds to an example of a first estimation process according to the present invention.

  Subsequently, for each of a plurality of approximate gray patch images generated based on a plurality of gray patch images (eight in the example of FIG. 4), the color levels of a plurality of pixels constituting each approximate gray patch image Are averaged, and the averaged color level is determined as the color level of each approximate gray patch image (step S14 in FIG. 3).

  When the color levels of each of the plurality of approximate gray patch images are determined, the linearity parameter 301 for correcting the color levels so as to change linearly is calculated (step S15 in FIG. 3). The process of calculating the linearity parameter 301 corresponds to an example of a gradation correction calculation process according to the present invention.

  Subsequently, instead of the gray step chart 500 shown in FIG. 4, a gray solid chart having a uniform gray is prepared, and the gray solid chart is read by the optical probe 10 to generate a gray solid image (step of FIG. 3). S16). The gray solid chart is an example of the third type color patch according to the present invention, and the process of generating the gray solid image corresponds to an example of the third acquisition process according to the present invention.

  FIG. 5 is a diagram illustrating an example of the gray solid chart 600.

  Since the gray solid chart 600 has a uniform gray, it is ideal that the gray solid images obtained by reading the gray solid chart 600 have the same color density level for each of a plurality of pixels. However, the gray solid image is generally darker in the peripheral part and brighter in the central part due to variations in light receiving sensitivity of each of the plurality of light receiving parts constituting the CCD 133 and unevenness of light emitted from the light source 20. is there.

  The gray solid image obtained by reading the gray solid chart 600 is transmitted from the optical probe 10 to the image processing apparatus 30, and the image processing apparatus 30 sequentially performs a plurality of image processing using adjustment parameters on the gray solid image. Is done. The post-processing gray solid image subjected to the image processing is transmitted to the parameter calculation unit 330.

  In the parameter calculation unit 330, the processed gray solid image is subjected to inverse conversion processing for each of the plurality of image processes using the adjustment parameter, and an approximate gray solid image that approximates the gray solid image input to the image processing device 30 is generated. (Step S17 in FIG. 3). The process of generating the approximate gray solid image corresponds to an example of a third estimation process according to the present invention.

  Further, the parameter calculation unit 330 performs linearity correction processing on the approximate gray solid image using the linearity parameter 301, and calculates a shading parameter 302 for reducing color density unevenness of the image based on the corrected approximate gray solid image. (Step S19 in FIG. 3). The process of calculating the shading parameter 302 corresponds to an example of a shading correction calculation process according to the present invention.

  When the linearity parameter 301 and the shading parameter 302 are calculated, a color chart in which a plurality of color patches having different colors are arranged is read instead of the gray solid chart 600 shown in FIG. Is generated (step S20 in FIG. 3). In the present embodiment, a color patch is arranged close to the center of the region Q within the shooting angle of view of the optical probe 10 as in the gray step chart 500 shown in FIG. 4, and C (cyan), A color chart in which color patches of M (magenta), Y (yellow), R (red), G (green), B (blue), white, and skin color are arranged is read. Each color patch constituting the color chart corresponds to an example of a second type color patch referred to in the present invention, and the process of generating a color patch image corresponds to an example of a second acquisition process referred to in the present invention.

  A plurality of color patch images obtained by reading each of the plurality of color patches on the color chart are sequentially subjected to image processing in the image processing device 30. The processed color patch image subjected to the image processing is transmitted to the parameter calculation unit 330.

  In the parameter calculation unit 330, the processed color patch image is subjected to inverse conversion processing of each of the plurality of image processing using the adjustment parameter, and an approximate color patch image that approximates the color patch image input to the image processing device 30 is obtained. It is generated (step S21 in FIG. 3). The process of generating the approximate color patch image corresponds to an example of a second estimation process according to the present invention.

  Subsequently, for each of the plurality of approximate color patch images, the color levels of the plurality of pixels constituting each approximate color patch are averaged (step S22 in FIG. 3), and an averaged color patch image is generated. .

  The generated average color patch image is subjected to linearity correction processing using the linearity parameter 301 (step S23 in FIG. 3), and further subjected to shading correction processing using the shading parameter 302 (step in FIG. 3). S24).

  The color of the averaged color patch image that has been subjected to the pre-processing (linearity correction processing and shading correction processing) as described above is subjected to inverse conversion processing on the post-processing color patch image, and further the averaging processing, linearity correction, It is ideal that the shading correction is applied and it is equal to the color of the original color patch. However, in practice, bit dropping or the like occurs in the process of processing, and the color of the image slightly changes. Therefore, the color of the color patch does not match the color of the averaged color patch image.

  When the averaged color patch image is preprocessed, the parameter calculation unit 330 uses a plurality of color conversion parameters (gain parameter 311, spectral correction matrix 321, gamma parameter 341, synchronization parameter) used by the color correction unit 31. 351, YCC conversion parameter 361, sharpness parameter 371A, low-pass parameter 371B, and display LUT 381) are sequentially calculated. The process of calculating the color conversion parameters used in the color correction unit 31 corresponds to an example of the color conversion calculation process according to the present invention.

  The storage unit 400 includes a gain correction unit 310, a spectral correction unit 320, a gamma correction unit 340, a synchronization processing unit 350, a YCC conversion unit 360, a sharpness processing unit 370A, a low-pass processing unit 370B, and a display adjustment unit. The target input color and the target output color in each processing step when the color correction processing is sequentially performed in 380 are stored.

  First, the color of the pre-processed averaged color patch image is determined as the estimated input color input to the gain correction unit 310 that executes the first color correction process, and the adjustment color set in step S11 is set. The color of the averaged color patch image that has been subjected to gain correction processing using the gain parameter 311 is determined as the estimated output color. Subsequently, based on the target input color and target output color stored in the storage unit 400 and the estimated input color and estimated output color in the gain correction unit 310, the estimated output color is changed to the target output color for the estimated input color. A gain parameter 311 for approaching is calculated (step S25 in FIG. 3).

  Subsequently, the color of the averaged color patch image that has been subjected to the gain correction process using the calculated gain parameter 311 is determined as the estimated input color that is input to the spectral correction unit 320 that executes the second color correction process. Then, spectral correction processing is performed on the averaged color patch image after gain correction processing using the spectral correction matrix 321 for adjustment, and the color of the averaged color patch image after processing is determined as the estimated output color. Further, the spectral correction matrix 321 is calculated based on the target input color and target output color in the spectral correction unit 320 stored in the storage unit 400 and the estimated input color and estimated output color of the spectral correction unit 320 (FIG. 3 step S25).

  Similarly, the color of the averaged color patch image that has been subjected to spectral correction processing using the spectral correction matrix 321 is determined as the estimated input color of the gamma correction unit 340 that executes the third color correction processing. Gamma parameter 341 is calculated based on the estimated input color and estimated output color of gamma correction unit 340 and the target input color and target output color, and the averaged color patch that has been subjected to gamma correction processing using gamma parameter 341 The color of the image is determined as the estimated input color of the synchronization processing unit 350 that executes the fourth color correction process, and is based on the estimated input color, estimated output color, target input color, and target output color of the synchronization processing unit 350 The synchronization parameter 351 is calculated, and the color of the averaged color patch image subjected to the synchronization process using the synchronization parameter 351 performs the fifth color correction process. The YCC conversion parameter 361 is calculated based on the estimated input color, estimated output color, target output color, and target output color of the YCC conversion unit 360, and the color is determined using the YCC conversion parameter 361. The luminance component of the converted YCC data is determined as the estimated input color of the sharpness processing unit 370A that executes the sixth color correction process, and the estimated input color, estimated output color, target input color, and target output of the sharpness processing unit 370A The sharpness parameter 371A is calculated based on the color, and the color difference component of the YCC data is determined as the estimated input color of the low-pass processing unit 370B that executes the seventh color correction process, and the estimated input color and estimated output color of the low-pass processing unit 370B , The low pass parameter 371B is calculated based on the target input color and the target output color. The estimated output colors of the processing unit 370A and the low-pass processing unit 370B are determined as the estimated input colors of the display adjustment unit 380 that executes the eighth color correction process, and the estimated input colors, estimated output colors, and target input colors of the display adjustment unit 380 And the display LUT 381 is calculated based on the target output color.

  When all the parameters are calculated (step S26 in FIG. 3: Yes), the calculated parameters are transmitted from the parameter calculation unit 330 to each element of the color correction unit 31 and set as color conversion parameters (in FIG. 3). Step S27).

  In order to speed up the processing, the image processing apparatus 30 includes a plurality of elements that execute each of a plurality of image processes on one circuit board, and extracts an actual processing result in each of the plurality of image processes. I can't. However, in the present embodiment, the output image output to the display device 40 is subjected to the inverse conversion process of the color conversion process, whereby the input image in each image processing process is estimated, and the estimated approximate input image By sequentially performing the image processing and estimating the output image, the color conversion parameter used for the image processing can be adjusted with high accuracy.

  Here, the example in which the color conversion parameter is calculated each time the optical probe is mounted has been described above. For example, an identification number for identifying the optical probe and the calculated color conversion parameter are stored in association with each other. Alternatively, an identification number may be detected by mounting an optical probe, and a color conversion parameter associated with the identification number may be used.

1 is a schematic configuration diagram of an endoscope system according to a first embodiment of the present invention. It is a schematic functional block diagram of an endoscope system. It is a flowchart figure which shows the flow of a series of processes which calculate a color conversion parameter. It is a figure which shows an example of a gray step chart. It is a figure which shows an example of a gray solid chart.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Endoscope system 10 Optical probe 20 Light source device 30 Image processing apparatus 40 Display apparatus 11 Probe part 12 Operation part 13 Light / signal guide 121 Bending operation lever 122 Shooting button 123 Selection button 131 Light guide 132 Signal line 133 CCD
140 Color Filter 150 A / D Conversion Unit 160 Imaging Control Unit 300 Preprocessing Unit 310 Gain Correction Unit 320 Spectral Correction Unit 330 Parameter Calculation Unit 340 Gamma Correction Unit 350 In-house Processing Unit 360 YCC Conversion Unit 370A Sharpness Processing Unit 370B Low Pass Processing Unit 380 Display adjustment unit 390 Overall control unit 400 Storage unit

Claims (11)

An image acquisition process for acquiring a color patch image obtained by photographing a color patch by an imaging device that captures a subject and obtaining a subject image and an image processing device that performs image processing on the subject image and outputs the image;
An image estimation process for performing an inverse transformation process of the image processing on the color patch image acquired in the image acquisition process, and estimating a color patch image before the image processing is performed;
A color conversion calculation method comprising: a color conversion calculation process for calculating a color conversion for converting the color of the color patch image estimated by the image estimation process into an original color of the color patch.   The color conversion calculation method according to claim 1, wherein the color conversion calculation step is a step of creating a color conversion definition that defines the color conversion. The image processing apparatus is configured to set processing parameters used for the image processing and execute the image processing using the set processing parameters.
Before the image acquisition processing, the image processing apparatus has a parameter setting process for setting a processing parameter to be converted as a processing parameter that can be converted without processing in the image processing. The color conversion calculation method according to claim 1 or 2, characterized in that: The image acquisition process is a process of acquiring a plurality of color patch images captured and output by the imaging device and the image processing device, each of a plurality of color patches having different color systems.
The image estimation process is a process of estimating the color of the color patch image before the image processing is performed for each of the plurality of color patch images;
In the color conversion calculation process, as the color conversion, a correction conversion for correcting a distortion of the color of the subject image caused by the imaging device, and a color estimated for each of the plurality of color patch images in the image estimation process The color conversion calculation method according to claim 1, wherein the calculation is based on an original color of each of the plurality of color patches. The image acquisition process is a process of acquiring a plurality of color patch images that are photographed and output by the imaging device and the image processing device, each of a plurality of color patches having similar colors and different densities.
The image estimation step is a step of estimating the density of the color patch image before the image processing is performed for each of the plurality of color patch images,
In the color conversion calculation process, as the color conversion, a gradation correction conversion for correcting the distortion of the gradation of the subject image caused by the imaging device is estimated for each of the plurality of color patch images in the image estimation process. 5. The color conversion calculation method according to claim 1, wherein the color conversion calculation method is a process of calculating based on the determined density and an original density of each of the plurality of color patches. The image acquisition process acquires a plurality of first-type color patch images obtained by photographing and outputting a plurality of first-type color patches having the same color and different densities from each other by the imaging device and the image processing apparatus. A process having a first acquisition process and a second acquisition process of acquiring a plurality of second-type color patch images obtained by photographing the plurality of second-type color patches having different color systems and output by the imaging device. Yes,
The image estimation process includes, for each of the plurality of first type color patch images, a first estimation process for estimating a density of the first type color patch image before the image processing is performed, and the plurality of second type patch images. Each of the color patch images has a second estimation step of estimating the color of the second type color patch image before the image processing is performed,
In the color conversion calculation process, the gradation correction conversion for correcting the distortion of the gradation of the subject image caused by the imaging device is estimated for each of the plurality of first-type color patch images in the first estimation process. A gradation correction calculation process that is calculated based on the density and the original density of each of the plurality of first-type color patches, and a correction conversion that corrects color distortion of the subject image caused by the imaging device, Color calculated based on a color obtained by performing the gradation correction conversion on the color estimated for each of the plurality of second-type color patch images in two estimation processes and an original color of each of the plurality of second-type color patches The color conversion calculation method according to claim 1, wherein the color conversion calculation method includes a correction calculation process. In the image acquisition process, one or more color patches are photographed and output by the imaging device and the image processing device so that colors of the same density and the same density exist at a plurality of locations within the photographing angle of view 1 The process of acquiring more than one color patch image,
The image estimation process is a process of estimating the density of the color patch image before the image processing is performed for each of the plurality of locations.
In the color conversion calculation process, as the color conversion, a shading correction conversion for correcting a sensitivity difference of the image pickup device within the shooting angle of view is performed, and the density and the color patch estimated for each of the plurality of locations in the image estimation process. The color conversion calculation method according to claim 1, wherein the color conversion calculation method is a process of calculating based on the original density of the color conversion. The image acquisition process acquires a plurality of first-type color patch images obtained by photographing and outputting a plurality of first-type color patches having the same color and different densities from each other by the imaging device and the image processing apparatus. A first acquisition process, a second acquisition process for acquiring a plurality of second-type color patch images obtained by photographing the plurality of second-type color patches having different color systems and output by the image pickup apparatus, and the image pickup apparatus. To obtain one or more third-type color patch images obtained by shooting and outputting one or more third-type color patches so that colors of the same density and the same density exist at a plurality of locations within the shooting angle of view. 3 acquisition process,
The image estimation process includes, for each of the plurality of first type color patch images, a first estimation process for estimating a density of the first type color patch image before the image processing is performed, and the plurality of second type patch images. For each color patch image, a second estimation process for estimating the color of the second type color patch image before the image processing is performed, and a third type before the image processing is performed for each of the plurality of locations. A third estimation process for estimating the density of the color patch image,
In the color conversion calculation process, the gradation correction conversion for correcting the distortion of the gradation of the subject image caused by the imaging device is estimated for each of the plurality of first-type color patch images in the first estimation process. A gradation correction calculation process that is calculated based on the density and the original density of each of the plurality of first-type color patches, and a shading correction conversion that corrects a sensitivity difference of the imaging device within the shooting angle of view. A shading correction calculation process for calculating based on the density obtained by performing the gradation correction conversion on the density estimated for each of the plurality of locations in the three estimation processes and the original density of the third type color patch; The correction conversion that corrects the resulting color distortion of the subject image is converted into the color estimated for each of the plurality of second-type color patch images in the second estimation process, 8. The process according to claim 1, further comprising a color correction calculation process for calculating a color based on a color subjected to shading correction conversion and an original color of each of the plurality of second-type color patches. The color conversion calculation method according to claim 1. An image acquisition unit that acquires a color patch image obtained by shooting a color patch by an imaging device that captures a subject and obtains a subject image and an image processing device that performs image processing on the subject image and outputs the image;
An image estimation unit that performs an inverse conversion process of the image processing on the color patch image acquired by the image acquisition unit, and estimates a color patch image before the image processing is performed;
A color conversion calculation apparatus comprising: a color conversion calculation unit that calculates a color conversion for converting a color of a color patch image estimated by the image estimation unit into an original color of the color patch. An image acquisition unit that acquires a color patch image obtained by shooting a color patch by an imaging device that captures a subject and obtains a subject image and an image processing device that performs image processing on the subject image and outputs the image;
An image estimation unit that performs an inverse conversion process of the image processing on the color patch image acquired by the image acquisition unit, and estimates a color patch image before the image processing is performed;
A color conversion calculation unit for calculating a color conversion for converting the color of the color patch image estimated by the image estimation unit into the original color of the color patch;
A color conversion apparatus comprising: a color conversion unit that performs color conversion processing on the subject image in accordance with the color conversion calculated by the color conversion calculation unit. A light source that emits light;
An optical probe including a light guide path that guides light emitted from the light source to a subject and an imaging device that captures the subject and obtains a subject image;
An image processing apparatus that performs image processing on an image of a subject obtained by the imaging device and outputs the image; and an image acquisition that acquires a color patch image obtained by photographing a color patch by the imaging device and the image processing device. And
An image estimation unit that performs an inverse conversion process of the image processing on the color patch image acquired by the image acquisition unit, and estimates a color patch image before the image processing is performed;
A color conversion calculation device including a color conversion calculation unit that calculates a color conversion for converting the color of the color patch image estimated by the image estimation unit into the original color of the color patch;
The image processing apparatus, wherein the subject image is subjected to color conversion processing according to the color conversion calculated by the color conversion calculation apparatus as at least a part of the image processing and is output. Mirror system.

Images