JPS63173182A - Color image processing system - Google Patents

Abstract

PURPOSE:To emphasize a fine color difference without varying the impression of an original image large by representing the color of color image information as a hue, saturation, and brightness, and emphasizing them independently of one another so that the distribution is widened based on a mean color as a center. CONSTITUTION:Image data from a TV camera 1 is read and stored in an image memory 3. Then, R, G, and B data are read out of the memory 3 and converted by brightness, saturation, and hue data through the linear conversion of the color coordinate conversion module 11 of an image processor 4. Then respective histograms of color mark coordinates are calculated by using a statistical calculation module 8. Further, a microcomputer 8 finds the mean color of the whole image from the calculated histograms, performs an emphasizing process by the brightness, saturation, and hue of the mean color, and stores the results in the memory 3. Then the data stored in the memory 3 is sent out to a D/A converter 5 and the emphasized image is observed on a color TV monitor 6. Consequently, the fine hue difference can be emphasized by said method without varying the impression of the image unnaturally.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a color image processing method for performing color enhancement processing on color images such as endoscopic images.

[Conventional technology]

Conventionally, several attempts have been made to make images easier to see by changing the brightness and tone of color images through image processing.

In particular, in diagnosis based on endoscopic images, healthy areas and diseased areas are distinguished by subtle changes in brightness and color tone, but most endoscopic images are reddish or skin-toned. Therefore, it is difficult for non-skilled doctors to distinguish the lesion, so it is desired that endoscopic images be made easier to see by emphasizing subtle changes in color tone.

As one means of such image enhancement processing, J.

Tsujiuchi, et al, +'Dig
Ital Processing of Endosc.
opic Co1or Images' 0ptica
lCommunication.

Vol, 55. No. 4 (1985), p.
The following method is proposed in p.242. In other words, when expressing a color image using the three primary colors of red (R), green (G), and blue (B), this method considers, for example, a cylindrical coordinate system with the R-G-B axis as the brightness axis. A line connecting the average values of the data distribution on several planes perpendicular to the plane (equal brightness planes) is set as a new brightness axis, and by a method using a linear transformation called color Laplacian, centering on this new brightness axis. The color tone is changed in the direction of increasing the vividness (saturation) of the color.

[Problem that the invention seeks to solve]

However, although the method proposed in the above literature is very simple and provides good effects, it has the following major drawbacks. In other words, since the colors are spread outward from the average color axis, not only the saturation but also the hue change, and as a result, colors that were originally close to neutral gray are now more similar to the average color in the image. The problem is that this may result in extremely unnatural color tones, such as colors that are complementary to other colors. Specifically, for example, in an endoscopic image, a problem arises in that a portion that appears whitish due to strong illumination changes to a cyan color, which is a complementary color to the average red color. These drawbacks are due to the fact that the hue, saturation, and brightness are not changed independently.

The present invention was made in order to solve the problems in the conventionally proposed color image enhancement processing method, and it emphasizes subtle differences in color tone without unnaturally changing the impression of the image.
It is an object of the present invention to provide a color image processing method that can change the image into an image that makes it easy to understand the difference in color tone.

[Means and operations for solving the problems] In order to solve the above problems, the present invention provides means for converting color image information into data with one hue of brightness and saturation, and means for calculating the average color of the entire image. , a calculation means for emphasizing the distribution of data centering around one hue of lightness and saturation of the average color, and expanding at least one of the hues for each hue of brightness and saturation; This system constitutes a color image processing method that performs color emphasis processing with means for converting data of brightness, saturation, and nine hues into a color representation method of an original color image.

In the color image processing method configured in this way, color image information is converted into data with one hue of brightness and one saturation, and the average value of each of the brightness and one hue of saturation of the color image information is calculated. , the distribution of at least one of the hues with two saturations is expanded around its average value, and the data of the hue with one hue of brightness and saturation processed to expand the distribution is returned to the original color image expression. be.

This expands the color distribution of the entire image, and without changing the average color too much, emphasizes subtle differences in tone without changing the impression of the image unnaturally, creating an image that makes it easy to understand the differences. It becomes possible to convert.

〔Example〕

Examples will be described below. Various proposals have been made regarding methods of expressing colors, some of which have been put into practical use or established by the CIB.

One of these methods, which is compatible with human color perception and easier to understand, is a method of expressing color using three attributes: hue, saturation, and brightness, which are treated as spatial coordinates. For simplicity, a rectangular coordinate system with brightness as one axis is considered, and the remaining two axes represent hue and saturation together.

In the present invention, the object is achieved by representing and processing a color image using hue, saturation, and brightness.

Next, a color expression method using these three attributes will be explained.

For example, the following methods can be considered for expressing an image expressed using the three primary colors R2O and B, as in a color television, in a form close to hue, saturation, and brightness.

(1) Utilization of uniform color space (CIELUV, CIEL
AB) For things like color television that express colors through additive color mixture and to which CIE's XYZ coordinate system theory can be applied, it is theoretically relatively easy to convert the color system to chromaticity coordinates. It is possible to do so.

Here, we will introduce the R, G, and B signals of a television system that conforms to the NTSC standard using the CIE1976 L”u
"v" (CIELUV), C'TE 1976L"a
An example of conversion to coordinates in two uniform color spaces of "b" (CIELAB) will be explained.

The conversion formula from the R, G, B system to the CIE xyz color system is shown below.

X = 0.607 R+ 0.173 G + 0.
201 BY = 0.299 R+ 0.587 G
+0.114 BZ = -0,066
0+1.118B Also, xyz color system brown L ” u
Conversion to the "V" and L"a"b" systems can be performed using the following equations, respectively.

u" = 13L" (u 'u@')v" = 13L
”(v' - V6')X+15Y+32 However, YO+ uo, V is Y on the reference white surface.

ul, v+ value However, Xo, Yo, and Zo are X on the reference white surface.

The value Ll of Y and Z represents the lightness, u 'I v '', a * b
If 0 hue, H1 saturation, and C1 lightness, which represent hue and saturation by *, are represented by V, then it can be defined as follows, for example.

(■), Use of simple coordinate representation An attempt was made to represent the R, G, and B signals of color television and other TV programs using questionable chromaticity, saturation, and brightness.
This is done in fields such as graphics. Such pseudo-representation methods include the H3I triangular pyramid color model, H3V hexagonal pyramid color model, H3L hexagonal pyramid color model, etc., which are based on finite R, G, and B values in the R, G, and B rectangular coordinate system. The main diagonal axis of the cube, which has a range of possible values, is defined as the lightness axis, and hue and saturation are defined in a plane perpendicular to this axis.

An example is shown below.

Next, emphasis on differences in color tone using the coordinate system shown above will be explained. In this example, Lla m b*
The coordinates of First, FIG. 1 shows an example of the data distribution of endoscopic image data in R, G, and B space. When the distribution of this data is examined in the L 11 a1″b1 coordinate system, it becomes as shown in FIG. 2.

As shown in FIG. 2, when examining the distribution of color on the chromaticity coordinates in the endoscopic image, it is found that blood vessels and redness do not appear in the skin-colored portion of the mucous membrane 101, which can be said to be the background color.
02. A distribution of self-answer 103 etc. exists in the vicinity,
It can be seen that the endoscopic image is composed of an extremely narrow range of colors. To further explain about FIG. 2, compared to the color of the normal mucous membrane 101, self-answer 103 generally has high brightness but low saturation, while blood vessels and redness 102 have high saturation but low brightness. low. Although there is often not much difference in hue, the color of normal mucous membrane 101 tends to be a little closer to yellow than that of blood vessels and redness 102. In addition, in FIG. 2, 105 indicates an average color.

Therefore, the distribution of normal mucous membrane 101 and blood vessels/redness 102
, self-answer 103, etc. on the chromaticity coordinates, that is, by increasing the color difference, diagnosis can be easily performed. However, if the impression changes too much compared to the original image, it may actually hinder diagnosis.

It has been found that it is necessary to pay attention to the following points in order to reduce the change in impression of such an image and obtain the effect of emphasis.

■ The average color (normal mucous membrane color) does not change before and after treatment.

■ Even if the color is a color other than the average color (redness of one blood vessel, color of self-identification, etc.), the hue and brightness should not be changed drastically.

In consideration of the above points, the present invention emphasizes color tones using the following method.

First, the values of Ls, am, and ba are calculated from the R, G, and B values for each pixel of the entire original image, and L′″, al
.

By determining the distribution in b0 space, the average color L*a'',
Obtain the 0th order average color L*, %, bll to obtain the 0-order average color L*, %, bll, and calculate the average values V, C, and H of the two hues.
Emphasizes the saturation 1 hue. The following function is used for emphasis.

V'-f(V) C'=g(C) H'=h(H) However, L, VL<V< VM, CL<C< CI
I+ HL <H<) (, If the lower upper limit of I, V and C is V,..., V□X+ am7°Csex, the functions f (V), g (C), h (H) are V
II(, ≦f (vt) < vL, f (V) =
V.

■H<f(v,I)≦■l, 1. ．． I Cmtn5g(C+,)<Ct2g(C)=C.

C, < g (Co)=C1゜H-π≦h (HL) < Ht, h (H)=
H.

)IN<h(Ho)≦π+π By using a monotonically increasing function as described above,
It has been found that images that are easy to diagnose can be obtained without significantly changing the image impression. An example of a particularly effective function is shown in Figure 3. The function is determined by the average value V, C, H
The observer may decide empirically each time by looking at the value of , τ, may be automatically determined based on the values of ``V, C from the R, G, and B values of the first pixel of the original image.

Calculate the value of H, v', c' by the above function.

H' is calculated, and R, G, and B values are calculated from this value and used as the value of the first pixel of the processed image. By sequentially performing this processing for each pixel, an image as a result of the processing is obtained.

The above processing procedure is shown as a flowchart in FIG. In addition, in FIG. 4, R (f, m), G (l.

m>, B(jl, m) represent the R component, G component, and B component of the original image, and R'(jl, m), G
' (1, m) and B' (l, m) represent the R component, G component, and B component of the processed image.

Furthermore, the data distribution in the L 11 am b * coordinate system in the processed image is shown in FIG. 5. As can be seen from FIG. Redness 102. The color difference between the average values and the distribution of data such as self-answer 103 is wider than that of the original image, making it easier to distinguish. Further, the average value of the data distribution of the normal mucous membrane and the data distribution of the entire image does not change much before and after processing, so the impression of the entire image does not change significantly. Although not shown in the figure, the brightness has a similar tendency.

In the first embodiment described above, the values of L", *, and ba are calculated from the R, G, and B data of the original image, the average value is determined, and the function for enhancement is determined from the average value. Ls again from the R, G, B data of the original image.

The values of a* and b'1 are calculated, V, C, and H are determined and emphasized, converted to R, G, and B data, and the resulting image is used as the processed image. The processing procedure according to this embodiment does not require a special image memory to store intermediate results of processing, and has the advantage that only one set of image memory is required if the original image data is rewritten with the processing result data. There is. However, there is a drawback that it is necessary to convert the R, G, and B data of the original image to L"+a"+b0 twice.

Next, we will show a second embodiment that requires a larger storage area than the first embodiment but has improved processing speed. In this embodiment, the RlG, B data of the original image is ”
, a", b" are created on the memory, and all are converted into the average color L+.

Find bo. From this, find V, C, H and emphasize function f
(V), g(C), and h(H) are determined. Next L9a*
For each pixel of image data represented by b*, Ls,
The values of V, C, and H are calculated from ao and bo, and v' is obtained from the emphasis functions r (V), g (C), and h (H).

C' and H' are obtained, and the values of R, G, and B are calculated backwards to obtain a processed image.

The above processing procedure is shown as a flowchart in FIG.
The processing results are exactly the same as in the first embodiment. However, the image data is an intermediate result.

If the bl data is set to the same number of bits as the original image data, the error may have a large effect when it is inverted to R, G, and B again after enhancement, so it is necessary to increase the number of bits. It may come. In this case, it is necessary to further increase the storage area used, but R, G
, B to the image data of Lm, ao, and bo is not required, and the processing time is shortened.

FIG. 7 is a block diagram showing an example of the hardware configuration of the color image processing method according to the present invention.

This hardware system has R,G
, a TV camera 1 that inputs image color signals such as B,
An A/D converter 2 that converts an analog signal from the TV camera 1 into a digital signal, an image memory 3 for storing input image data and processed image data, and data stored in the image memory 3. or an image processing processor 4 for performing predetermined processing on the input data obtained through the A/D converter 2; a D for converting the output data (digital signal) of the image memory 3 or the image processing processor 4 into an analog signal; /A converter 5, and a color image monitor 6 that displays an image based on the output data of the D/A converter 5.
, the image memory 39 image processing processor 4. A/D converter 2. An image bus 7 and an A/D converter 2 installed for image data transfer between the D/A converters 5 and 2. D/A converter 5
1 Image memory 33 A microcomputer 8 for sending various control signals to the image processing processor 4, etc., and calculating parameters.8 A system bus 9 for exchanging data between the microcomputer 8 and each of the above components. It is made up of. Also, the image processing processor 4
Inside, there are provided a statistical calculation module 101 for performing histogram calculations, a color coordinate conversion module 11 for performing color coordinate conversion, and a two-person table conversion circuit 12 for performing table conversion between variables. There is.

In this configuration example, to simplify the explanation, the image bus 7
has at least 6 buses A, B, C, A'.

B' and C', and regarding the image processing processor 4, the image buses A, B, and C are input signals, and the image buses A' and B
', C' are output signal transfer paths, and image buses A, B with respect to the image memory 3.

C is assumed to be an output signal, and image buses A', B', and C' are assumed to be input signal transfer paths.

FIG. 8 is a block diagram showing the inside of the color coordinate conversion module 11. As shown in FIG. This configuration is based on patent application No. 61-990.
It has the same structure as No. 69, and has image buses A, B, and C.
Nine look-up table memories 13 to 21, look-up table memories 13 and 14 that convert the above data
．． 15 sets, lookup table memory 16.17.
Group 18, Lookup Table Memo January 9.20.2
It consists of a register 25 for specifying the contents of the lookup table memo January 3 to 21, and an image bus A. , B, and C are subjected to predetermined processing and sent to image buses A', B', and C'.

FIG. 9 is a block diagram showing a histogram generation circuit within the statistical calculation module 10.

This histogram generation circuit includes an image bus data input selection circuit 26 that selects one of the image buses A, B, and C1 or commands from the microcomputer 8 according to instructions from the microcomputer 8, and an output of the selection circuit 26. and the output of the register 28 that holds the control signal from the Microninputer 8 as address inputs, a histogram memory 27 for writing the histogram contents, and an incrementer 29 for incrementing the output of the histogram memo + J27 by one. It is configured. Then, the output of the incrementer 29 is again sent to the histogram memory 27 and written therein, and the incrementer 29 adds 1 to the frequency of the histogram corresponding to the density value of the image data sequentially read out. After all pixels are read out, the histogram frequency is stored in the histogram memory 27.

The stored results are configured so that the microcomputer 8 can view them via the system bus 9.

FIG. 1O is a block diagram showing the inside of the two-manpower table conversion module 12. As shown in FIG. This two-man power table conversion module 12 has image buses A and B.

An image bus data manual selection circuit 30 that selects two of C, and a look amplifier table memory 31.8 that configures a two-output lookup table for the outputs P and Q of the selection circuit 30. table memory 31
A register 32 that holds a signal for selecting the table of. Two outputs P', Q' of the look amplifier table memory 31
image buses A', B'.

and an image bus data output selection circuit 33 that switches to output to any two of the image bus data C'. The look amplifier table memory 31 is composed of a ROM or a RAM, and if it is composed of a RAM, it can also be accessed from a microcomputer.

Next, the operation of the hardware system configured as described above will be explained. First, the case of image enhancement in the "a*b* space will be described. The processing steps in the L 11 am b* space are as follows.

(2) Reading and storing image data from the TV camera 1, etc. into the image memory 3;

■ The R, G, and B data are sequentially read from the image memory 3 and sent to the image buses A, B, and C. The color coordinate conversion module 11 performs the following linear transformation to convert the image buses A', B', and C'. The image is then stored in the image memory 3 again.

■ The data X, Y, and Z stored in the image memory 3 are sequentially read out and sent to the image buses A, B, and C. The color coordinate conversion module 11 performs calculations as follows, and the data are sent to the image buses A' and B' again. , C' has the result 111. am, b" are sent and stored in the image memory 3.

■ Lm, am, b* in the image memory 3 using the histogram generation circuit in the statistical calculation module IO
Take a histogram for each data. When taking a histogram, change the value of the register 28 to switch the histogram bank, and store Lm, Lm,
All histograms of am and b* can be taken.

■ Concerning the histogram calculated in step (■) using the microcomputer 8, its average values V, a'', b
'', and then calculate the average value π of hue H1 saturation C based on the following equation.

■ Among Lm, am, and b* stored in the image memory 3, the hue H3 and saturation C are sequentially converted based on (a''+b'') using the two-person table conversion module 12. I ask. That is, the two-dimensional table transformation becomes c=7 violet=f(a", b") a"H=jan-'-=g(a", b")b".

This can be done, for example, by applying am and b* to the two-person conversion module 12 using image buses B and C, respectively, and then applying the resulting hue H and saturation C to image buses B' and C'. Store it in the image memory 3.

■ Define a color emphasis function in the microcomputer 8 using the average value T and τ calculated in step ■ above, and convert the function using the lookup table memories 13 to 21 in the color coordinate conversion module 11. cent. Regarding this cent, lookup table memo 1 month 4.15.16.18.19゜20 is cent as identity function, lookup table memory 13
f (V) in the look amplifier table memory 17, and g in the look amplifier table memory 17.
(C) in the look amplifier table memory 21 h (H
) in cents.

■ The color coordinate transformation defined in step ■ above is applied to V, C, and H stored in the image memory 3, and the result (
V', C', H') are stored in the image memory 3 again.

■ Stored in the image memory 3 (V', C').

H'), amj again for the pair (C', H')
, bml using the two-person table conversion module 12. 2 person table conversion module 1
2, calculate a′″1.bml, and the result a′Z
b$l is stored in the image memory 3 together with L"'=V'.

[Phase] L0' stored in step (2) above.

For amr and bml, the coordinates (X', Y', Z') are determined using the color coordinate conversion module 11.

This can be easily obtained by the following transformation, and the transformation result is stored in the image memory 3 again.

(2) Using the color coordinate conversion module 11 again for (X', Y', Z') stored in the image memory 3, the values of (R', G', B') are determined.

This value can be determined by linear transformation as follows, and the transformation result is stored in the image memory 3 again.

@ Stored in step O above (R', G'.

The value of B') is sent to the D/A converter 5, and the enhanced image is observed on the color TV monitor 6.

By going through the above steps ① to ＠, color emphasis processing can be performed on a color image.

The above emphasis processing step includes L*, a 11 ,
Although we have shown image processing in b* space, it is possible to use a different coordinate system (v.

α, β) can be similarly implemented using the above hardware system. In this case, the above steps ■ and ■ become one, and the linear transformation becomes . Also, the steps [phase] and ■ become one, and the linear transformation becomes . can do.

As described above, by adopting the above hardware configuration, color emphasis processing can be carried out efficiently.

As described above, the color image processing method according to the present invention performs processing to emphasize subtle color differences on images that have colors concentrated in a specific range, making it easier to understand the differences. It is used to convert images into images, expressing colors in terms of hue, saturation, and brightness that are easy to understand in correspondence with human color perception, and emphasizing each independently to widen the distribution around the average color. By using this method, it is possible to prevent the problems that occur with conventional enhancement methods, such as colors changing to complementary colors and greatly differing from the impression of the original image. Differences can be emphasized.

In each of the above embodiments, endoscopic images were explained, but in microscopic images as well, such as stained specimens and metallurgical microscope images, the images concentrate on the colors of specific areas, and subtle colors within these images are concentrated. There are many images in which it is desirable to be able to clearly observe the difference in image quality, and it has been confirmed that the enhancement effect can be produced by applying the method according to the present invention to these images as well.

In addition, in each of the above examples, hue, saturation, and brightness were calculated and the average value of each was calculated, but the coordinates used in the initial expression, such as R, G, and B coordinates, Find the average values R, G, and B, and use those values to calculate hue, saturation,
Almost the same results were obtained by calculating the brightness and using it as the average value of hue, saturation, and brightness for color enhancement. In short, the basic processing in the present invention is to find the average color,
The point is to emphasize color by expanding the distribution of data independently for hue, saturation, and brightness around the average color.

Further, in each of the above embodiments, all of the hue, saturation, and lightness were emphasized, but good results were also obtained by emphasizing at least one of them, especially only the saturation.

Furthermore, in each of the above embodiments, an example was shown in which the image to be emphasized is expressed in the three primary colors of R, G, and B, such as in a color television; Of course, it can be applied to images represented by C, M, Y, etc., and can be applied to any image as long as it is an image whose brightness, saturation, and hue can be calculated.

〔Effect of the invention〕

As described above in detail based on the embodiments, according to the color image processing method according to the present invention, the color distribution of the entire image is widened and the average color is not changed much, so that the image becomes unnatural. It is possible to emphasize subtle differences in color tone and convert them into images that are easy to understand without changing the impression.

[Brief explanation of the drawing]

Fig. 1 is a diagram showing an example of the distribution of endoscopic image data in R, G, and B space, and Fig. 2 is a diagram showing an example of the data distribution of endoscopic image data in the A diagram showing the distribution expressed by , Figure 3 ^, (B) is a diagram showing an example of the emphasis function,
FIG. 4 is a flowchart showing the processing procedure of the first embodiment of the present invention, and FIG. 5 shows the processing images Lm, am,
Figure 6, a diagram showing the distribution of data expressed in the ba coordinate system, is
A flowchart showing the processing procedure of another embodiment of the present invention,
FIG. 7 is a block diagram showing an example of hardware constituting the color image processing method according to the present invention, FIG. 8 is a block diagram showing the internal configuration of the color coordinate conversion module, and FIG. 9 is a statistical calculation module. FIG. 10 is a block diagram showing the internal configuration of the two-person table conversion module. In the figure, 101 is normal mucous membrane, 102 is blood vessel/redness,
103 indicates the answer, 104 indicates the halation, and 105 indicates the center color. Patent applicant: Olympus Optical Industry Co., Ltd. Figure 1 Figure 1 Figure 3 (A) (B) Figure 4 Figure 6 Figure 9

Claims (1)

[Claims] means for converting color image information into data on brightness, saturation, and hue; means for determining the average color of the entire image;
calculation means for expanding emphasis on at least one of saturation and hue, and means for converting the brightness, saturation, and hue data calculated by the calculation means into a color expression method of the original color image. A color image processing method that performs color enhancement.