JPH06187443A - Color picture highlight area extracting device using reflection component and color picture converter - Google Patents

Abstract

PURPOSE:To correctly extract the highlight area by regarding the highlight being including only when two kinds of vectors are obtained by vector synthesis. CONSTITUTION:A high luminance area extracting part 107 extracts picture elements whose luminances are higher than a threshold being the minimum value of luminance at zero-crossing points of a luminance picture. A similar color area extracting part 108 obtains areas consisting similar colors for each picture element of a high-luminance area. A reflection component number discriminating part 110 judges that the objective area consisting of similar colors includes a highlight if two kinds of vectors are obtained by a vector synthesis part 109. A highlight area extracting part 106 regards a cluster having a higher luminance out of two clusters obtained by a cluster preparing part 104 as the cluster generated by the vector representing the specular reflection component and extracts picture elements included in the cluster as picture elements from the highlight area.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention extracts a reflection component for a region having a similar color from an image in which a highlight and a region having a color or brightness similar to the highlight are mixed.
By extracting the pixels included in the cluster that represents the specular reflection component in the three-dimensional space around the three primary color components of the pixel, the correct highlight area is extracted and the reflection component that performs color conversion of the color image is used. The present invention relates to a highlight area extraction device for a color image and a color image conversion device.

The present invention relates to a technique necessary for estimating the surface reflectance of an object, and is related to FA field such as identification of an object in an image, broadcasting such as color conversion of an image, communication field, product catalog,
In posters and photographs, it can be used to emphasize or suppress specular reflection components to bring more sense of reality.

[0003]

2. Description of the Related Art When an area is divided by surface reflectance using the intensities of three primary color components of a pixel in a color image, a highlight area having a pixel value reflecting the color of a light source is different from an area other than the highlight area. It is divided into areas. Therefore, in order to correctly perform area division based on the surface reflectance even for an image including highlights, a technique for correctly extracting highlight areas is required.

Conventionally, a method of extracting a highlight region from an image has a difference in reflection characteristics on the object surface, that is, specular reflection that reflects the spectral distribution of the light source as it is and diffuse reflection that reflects the spectral reflection characteristics of the object surface. There is one that utilizes two reflection components (see reference [1]). (References) [1] Gudrun J. Klinker, Steven A. Shafer, and Takeo
Kanade: "USING A COLOR REFLECTION MODEL TO SEPAR
ATE HIGHLIGHTS FROM OBJECT COLOR ", Proceedings of
the First International Conference on Computer Vis
ion (ICCV), pp.145-150 (1987). In this method, the value of each pixel is projected onto a three-dimensional color space centered on the three primary color components of the pixel and its distribution is used. As a three-dimensional color space, HVC, L ^* u ^* v
^Although various color spaces such as ^* can be considered, the RGB color space having three primary colors of red (R), green (G), and blue (B) is generally used, so the following explanation will be made using this. To do. When the R, G, and B intensities are measured for each pixel in the region including the highlight, if the region has the same surface reflectance, one locus in the RGB color space as shown in FIG. Make up. This locus is represented by two straight lines, namely a highlight line (straight line L _S ) by specular reflection and a matte by diffuse reflection.
can be approximated to line (straight line L _D ). here,
The direction vector of the straight line L _S of the locus due to specular reflection indicates the color component of the light source. The highlight area is obtained by extracting the area corresponding to the specular reflection on the locus.

FIG. 7 is a block diagram showing an example of the configuration of a conventional color image highlight area extracting apparatus. In FIG. 7, 101 is an RGB camera unit, 102 is an eigenvalue / eigenvector calculation unit, 103 is a direction vector extraction unit, 104
Is a cluster generation unit, 105 is a specular reflection component detection unit, 10
6 is a highlight area extraction unit.

Next, the operation of each section shown in FIG. 7 will be described. First, the RGB camera unit 101 inputs a color image. By the eigenvalue / eigenvector calculation unit 102,
The color image is divided into small areas, and the eigenvalues and eigenvectors of the covariance matrix at the pixels in each small area are calculated. The direction vector extraction unit 103 determines the shape of the pixel value distribution of the small area in the RGB color space from the eigenvalues, and averages the vectors of the small areas having similar colors to the small area whose shape is determined to be linear. To do. In the cluster generation unit 104, a straight line is obtained by using the vector obtained by the direction vector extraction unit 103 as a direction vector of the straight line in the RGB color space, and pixel values of the color image are projected centering on this straight line to generate a cluster. (See the above-mentioned reference [1]).

In the specular reflection component detection unit 105, a combination of clusters representing the diffuse reflection component and the specular reflection component is determined based on the distances between the clusters obtained by the cluster generation unit 104 and the positions of the intersections. The highlight area extraction unit 106 extracts, as a highlight area, a set of pixels included in the cluster determined to represent the specular reflection component by the specular reflection component detection unit 105.

On the other hand, conventionally, color conversion for a color image is usually performed by a skilled engineer by adjusting the three attributes of color, for example, hue, lightness, and saturation, based on experience and intuition. It was

Further, in recent years, the intensity of three color components of a color image, for example, red (R), green (G), and blue (B), is increased or decreased to make the color exactly the same as the display. It is practiced to create a printed matter or, conversely, to generate a display display that has exactly the same color as the printed matter.

However, for a color image including highlights, a device for automatically extracting a specular reflection component and a diffuse reflection component, which are two components of reflection, and independently operating the intensities of the specular reflection component and the diffuse reflection component. Does not exist yet.

An example of the configuration of a conventional color image conversion device is shown in the block diagram of FIG. In FIG. 8, 202 is a color image input unit, 204 is a color changing unit, and 205 is a color image output unit.

Next, the operation will be described. First, the color image input unit 202 inputs a color image. next,
The color changing unit 204 increases or decreases each of the three color attributes of the color image based on the feeling of a skilled person to generate a desired image. The color image output unit 205 outputs the image generated by the color changing unit 204.

[0013]

As described above, in the conventional highlight area extraction device, since the pixels included in the cluster representing the specular reflection component are simply extracted as the highlight area, the brightness is high to some extent and If there is a region with a color similar to that of the light source, the region other than the highlight region may be erroneously extracted as the highlight region because it is included in the cluster that represents the specular reflection component of the highlight region. .

FIG. 9 shows an example in which a region other than the highlight region is erroneously extracted in the highlight region extraction to which the conventional method is applied. 9A shows an original image, and FIG. 9B shows a highlight area extracted by a conventional method. Figure 9
(C) shows a highlight area of a desired extraction result.
The original image shown in FIG. 9A has a white area in the lower part of the black background, a green area in the upper right corner, and a highlight area in the green area.

According to the conventional method, the white area, the highlight area and the periphery thereof are extracted as the highlight area as shown in FIG. 9B. In order to correctly extract the highlight area, it is necessary to distinguish the white area from the highlight area and extract the white area like the highlight portion shown in FIG. 9C.

First, the present invention solves the above-mentioned drawbacks, and even if there is a region on the image having a color similar to the color component of the light source and having high luminance, the region is not extracted as a highlight region. , The purpose is to correctly extract only the area including highlights.

Further, in the conventional color image conversion apparatus, as described above, there is a large part that depends on the sensation of an expert, and it is not possible for a general layman to easily operate, and the specular reflection component and diffuse reflection There is a problem that the components cannot be separated and they cannot be changed independently. Therefore, there is a need for a technology that automatically extracts two reflection components, specular reflection and diffuse reflection, from a color image and realizes a function that enables independent operation of each reflection component intensity.

Secondly, the present invention solves the problems of the conventional color image conversion apparatus, so that even an ordinary operator who is not an expert can
The purpose is to enable color conversion of a realistic and natural color image.

[0019]

A highlight area extracting device for a color image using a reflection component of the present invention comprises a high brightness area extracting means for extracting a high brightness area from a color image,
Similar color region extraction means for dividing the obtained high-brightness region into regions of similar color, vector integration means for integrating the eigenvectors obtained by dividing the similar color region into small regions, and the result of the integration In the case where there are two kinds of obtained vectors, by having a means for extracting the pixels included in the cluster corresponding to one of the vectors as a highlight area, the brightness is high to some extent and the color is similar to the light source. Without mistakenly extracting the area as the highlight area,
Only the area including the highlight can be extracted correctly.

Further, the color image conversion device of the second invention independently operates the intensities of the specular reflection component and the diffuse reflection component extracted by the highlight region extraction device by the color changing means. The color changing means is
The intensity is freely changed by moving the pixel value on a straight line representing the specular reflection component or the diffuse reflection component. By using this means, it is possible to generate an image in which the intensities of the specular reflection component and the diffuse reflection component existing in the image are freely changed. The changed color image is output by the color image output means.

[0021]

In the highlight area extracting apparatus for a color image using the reflection component of the first aspect of the present invention, a target scene is input by using an input means such as a camera to obtain a color image. It is converted into a luminance image and pixels with luminance higher than a certain value are extracted as the high luminance area. The obtained high brightness area is divided into areas having similar colors. Regions with similar colors are divided into small regions, the covariance matrix of pixels is calculated for each small region, and the eigenvalues and eigenvectors of the matrix are calculated. The distribution shape when the pixel values of the small area are projected onto the color space is determined from the sizes of the three types of eigenvalues obtained from the covariance matrix. The pixel value distribution in the small area, which is determined to be linear, is adjacent on the image, and
Average the eigenvectors of small areas with similar colors.

The eigenvectors obtained as a result are integrated into several vectors by grouping together the vectors whose three components are close to each other. If there are two types of vectors obtained as a result of the integration, the target area (area of similar color) is considered to include highlights. When the pixel values of the color image are projected onto the color space, the distribution of pixel values is distributed around the straight line having two types of vectors as direction vectors in the color space. Therefore, a cluster centered on this straight line is generated. Since the two types of vectors obtained here are vectors representing the specular reflection component and the diffuse reflection component, the pixels included in the cluster obtained by the vector representing the specular reflection component are extracted as the highlight region.

In the color image conversion apparatus of the second invention, the two types of reflection components, that is, the specular reflection component and the diffuse reflection component are independently controlled, so that the intensity of the specular reflection component and the diffuse reflection component can be independently controlled. To enable.
When the specular reflection component and the diffuse reflection component are separated from the color image including the specular reflection, and the two reflection components are independently increased or decreased, the intensity of the specular reflection component is changed while the diffuse reflection component remains, and conversely, The intensity of the diffuse reflection component can be changed while leaving the specular reflection component. That is, by separating the specular reflection component and the diffuse reflection component and operating each independently, it is possible for an unskilled person to enhance or suppress the specular reflection component and to naturally change the color intensity of the original object. Can be easily realized by the operator.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, an embodiment of a color image highlight area extracting apparatus using a reflection component of the present invention will be described with reference to the drawings. FIG. 1 shows a block diagram of the embodiment. The highlight area extraction device, as shown in FIG.
RGB camera section 101, high brightness area extraction section 107, similar color area extraction section 108, eigenvalue / eigenvector calculation section 10
2, a vector integration unit 109, a reflection component number determination unit 110,
The cluster generation unit 104 and the highlight area extraction unit 106 are included.

A color image is input by the RGB camera unit 101. It is sent to the high brightness area extraction unit 107. The high-brightness area extraction unit 107 converts the color image into a brightness image, extracts a pixel having a brightness higher than the threshold as a high-brightness area, with the minimum value of the brightness at the zero crossing point of the brightness image as a threshold value. . Next, the similar color area extraction unit 1
In 08, the uniform color space L ^{* is set} for each pixel in the high-luminance region ^.
Euclidean distance (color difference) of chromaticity coordinates in u ^* v ^*
Use to get regions of similar color.

In the eigenvalue / eigenvector calculation unit 102,
Regions with similar colors are divided into small regions, the covariance matrix of pixels is calculated for each small region, and the eigenvalues and eigenvectors of the matrix are calculated. The covariance matrix obtained here is composed of 3 rows and 3 columns because each element represents the variance of the RGB components of the small area. Therefore, there are three eigenvalues. Of the three eigenvalues, the number of eigenvalues larger than a certain threshold (0, 1, 2, 3) makes it possible to estimate the shape of the pixel distribution when the small area is projected on the RGB color space.

In view of this, the vector integration unit 109 targets small areas of which the number is 1 (the distribution in the RGB color space is linear) between adjacent areas on the image and having similar color components. Average the vectors. Further, the obtained vectors are averaged when the three components of the vectors are close to each other with the inner product between the vectors as the criterion, and further normalized (the size of the vector is set to 1).

The reflection component number determination unit 110 determines that the region of similar color of interest includes highlights, as compared with the case where there are two types of vectors obtained by the vector integration unit 109. To do. When the pixel values of the color image are projected onto the RGB color space, the vector integration unit 1
Since the vector obtained by 09 represents the direction of the pixel value distribution with respect to the region where the distribution in the RGB color space of each small region is linear, the periphery of the straight line whose direction vector is that The pixel values are distributed in.

The cluster generation unit 104 generates a cluster of pixel values centered on two straight lines. The highlight area extraction unit 106 regards the cluster with the higher brightness of the two clusters obtained by the cluster generation unit 104 as the cluster generated by the vector representing the specular reflection component, and includes it in this cluster. Extracted pixels are extracted as pixels from the highlight area.

When a highlight area is extracted by inputting an image as shown in FIG. 9 (A) with this apparatus, the lower white area is distinguished from the highlight area, and as shown in FIG. 9 (C). The desired highlight area has been extracted.

Here, as a criterion of similarity for extracting a similar color area, a uniform color space L ^* u ^* v ^{* is used.}
, The uniform color space L ^* a ^* b ^* or H
It is possible to use an arbitrary color space such as a VC space.

Next, an embodiment of the color image conversion apparatus of the present invention will be described with reference to the drawings. FIG. 2 shows a block diagram of an embodiment of the color image conversion apparatus. As shown in FIG. 2, the color image conversion device includes a highlight area extraction device 201, a color changing unit 203, and a color image output unit 205. In the following, for simplification, an explanation will be given by taking as an example an RGB space whose axes are red (R), green (G), and blue (B), which are three color components that a color image has in a pixel value space to be used ( (See FIG. 3).

In the highlight area extraction device 201, the equations of the straight lines L _D and L _S representing the reflection components in the RGB space as shown in FIG. 3 are obtained by using the above-mentioned technique. Therefore, the color changing unit 203 obtains an equation of a straight line that passes through that point and is parallel to the straight line representing the specular reflection component or the diffuse reflection component for any point in the RGB space, and determines the pixel value as the one with higher brightness or , The intensity of the specular reflection component or diffuse reflection component is changed by moving to the lower brightness.

When changing the intensity of the specular reflection component in FIG.
The conceptual diagram of is shown. Two straight lines in the RGB space shown in FIG.
Are the direct reflection components that represent the diffuse reflection components in order from the origin.
Line L _D, A straight line L representing the specular reflection component_SIs. Point P (r
_P, G_P, B_P) Is the RGB space at a pixel
Represents a pixel value.

In order to change the intensity of the specular reflection component,
In the RGB space, the equation of a straight line L ₁ that passes through the point P and is parallel to the straight line L _S that represents the specular reflection component is obtained. Straight line L ₁
In the equation, when t is a variable and the direction vector v of the straight line representing the specular reflection component is (v _R , v _G , v _B ),
R = v _R · t + r _P , G = v _G · t + g _P , B = v _B ·
It is represented by t + b _P. In the RGB space, all the components of the direction vector (v _R , v _G , v _B ) of the straight line representing the reflection component are expressed as positive, so that t is a positive direction (a direction approaching white, and H in the figure). _The specular reflection component can be emphasized by increasing in the direction of _P ), and the specular reflection component can be suppressed by decreasing in the negative direction (direction toward black, which indicates the direction of L _{P in} the figure).

Also when changing the intensity of the diffuse reflection component, the point Q is a straight line L _D parallel to the straight line L _D representing the diffuse reflection component, similarly to the operation for changing the intensity of the specular reflection component.
_{On the} upper part of ₂ , move in the positive direction (H _Q direction in the figure) to enhance the diffuse reflection component, and in the negative direction (L _Q direction in the figure).
The diffuse reflection component can be suppressed by moving to.

FIG. 5 shows a conceptual diagram of the result of changing the intensity of the specular reflection component. As shown in FIG. 5A, the input image 6
00 is three kinds of object areas (rectangular area 6
06, triangular area 607, elliptical area 608), and each object area includes a highlight (indicated by reference numeral 609 in the figure).

FIG. 5B shows a pixel value distribution curve 610 on the straight line L _Y in the input image 600. The pixel value distribution curve is the pixel value of coordinates (x _i , L _Y ) in the image,
The horizontal axis is the image width, the vertical axis is the pixel value (the maximum pixel value is 100
Is a normalized value) as a graph on the coordinate axis. Here, x _i represents the coordinate in the horizontal axis direction, and i takes a value from 0 to the horizontal size of the image. It is shown that the pixel value from the diffuse reflection area to the specular reflection area gradually increases.

The solid line in FIG. 5C shows the pixel value distribution curve 611 on the straight line L _{Y of} the image in which the specular reflection component is emphasized. The broken line is the pixel value distribution curve 610 before enhancement shown in FIG. In FIG. 5C, the pixel value is higher in the specular reflection portion and its periphery than in the input image, and the specular reflection component is emphasized.

The solid line in FIG. 5D is the pixel value distribution curve 612 on the straight line L _{Y of} the image in which the specular reflection component is suppressed. The broken line is the pixel value distribution curve 610 before suppression shown in FIG. In FIG. 5D, the pixel value is lower in the specular reflection portion and its periphery than in the input image, and the specular reflection component is suppressed.

FIG. 6 shows a conceptual diagram of a conversion result in which the intensity of the diffuse reflection component is changed. The input image 60 shown in FIG.
0 is the same image as the input image 600 of FIG. 5, and is composed of three kinds of object regions (rectangular region 606, triangular region 607, elliptical region 608) having different reflectances, and each object region is highlighted (in the figure, , 609) are included. FIG. 6B shows a pixel value distribution curve 610 on the straight line L _Y in the input image 600.

The solid line in FIG. 6C shows the pixel value distribution curve 613 on the straight line L _{Y of} the image in which the diffuse reflection component is emphasized. The broken line is the pixel value distribution curve 610 before enhancement shown in FIG. In FIG. 6C, the pixel value of the diffuse reflection portion is uniformly higher than that of the input image, and the diffuse reflection component is emphasized.

The solid line in FIG. 6D is the pixel value distribution curve 614 on the straight line L _{Y of} the image in which the diffuse reflection component is suppressed. The broken line is the pixel value distribution curve 610 before suppression shown in FIG. In FIG. 6D, the pixel value of the diffuse reflection portion is uniformly lower than that of the input image, and the diffuse reflection component is suppressed.

The color image output unit 205 outputs a color image having the pixel value newly obtained by the color changing unit 203. Note that here, as the pixel value space, R
Although the example using the GB space has been described, the Yxy space, L ^* u
It is possible to use any space including the ^* v ^* uniform color space and the HVC space.

[0045]

EFFECTS OF THE INVENTION If there is a region in the image that is not a highlight but has a color similar to the highlight, that is, if there is a region with a high brightness and a color similar to the light source, the region of similar color is When the obtained vectors are integrated, originally, the region does not include the highlight, so that only one type of vector (a vector representing the diffuse reflection component) is extracted.

The present invention pays attention to this point and considers that the highlight is included only when two types of vectors are obtained as a result of vector integration. Therefore, the brightness is high and the color is similar to that of the light source. The area is never extracted as a highlight area. Therefore, according to the present invention, it is possible to correctly extract the highlight region, to correctly estimate the color of the light source, and to estimate the surface reflectance without the influence of the light source.

For an image including highlights, the pixel value distribution in the pixel value space is a straight line representing the specular reflection component,
By taking advantage of the fact that it can be approximated by a straight line representing a diffuse reflection component, the intensity of the specular reflection component and the diffuse reflection component can be changed by moving an arbitrary pixel value in parallel with the specular reflection component or the straight line representing the diffuse reflection component by a certain distance. ， It becomes possible to enhance and suppress natural colors with a sense of reality.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention (claim 1).

FIG. 2 is a block diagram showing a configuration of an embodiment of the present invention (claim 2).

FIG. 3 is a diagram showing a linear approximation by a pixel value distribution and a reflection component in a color image.

FIG. 4 is a diagram showing a concept of changing the intensity of a specular reflection component in a pixel value space.

FIG. 5 is a diagram showing a row of images in which the intensity of the specular reflection component is changed.

FIG. 6 is a diagram showing an example of an image in which the intensity of a diffuse reflection component is changed.

FIG. 7 is a block diagram showing a configuration of a conventional highlight area extraction device.

FIG. 8 is a block diagram showing a configuration of a conventional color image conversion device.

FIG. 9 is a diagram showing an example of erroneously extracting a region other than the highlight region in the highlight region extraction to which the conventional method is applied.

[Explanation of symbols]

 101 RGB camera unit 102 Eigenvalue / eigenvector calculation unit 103 Directional vector extraction unit 104 Cluster generation unit 105 Specular reflection component detection unit 106 Highlight region extraction unit 107 High brightness region extraction unit 108 Similar color region extraction unit 109 Vector integration unit 110 Reflection component Number determination unit 201 Highlight region extraction device 202 Color image input unit 203 Color changing unit 204 Color changing unit 205 Color image output unit 600 Input image 606 Rectangular region 607 Triangular region 608 Elliptical region 609 Highlight 610 Pixel value distribution curve of input image 611 Pixel value distribution curve of specular reflection component emphasized image 612 Pixel value distribution curve of specular reflection component suppressed image 613 Pixel value distribution curve of diffuse reflection component emphasized image 614 Pixel value distribution curve of diffuse reflection component suppressed image

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification number Office reference number FI technical display location H04N 1/46 9068-5C (72) Inventor Shin Kosugi 1-6, Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation

Claims (2)

[Claims] 1. A color image input section for inputting a color image, and a high brightness area for generating a brightness image from the intensities of the three primary color components in each pixel of the image obtained by the color image input section and extracting a high brightness area. Extracting unit, dividing the high-luminance region obtained by the high-luminance region extracting unit into regions having similar colors, and dividing the similar-color region obtained by the similar-color region extracting unit into small regions Then, the eigenvalue / eigenvector calculation unit that calculates the eigenvalues and eigenvectors of the covariance matrix in the pixels of each small region, the vector integration unit that integrates the eigenvectors obtained by the eigenvalue / eigenvector calculation unit into several vectors, and the vector integration unit If the number of vectors obtained from is two, the pixel values of the color image are projected onto a three-dimensional space centered on the three primary color components, and The cluster generation unit that generates a cluster around a straight line whose direction vector is the two types of vectors obtained by the Khuttle integration unit, and the cluster that represents the specular reflection component of the two types of clusters obtained by the cluster generation unit. A highlight area extracting device for a color image using a reflection component, comprising: a highlight area extracting section for extracting included pixels as a highlight area. 2. A color image input section for inputting a color image, and a high brightness area for generating a brightness image from the intensities of the three primary color components in each pixel of the image obtained by the color image input section and extracting a high brightness area. Extracting unit, dividing the high-luminance region obtained by the high-luminance region extracting unit into regions having similar colors, and dividing the similar-color region obtained by the similar-color region extracting unit into small regions Then, the eigenvalue / eigenvector calculation unit that calculates the eigenvalues and eigenvectors of the covariance matrix in the pixels of each small region, the vector integration unit that integrates the eigenvectors obtained by the eigenvalue / eigenvector calculation unit into several vectors, and the vector integration unit If the number of vectors obtained from is two, the pixel values of the color image are projected onto a three-dimensional space centered on the three primary color components, and Based on the two kinds of direction vectors obtained by the vector integration unit, and the cluster generation unit that generates a cluster centered on a straight line whose direction vector is the two kinds of vectors obtained by the Kuttle integration unit, A color changing unit for changing the pixel values of the input image by independently operating the intensities of the specular reflection component and the diffuse reflection component, and a color image output unit for outputting the color image obtained by the color changing unit A color image conversion device characterized by.