JP3381039B2 - Contour extraction method of color image applying fuzzy inference - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention uses fuzzy inference to apply uniform inference to each method and to extract contours of color images by applying fuzzy inference for easy adjustment of parameters. Regarding the method.

[0002]

2. Description of the Related Art As a conventionally known contour extraction method, a contour extraction method (eddge
Tection) and each pixel (each point of an image) are classified according to a pixel value (brightness or hue) and divided into regions (region extraction).

The following four methods have been proposed for the above-mentioned middle contour extraction method. 1. Method using first derivative 2. Method using second derivative 3. Method using average value differentiation 4. Method using template matching

[0004]

The conventional method using the differential in the contour extraction method has a drawback of picking up noise, and there is a problem that the tendency becomes stronger as the order of the differential increases.

Since Labrian is a second derivative,
There is a merit that it is not possible to detect a contour having a linear inclination, and it is more suitable for detecting points, lines, corners, etc. than a contour. Therefore, a method using an average value to remove noise has been proposed.

Proposed as the contour extraction method
~ 4. Although each method is basically based on the use of brightness (grayscale value), there are cases where sufficient area division cannot be performed only with brightness information.

[0007] For example, in the case of an outdoor object and its shadow on a sunny day, there is a problem that the object and the shadow are extracted as their respective areas.

In order to solve the above problems, it has been attempted to make a difference depending on the level of saturation, extract a region by a hue value, or divide a region using color information.

Each of the above methods has its own characteristics, but has the following problems. 1. There is no unity. 2. Each method has its own orientation. 3. Parameter adjustment is difficult.

That is, each of the conventional methods has its own algorithm by its own approach and has no uniformity. Depending on the extraction method, each method has advantages and disadvantages, such as a method suitable only for a specific image and a method that is greatly affected by noise. Also, some methods require adjustment of parameters.

Therefore, even if a person who does not have an in-depth knowledge of image processing tries to use these algorithms, he or she cannot use them without the help of a specialist.

In order to solve such a problem, a method applicable to any image is required.
It is effective to apply the idea of flexible fuzzy theory to this. Fuzzy theory can express adjectives that we use everyday, such as "bright", "dark", and "blue", by means of a member function, and can be considered to be easy to use as an interface.

[0013] Russo shows that contour extraction is possible by simple fuzzy inference, but the inference rule is that if a pixel's eight neighbors are "same" pixel values. , There is no contour. The Rousseau method is a very simple method, but has the following two problems.

One of them is the "direction" of the contour in the inference result.
Is not included. In the extraction method in which the "direction" of the contour cannot be obtained, it may be difficult to connect the contours as lines thereafter. The other is that it is targeted for grayscale images. From these things, we propose to apply a rule considering directionality to both lightness and hue.

[0015]

[Means for solving the problem] That is, as a method of inference,
If we look at the contour part for 3 × 3 9 pixels,
It can be considered that there are basically six types of patterns as shown in FIG. In this figure, pattern (a) is a vertical contour, pattern (b) is a horizontal contour, patterns (c) and (d) are right-downward contours, and patterns (e) and (f) are upper-rightward contours. Is.

The shaded portions and other portions in FIG. 1 indicate that they belong to the same "brightness" (or hue). To determine whether or not the pixel values in the above are "same", a membership function as shown in FIG. 2 is used.

For example, in the case of the pattern (a) shown in FIG. 1, the 3 pixel values in the left 1 column are the same (any mountain in FIG. 2 contains 3 pixel values), and the 3 pixel values in the right 1 column are Is "same" (any mountain in FIG. 2 contains 3 pixel values), and if the two are different, it can be said that there is a vertical contour.

That is, the present invention uses a sample obtained from a sample image.
Membership for fuzzy inference based on sample data
After automatically generating the function, for the brightness value of each pixel of the image, the vertical outline, horizontal outline, downward right outline and upper right
Apply the six rule of six patterns of Ri contour, off
Performed six inference in Ajii inference employs a rule that a maximum value is obtained therein, the pattern of the rule and the direction of the contour, the inference result of the rule and the edge intensity, hue value for each pixel The same reasoning as above is performed for
The contour direction and intensity are calculated for each pixel, and the contour intensity obtained by the inference of the lightness value and the contour intensity obtained by the inference of the hue value are ANDed or ORed, and the pixel value of each pixel is calculated. This is a color image contour extraction method that uses fuzzy inference, which uses saturation as a weight value.

Another aspect of the present invention is to extract a contour from fuzzy inference.
And set the membership function according to the characteristics of the image
It is possible to, and than also you contour extraction function variable, the lower part of the saturation priority to brightness information, in the portion having a high saturation is obtained by giving priority to color information.

[0020] Next, to select a certain sample image, tracing the location of the contour to be extracted in the sample image, and automatically generate the membership functions for fuzzy inference on the basis of the sample data of the corresponding contour, the contour of the similar image Color images can also be extracted by applying fuzzy inference, which is characterized by extraction.

It is standard that the color information in the above is represented by three colors of red (R), green (G) and blue (B) in a digital image. However, an example of the conversion method for the three attributes of hue, lightness, and saturation is as follows.

First, the brightness V can be calculated from the R, G, and B as follows.

[0023]

[Equation 1]

Next, the color difference signals C ₁ and C ₂ are expressed by the following equation.

[0025]

[Equation 2]

[Equation 3]

## EQU2 ## The hue H and the saturation S are calculated by the equation 2 and the values S and S are obtained from the following equations.

[0027]

[Equation 4]

[Equation 5]

An AND operation (or OR of the contour strength obtained by the above-mentioned brightness and the contour strength obtained by the hue)
Calculation). At this time, the saturation (sat) of each pixel value
Are weighted as follows.

[0029]

[Equation 6]

The evaluation of the algorithm in the contour extraction depends on the subjectivity of most people. This is because the target image is usually natural and there is no clear definition of what the resulting contour should be. However, since it is possible to perform a quantitative evaluation by using a sample image artificially created, the evaluation of the present invention will be performed below. The following formula is used for quantitative evaluation.

[0031]

[Equation 7]

[Equation 8]

In the above description, R ₁ is a numerical value representing how much is actually extracted with respect to the ideal point of the defined contour. Further, R _N represents the ratio of the extracted points other than the ideal points. It refers to a point included in the candidate group as defined in the contour from the ideal point in R _N.

The samples used for the quantitative evaluation are the four images shown in FIG.

Quantitative evaluation of the contour extraction method was carried out using FIG. 3 described above. However, a significant difference from the existing contour extraction method occurred in the following cases.

1. When the contour contrast (difference between pixel values of two regions) is small. The contour extraction method applying the fuzzy theory gave good results regardless of the contrast of the contour. It can be seen that the noise generated increases.

2. The effect of noise in the image. Rousseau's method is greatly affected especially for images containing binary noise and Gaussian noise. This is because Rousseau's method uses the rule that "if all the surrounding pixels are the same, it is not a contour", and if any one of the surrounding pixels is different, it is judged as a contour. Because it is done.

As a qualitative evaluation, contour extraction using a natural image was performed. The results obtained by using the two types of images (a) and (b) of FIG. 4 are shown in FIG. The result of using the Prewitt differential operator on the image of FIG. 4 is shown in FIG.
5 (a) and 5 (c), and the results of the fuzzy inference according to the present invention are shown in FIGS. 5 (b) and 5 (d).

Therefore, comparing (a) and (b) of FIG.
Although similar results have been obtained by the method using the Prewitt differential operator and the fuzzy inference, a clear difference can be seen by comparing the two in FIGS. 5C and 5D.

The biggest difference is that the method using the differential operator in (c) extracts not only the contour of the object but also the shadow of the object. This is because the method of Prewitt's differential operator only deals with lightness information. On the other hand, the method based on fuzzy inference according to the present invention uses hue information and saturation information in addition to the lightness information, so that the result as shown in (d) can be obtained without extracting the shadow of the object. As can be seen from the results of FIGS. 5C and 5D, it is very important to use the color information for contour extraction in order to improve the accuracy.

[0040]

EXAMPLE An example of carrying out the present invention by extracting an arbitrary sample image will be described with reference to FIG. First, a sample image for tracing the contour is selected from the images of the target type (a).

Next, with respect to this sample image, a portion where the user thinks the contour is traced (b). By this work, it is input to the information system about the contour periphery.

A member function for fuzzy inference is automatically generated based on the sample data of the contour input by the tracing (c). Therefore, the rules for fuzzy reasoning are automatically generated.

According to the generated inference rule, the contours of the sample image used above and the same type of image can be extracted (d).

As described above, by tracing by the user, the membership function is automatically generated based on the input sample data regarding the contour. The algorithm is shown in FIG.

The sample data relating to the contour is handled in the form of a 3 × 3 matrix as shown in FIG. Numerical values in the matrix are lightness values or hue values, and the following operations are performed twice for lightness and hue. Therefore, an inference rule for lightness and an inference rule for hue are generated respectively. First, 3 × 3 sample data is classified into classes based on the similarity of the patterns. At this time,
The sample data without features, that is, the nine values that are almost the same and are not obviously the data around the contour are excluded.

By performing the classification as described above, the data regarding the contour is classified according to its directionality and density difference.

Therefore, in one class, data about contours having a specific directionality and having a specific density difference is collected.

Next, the member function is set for each pixel of 3 × 3 data of each class as shown in FIG. For this, the sample data in each pixel is used individually. For example, there are m samples for the class n, and the sample data for the pixel at the upper left position is P ₁ , P ₂ , ..., P _m .
At this time, the membership function for the upper left pixel becomes a triangular function having the average value of the sample data as the vertex and the result of multiplying the variance by the coefficient α as the slope (FIG. 8).

According to the above method, it is possible to automatically generate a fuzzy inference rule for contour extraction which matches the characteristics of the image based on the contour information traced and input by the user. The inference rule thus generated can be applied to an image having the same characteristics as the image used as the sample image.

First, it is assumed that the number of sample data relating to the lightness is S _VAL as a result of excluding those without features. Then, the number of rules generated using those sample data is set as R _VAL . These rules RULE ₁ , RUL
When the inference results by E ₂ , ..., RULE _RVAL are E ₁ , E ₂ , ..., E _RVAL respectively, the final inference result EDGE _VAL using the lightness information is calculated as follows.

[0051]

[Equation 9]

In this way, the reason why only the maximum value is taken from the inference result by the rule concerning the lightness is that each inference rule is to extract the contour of different directionality or different density difference, and one pixel is This is because only one directionality or one density difference contour can be formed. Similarly, the number of sample data relating to hue is S _HUE , the number of rules generated based on them is R _HUE , and the final inference result by hue is EDGE _HUE .

At this time, the final inference result EDGE _ALL considering both lightness and hue is obtained by weighting as follows using the number of samples relating to lightness and hue.

[0054]

[Equation 10]

Such weighting is performed in S if the lightness feature of the contour information input by the user is large.
_VAL increases, because S _HUE increases the larger the characteristics of the hue.

That is, this algorithm judges whether the contour is due to the difference in brightness or the difference in density, and if it is due to the difference in brightness, the brightness information is preferentially used for the contour extraction,
If the difference is due to the hue difference, the hue information is preferentially used for contour extraction.

The contour is traced by using FIG. 3, and the extraction force by the directionality of the contour is examined by using the rule obtained thereby. As a result, for four types of contours of horizontal, vertical, diagonal and curved, And good results were obtained. However, when a ramp-shaped edge was used, some noise was observed.

Further, as a result of checking the extraction rate when the contrast of the contour of the image for which contour extraction is actually performed (taken in 64 steps from 0 to 63) is different, for example, the contour having the contrast of 30 is traced and inferred. When the rule was generated, it was possible to sufficiently extract contours with a contrast of 34, but when the contrast was 35 or higher, the extraction rate of the contours began to drop sharply, and almost anything with a contrast of 50 or higher was extracted. I got the result that is not done.

This means that if a rule with a low contrast is traced to generate a rule, the rule does not extract a contour with a high contrast, but if a rule is traced with a contour with a high contrast to generate a rule, It shows that a small contour is not extracted. After all, this means that the inference rules that match the characteristics of the traced image are generated.

The sample image containing noise is traced, the contour of the original sample image is extracted using the inference rule generated as a result, and the extraction power for the image containing noise is examined. As a result, the same image is used. However, it was confirmed that the contour extraction rate decreased as the amount of noise increased.

It is considered that the influence of noise is that when noise exists around the contour, information including it is input as contour information and used for generation of the inference rule. When a person looks at an image with his or her own eyes, it is possible to determine the contour by grasping the entire image even if it contains some noise, but it is difficult to give such a capability to the system. As for, there are still problems.

The result obtained by tracing only a specific object in the image of FIG. 4 is shown in FIG.
Shown in. It can be seen from FIG. 9 that only the red object can be extracted by using the inference rule generated when the red object is traced, and only the yellow object can be extracted by tracing the yellow object. That is.

That is, by tracing only a specific contour from a certain image and generating a rule, an inference rule for such a special request can be obtained.
The work of extracting only a specific contour in this way cannot be performed by an existing method, and is a part that has been performed as a process after the contour extraction until now. However, according to the present invention, since the user traces what he / she thinks of a contour and inputs it to the system, it is possible to let the system learn information about a specific contour.

[0064]

According to the present invention, the inference rule for contour extraction can be automatically generated, so that it has the following characteristics.

1. By the simple operation of tracing the contour of the sample image, fuzzy inference rules matching the characteristics of the image are automatically generated.

2. A fuzzy inference rule can be generated so that only a specific contour desired by the user (for example, an object contour of a specific color) can be extracted.

3. Since the user has a system having a simple interface that only traces the contour of the sample image, it can be easily used even by a person who is not an image processing expert.

Further, according to the present invention, by adjusting the membership function, it is possible to extract a flexible contour in accordance with the characteristics of the image.

As described above, no contour extraction system intended for use by a user who does not have exclusive knowledge of image processing has been proposed so far, but this is an improvement.

[Brief description of drawings]

1 (a), (b), (c), (d), (e),
(F) The figure which shows the outline pattern of the Example of this invention.

FIG. 2 is a diagram of a membership function also relating to pixel values.

3A, 3B, 3C, and 3D are sample images.

4A and 4B are also original images.

FIG. 5 (a), (b), (c), (d) similarly extracted images.

6A, 6B, 6C, and 6D are explanatory views of the same automatic generation method.

FIG. 7 is an explanatory diagram of an algorithm of automatic generation.

FIG. 8 is an explanatory diagram of setting of a member function.

9A and 9B are views obtained by tracing a specific object in the image of FIG.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-3-268072 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 7/60 250 JISST file (JOIS)

Claims (3)

(57) [Claims] 1. To sample data obtained from a sample image
After automatically generating <br/> membership functions for fuzzy inference based, vertical wheel lightness value of each pixel of the image
Contour, horizontal contour, downward right contour, and upper right contour
Apply 6 rules from the pattern, 6 by fuzzy reasoning
One of performed inference employs a rule that a maximum value is obtained therein, the pattern of the rule and the direction of the contour, the inference result of the <br/> rules and edge intensity, hue value for each pixel For the above, the same inference as described above is performed, the contour direction and intensity are calculated for each pixel, and the AND of the contour intensity obtained by the inference of the lightness value and the contour intensity obtained by the inference of the hue value, or While performing the OR operation,
A color image contour extraction method using fuzzy inference, which uses the saturation of each pixel value as a weight value. 2. A fuzzy inference can be used for contour extraction, and a membership function can be set according to the characteristics of the image.
Therefore , the contour extracting function is made variable so that the contour extracting method of the color image applying fuzzy inference according to claim 1. 3. Lightness information is prioritized in a low saturation portion,
2. A color image contour extraction method applying fuzzy inference according to claim 1, wherein hue information is prioritized in a high saturation portion.