JPH041863A - Method and device for detecting mouth split line - Google Patents

Abstract

PURPOSE:To detect a mouth split line with the simple processing and in a small number of parameters by detecting a picture element darker than the upper and lower picture elements of an image, searching another edge line out of the detected edge lines in the linear direction, and connecting the corresponding edge lines if exist to each other in order to detect the mouth split line out of the image. CONSTITUTION:A face image storage means 10 is provided to store an image obtained by photographing a normal human face together with a mouth part image segmenting means 20 which segments only the mouth part out of the image, and a V-shape edge detection means 30 which detects a picture element darker than the upper and lower picture elements of a gradation image. Furthermore a right/left mouth split line connection means 70 is added to connect the segment connected to the right side of the mouth to the segment connected to the left side of the mouth together with a split line image storage means 80, etc. If the similar connection lines are searched in the direction of an axis (x), they are judged as the mouth split lines and connected together. If such connection lines are not searched, the existence of a mouth split line is judged and the search of the connection lines is finished. Thus the mouth split line is detected with the simple processing and in a small number of parameters.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a technique for detecting a cleft line, which is the region where the upper and lower lips meet, from an image of a normal human face.

[Conventional technology]

As an example of a conventional method for detecting cleft lines from normal human face images, the document [R. Buhr, "ENTWICKLUN"]
G EINESADAPTIVEN EXTRAKTI
ONS-UND KLASSI FIKATIONS-5
YSTEMS ZURERKENNUNG VON B
ILDERN (GESICHTER) ``Doctor of Science dissertation for biology research program at Johannes Gutenberg University,
(1984)].

As shown in FIG. 2, it is assumed that an image A (x, y) of the eye area 201 has already been cut out from the face image using a certain method. Also, the original coordinates of the image are X.

y, the straight line including the line segment connecting the pupils is the y' axis of the face, and the bisector of that line segment is the y' axis.

A ΔG filter with relatively large σ in this day region 201 (Reference [D. Marr, “Vision, W.
,H.

Free+an and Co11pany+ (1
9B2) ] ) to obtain image B(x,y).

The ΔG filter is expressed by the following formula.

Figure 3 shows the one-dimensional case of the two functions (see above, Ma
rr+ “Vision + 5.4 page, 2-9
(quoted from the figure).

There is no noticeable difference in color between the lip area and the surrounding skin, so
Edge strength is not very strong. For this reason, it is difficult to distinguish lips based on edge intensity. If σ is made small, many unnecessary and complicated contour lines appear, making subsequent processing difficult.

Next, a low-pass filter is similarly applied, and the image C(x,
y) is obtained. This image C (x, y) is a blurred version of the original image, and corresponds to the approximate brightness around the image A (x, y). At each pixel, B (x, y) becomes C (x, y)
Divide by, image D (x.

y) is obtained. This makes it possible to emphasize the dark region near the 0 fissure. This image D (x, y) is binarized using an appropriate threshold to create a contour image E (x, y).

y). The darkness value is determined using the percentile method. The percentile method will be explained with reference to FIG. The percentile method is a method of creating a density distribution histogram in a target image and determining a threshold value 401 that divides the area of the histogram by a percent.

In the image E(x, y), as shown in Figure 5, an additive projection is taken in the y-axis direction, an appropriate threshold is determined, and the number of peaks exceeding the threshold is 3 (one cleft upper lip, one cleft lower lip). ) If they exist close to each other and the width of the peak with the maximum area is equal to or greater than the set value, it is considered a zero fissure, and the center coordinate y□ of the zero fissure is determined.

The X coordinate value xws is the face coordinate axis X/, L,
Take it so that it becomes 0. If this condition is not met, the threshold value is lowered step by step. The initial darkness value is given as a value proportional to the maximum peak value.

A homemade line is extracted from the original image A (x, y) based on the zero fissure center coordinates (X□, yxs). First (x, 4s,
In the vicinity of V□), the average brightness value m M S
seek. It is assumed that the 0-fiber pixel has a value similar to this value and is the darkest compared to neighboring pixels, and curve tracing is performed left and right.

First, it is assumed that certain coordinates (xa, ya) are already known to be a homemade point. At this time (XA-1, 7a
-to), (XA-1, 3'A-9), ・”
Using (XA-In'j a++o) as a candidate, it is obtained as a homemade point in the next process.

1) Darkest candidate point (xA-4, yA, wa) (where n=
-10, -..., 10), the average value m0 in the vicinity of the pixel is determined.

2) Find the standard deviation of brightness at all candidate points and calculate it as m. Calculate the value σ, normalized by .

3) When m is less than or equal to m□, set it as a homemade point and continue tracking.

4) Otherwise, if σ9 is larger than 1/3, consider that the 0-fissure is still continuing, and continue to track it as a self-made point.

5) If none of the conditions apply, consider that the homemade end has been reached and stop tracking.

In this way, a homemade wire can be obtained.

However, it can be seen that there are many situations in which it is necessary to set an appropriate threshold value. Furthermore, the vicinity of the pixel must be appropriately determined each time. In order to be able to extract zero fissures from various images through such processing, it is difficult to adjust parameters and it also takes processing time.

[Problem to be solved by the invention]

In the conventional method, since an operator applied to general images is used, it is necessary to perform complicated processing and set many parameters.

An object of the present invention is to reduce processing and parameters by applying an operator according to the shape of the mouth area, and to provide a simple and effective homemade line detection method and device.

[Means to solve the problem]

The homemade line detection method of the present invention includes detecting a pixel darker than any of the upper and lower pixels of this image in an image of a person's normal face, searching for another edge line in the line direction from the detected edge line, The method is characterized in that if corresponding edge lines exist, the homemade lines are detected from the image by connecting them.

The mouth-shaped line detection device of the present invention receives an image of a person's normal face as input and detects a pixel darker than any of the upper and lower pixels of the image, and an output from the V-shaped edge detection device. edge image storage means for storing edge images stored in the edge image storage means; edge line search means for searching for two connected edge lines from the edge images stored in the edge image storage means; The present invention is characterized by comprising an edge line search/connection means for searching for lines and connecting them if they exist within the search range, and a homemade line extraction means for selecting a homemade line from the connected edge lines.

[Effect] The principle of the present invention will be explained below.

The mouth consists of an upper lip and a lower lip. Upper and lower lips are y
It has a steep convex shape in the axial direction compared to the surrounding area, and a gentle convex shape in the X-axis direction. If you look closely, you can see that each one has fine wrinkles running vertically.

The homemade version is the part where the upper and lower lips meet, and assuming the lips are smooth enough, the 0 cleft position is the darkest in the y-axis direction (
, before and after is brighter than that. Furthermore, the wrinkles running in the X-axis direction on the lips are not as deep as those made in-house, and there is no correlation between adjacent wrinkles. Also, in other parts around the mouth
There is no structure in the axial direction.

For these reasons, a homemade wire can be easily obtained by the following process. First, assume that there is an image in which the area around the mouth has already been extracted using the conventional technology.

1) A 7-character edge operator that is sensitive only in the y-axis direction is applied to the original image, and an edge exists. Don't do it 2
Value.

2) Assume that the longest one connected in the x-axis direction is part of the homemade wire.

3) Search for a similar connection line in the x-axis direction, and if it exists, consider it to be a homemade line and connect it.

If not, it is determined that the homemade end has been reached and the search ends there.

The parameters here are only the shape of the edge operator and the range to be searched, which are much smaller than in the conventional method. The processing procedure is also very simplified.

[Example〕 Examples of the present invention will be shown below using the drawings.

FIG. 1 is a block diagram showing an embodiment of a mouth line detection device.

This zero fissure line detection device includes a face image storage means 10 for storing an image of a person's normal face, a mouth part image cutting means 20 for cutting out only the mouth part in the image, and a top and bottom pixel of a grayscale image. A V-shaped edge detection means 30 that detects dark pixels, an edge image storage means 40 that stores an edge image output from this V-shaped edge detection means, and an edge image stored in this edge image storage means. Partial opening type line detection means 50 for detecting connected edges
Then, a line segment search/connection means 60 and 61 searches for another partially homemade line on the left and right side in the direction of the partially homemade line, and connects it if it exists in the search range, and a line segment connected on the left side and the right side. Left and right homemade line connecting means 70 that connects the connected line segments to create a homemade line, and homemade line image storage means 80 that stores the homemade line.
It is equipped with

Face images (shaded images) are stored in a face image storage means 10, and only the mouth part in the grayscale image is extracted from this face image by a mouth part image cutting means 20. This can be realized by conventional methods (for example, the above-mentioned document [R, Bu
hr, ”ENTWICKLUNG EFNESAD
APTIVEN EXTRAKTIONS-UND K
LASSIFIKATIONSSYSTEMS ZUR
ERKENNUNG VON BILDERN (GE
SICI(TER)'.

(described in detail in Johannes Gutenberg University's Doctor of Science thesis for Biology Research Program, (1984)).

A V-shaped edge detection means 30 for detecting V-shaped edges in the y-axis direction is applied to the cut out grayscale image A(x, y) to obtain an edge image F (x, y).

For example, a simple operator such as the following can be considered.

F (x, y) Partial (A (x, y+1) - A (x, y)) Xθ (
A (x, y-1) -A (x, y)) (2
) Here, θ() is a step function, and its graph is
Equation (2) shown in the figure shows that when the image is cut by a straight line parallel to the y-axis as shown in Figure 7, the density of the pixel of interest is r valley 1.
It is 1 at positions 701, 702, and 703 where , and 0 otherwise. As shown in FIG. 8, the image obtained by applying this operator includes a self-made line element AO-A8 extending in the X'-axis direction and a relatively isolated noise BO caused by lip wrinkles, etc. ~B12. This edge image F (x, y) is
, the line segment search/connection means 60, 61, and the left and right homemade line connection means 70 are stored in the edge image storage means 40 for reference.

Line segments in the edge image F (x, y) will be explained using FIG. 9. When an edge point exists in region 901,
If other edge points exist in the surrounding areas 902 to 907, the 0-section 0-break line detection means 50 determines that they are connected, and detects the longest one among the line segments connected in this manner. Look for it and assume it's part of the homemade wire. In FIG. 8, line segment A7 is determined to be part of the homemade line.

Such a determination result is sent as edge connection information to the line segment search/connection means 60 and 61.

From the extracted line segment A7, line segment search/connection means 60°61
Search for items connected to the left and right sides, respectively. The search is performed in a range weighted in the direction of the line segment. If a connecting line segment is found, the search is performed further from the end of the line segment. If not, it is determined that the end point of the homemade line has been reached, and the search is terminated.

In the example of FIG. 8, when searching in the right direction from line segment A7, line segment A8 is found nearby, so it is connected. Since there is no edge close to the right of line segment A8, the search is terminated. To the left from line segment A7, line segment A6. A5
．． A4. A3. A2° is connected to A1, and the search ends. The search results are sent to the left and right homemade line connecting means 70 as edge information.

Finally, the left and right homemade lines A8゜A6-A centering on line segment A7
5-A4-A3-A2-AI with left and right homemade wire connecting means 70
A self-made line is obtained, and its image is sent to the 0-fiber line image storage means 80.

As mentioned in the section on effects, the only parameters are the size of the operator and the line segment search range, and the processing itself is extremely simplified.

Instead of the operator shown in equation (2), any operator may be used as long as it detects that the profile in the direction approximately parallel to the facial coordinate axis y' forms a figure 7 shape. In addition, the line segment search range may be one that is weighted in the line direction of the partial mouth type line.

In the embodiment, the longest line segment 1 was used for detecting the partial opening type line. It is sufficient to apply the connecting means one after another and finally perform a zero-fissure determination.This determination can be performed, for example, by the method described in the prior art.

〔Effect of the invention〕

According to the present invention, which performs processing that takes into account the structure around a person's mouth, homemade lines can be detected with simpler processing and fewer parameters than conventional methods.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an embodiment of the zero fissure line detection device of the present invention, FIG. 2 is an explanatory diagram of a face image to be processed, and FIG.
A diagram showing a graph of the one-dimensional case of the ΔG operator, FIG. 4 is an explanatory diagram of the percentile method, and FIG. 5 is a diagram showing the contour line position added and projected onto the y-axis in the prior art. FIG. 6 is a diagram showing a graph of a step function, FIG. 7 is an explanatory diagram of the result when the 7-character edge detection operator shown in the embodiment of the present invention is applied, and FIG. 8 is a diagram showing the line segment in the embodiment. search·
FIG. 9, an explanatory diagram of the joining process, is a diagram showing a connection determination area between edge points in the embodiment. 10...Face image storage means 20...Mouth part image cutting means 30...
V-shaped edge detection means 40...Edge image storage means 50...Partial 0-fissure line detection means 60, 61...Line segment search/connection means 70...Left and right homemade line connection means 80... Cleft line image storage means

Claims (2)

[Claims] (1) In an image of a person's normal face, detect a pixel that is darker than any of the upper and lower pixels of this image, search for another edge line in the line direction from the detected edge line, and find the corresponding edge line. A method for detecting a cleft line, characterized in that the cleft line is detected from the image by connecting the cleft lines. (2) V-shaped edge detection means that receives an image of a person's normal face as input and detects a pixel darker than either the upper or lower pixel of the image; and stores the edge image output from this V-shaped edge detection means. an edge image storage means; an edge line search means for searching connected edge lines from edge images stored in the edge image storage means; searching for another edge line in the direction of the edge line; A cleft line detection device comprising: an edge line search/connection means for connecting edge lines if they exist; and a cleft line extraction means for selecting a cleft line from the connected edge lines.