JP2671556B2 - Cleft line detection method and device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a technique for detecting a cleft line showing a cleft, which is a site where an upper lip and a lower lip are combined, from a person's normal face image.

[Conventional technology]

As an example of a conventional method for detecting a cleft line from a person's normal face image, see [R. Buhr, “ENTWICKLUNG EINES ADAPTIVEN
EXTRAKTIONS-UND KLASSIFIKATIONS-SYSTEMS ZUR ERK
ENNUNG VON BILDERN (GESICHTER) ", Johannes Gutenberg University Doctoral Thesis Request for Biological Research, (19
84)].

As shown in FIG. 2, first, it is assumed that the image A (x, y) of the mouth region 201 is cut out from the face image by an existing method. The original coordinates of the image are x, y, the straight line including the line segment connecting the pupils is the x'axis of the face, and the bisector of the line segment is the y'axis.

A ΔG filter having a relatively large σ in the mouth region 201 (reference document [D. Marr, “Vision”, WH Freeman and Com
pany, (1982)]) to obtain image B (x, y). Δ
The G filter is expressed by the following equation.

The one-dimensional case of this function is shown in Fig. 3 (D.Mar above).
r, "Vision", page 54, quoted from Figure 2-9).

The lip strength is not so different from the skin around it, so the edge strength is not very strong. Therefore, it is difficult to distinguish the lips by the edge strength. If σ is made small, many unnecessary and complicated contour lines appear, which makes subsequent processing difficult.

Next, a low pass filter is applied in the same way, and the image C (x,
get y). The image C (x, y) is a blurred version of the original image and corresponds to the general brightness around the image A (x, y). At each pixel, B (x, y) is divided by C (x, y) to obtain an image D (x, y). As a result, the vicinity of the cleft, which is a dark area, can be emphasized. The contour line image E (x, y) is generated by binarizing the image D (x, y) with an appropriate threshold value. The threshold is obtained by using the percentile method. The percentile method will be described with reference to FIG. The percentile method is a method of creating a density distribution histogram in a target image and determining a threshold value 401 that divides the area of the histogram by a percentage.

In image E (x, y), as shown in FIG. 5, additional projection is performed in the y-axis direction, an appropriate threshold value is set, and the number of peaks exceeding that is three (upper lip, cleft lip, lower lip). ) If they are present close to each other and the width of the peak of the maximum area is equal to or larger than the set value, it is regarded as a cleft, and the center coordinate y _{MS of the} cleft is determined.
The x coordinate value x _MS is set to be 0 on the face coordinate axis x′-y ′. If this condition is not satisfied, the threshold value is lowered stepwise and the check is continued. The first threshold is given as a value proportional to the maximum peak value.

Based on the cleft center coordinates (x _MS , y _MS ), the original image A (x,
Extract the cleft line from y). First, in the vicinity of (x _MS , y _MS ), the average brightness m _MS is obtained. The pixel of the cleft has a value similar to this value, and the curve is traced to the left and right on the assumption that it is the darkest compared to the neighboring pixels.

First, suppose that a certain coordinate (x _A , y _A ) is already known as a point on the cleft. At this time (x _A-1 , y _A-10 ),
(X _A-1 , y _A-9 ), ..., (x _A-1 , y _{A + 10} ) are obtained as candidates, and are obtained as cleft points in the next process.

1) The darkest candidate point (x _A-1 , y _{A + n} ) (where n = −10, ..., 1
Average value m _{D in the} vicinity of the pixel
Ask for.

2) The standard deviation of brightness is obtained at all candidate points, and the value σ _N is calculated by normalizing it with m _MS .

3) When m _D is less than or equal to m _MS , let it be the point on the cleft,
Continue tracking further.

4) Elsewhere if σ _N is greater than 1/3, consider that the cleft is still going on and continue tracking it as a point on the cleft.

5) If none of the above is true, it is considered that the cleft edge has been reached, and the tracking is terminated.

 Thereby, a cleft can be obtained.

However, it can be seen that there are many occasions when an appropriate threshold must be set. Also, the neighborhood of the pixel must be appropriately determined each time. In order to be able to extract clefts from various images by such processing, adjustment of parameters is troublesome, and further processing time is required.

[Problems to be solved by the invention]

Since the conventional method uses an operator applied to a general image, it is necessary to set complicated parameters and a lot of parameters.

It is an object of the present invention to provide a simple and effective cleft tear detection method and apparatus by reducing the processing and parameters by applying an operator according to the shape around the mouth.

[Means for solving the problem]

The cleft line detection method of the present invention detects a darker pixel than any of the upper and lower pixels of an image of a normal face of a person and searches another edge line in a line direction from the detected edge line. If there is a corresponding edge line, the cleft line is detected from the image by connecting them.

The cleft detection device of the present invention receives a picture of a normal face of a person as an input, and V-shaped edge detection means for detecting a pixel darker than both upper and lower pixels of the image, and outputs from the V-shaped edge detection means. Edge image storage means for storing the edge image to be stored, edge line search means for searching the connected edge line from the edge images stored in the edge image storage means, and another edge line in the edge line direction. And edge line searching / connecting means for connecting them if present in the search range, and cleft line extracting means for selecting a cleft line from the connected edge lines.

[Action]

 The principle of the present invention will be described below.

The mouth consists of an upper lip and a lower lip. The upper lip and the lower lip have a steep convex shape in the y-axis direction and a gentle convex shape in the x-axis direction as compared with the periphery. Also, in detail, each has fine wrinkles running vertically. The cleft is where the upper and lower lips meet, and if the lips are smooth enough, y
The cleft position is the darkest in the axial direction, and the areas before and after it are brighter than that. Further, the wrinkles running in the x-axis direction on the lips are not deeper than those of the cleft, and there is no correlation between adjacent wrinkles. Further, there is no structure in the x-axis direction in other parts around the mouth.

From these things, a cleft can be easily obtained by the following process. First, it is assumed that there is an image in which the periphery of the mouth has already been cut out by the conventional technique.

1) Apply a V-shaped edge operator that is sensitive only in the y-axis direction to the original image, and if there are edges, 2
Value.

2) It is assumed that the longest one connected in the x-axis direction is a part of the cleft line.

3) Search for a similar connecting line in the x-axis direction, and if it exists, consider it as a cleft and connect it. If not, it is judged that the end of the cleft has been reached, and the search ends there.

Here, the only parameters are the shape of the edge operator and the range to be searched, which is much smaller than the conventional method. The processing procedure is also very simplified.

〔Example〕

Examples of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the cleft tear detection device.

This cleft detection device includes a face image storage unit 10 that stores an image of a normal face of a person, a mouth portion image cutout unit 20 that cuts out only the mouth portion in the image, and upper and lower pixels of a grayscale image. A V-shaped edge detecting means 30 for detecting dark pixels, an edge image storing means 40 for storing an edge image output from the V-shaped edge detecting means, and an edge image stored in the edge image storing means. Partial cleft line detecting means 50 for detecting the connected edges, and search for another partial cleft line on the left and right sides in the partial cleft line direction,
Left and right cleft line connecting means for connecting the line segment search / connection means 60, 61 for connecting the line segments connected to the left side and the line segment connected to the right side to form a cleft line if they exist in the search range 70 and a cleft line image storage means 80 for storing the cleft line.

A face image (grayscale image) is stored in the face image storage means 10, and only the mouth portion in the grayscale image is cut out from this face image by the mouth part image cutout means 20. This can be achieved by the conventional method (for example, the above-mentioned document [R. Buhr, “E
NTWICKLUNG EINES ADAPTIVEN EXTRAKTIONS-UND KLASSI
FIKATIONS-SYSTEMS ZUR ERKENNUNG VON BILDERN (GESI
CHTER) ", Johannes Gutenberg University Ph.D. Ph.D. Request for Biological Research, (1984)].

V-shaped edge detecting means 30 for detecting a V-shaped edge in the y-axis direction is applied to the cut-out grayscale image A (x, y) to obtain an edge image F (x, y). This can be a simple operator, for example:

F (x, y) = θ {A (x, y + 1) −A (x, y)} × θ {A (x, y−1) −A (x, y)} (2) where θ { } Is the step function, and the graph is
Shown in the figure. Equation (2) shows that when the image is cut by a straight line parallel to the y-axis as shown in FIG. 7, the density of the pixel of interest is “valley”.
It is 1 at the positions 701, 702, and 703 that are set to 0, and 0 otherwise. The image obtained by applying this operator is, as shown in FIG. 8, the elements A0 to A8 of the cleft line extending in the x'axis direction and the relatively isolated noise caused by wrinkles on the lips. It consists of B0 to B12. This edge image F (x, y)
Is a partial cleft line detecting means 50, line segment searching / connecting means 60, 6
1. Stored in the edge image storage means 40 for reference by the left and right cleft palate connecting means 70.

The line segment in the edge image F (x, y) will be described with reference to FIG. When an edge point exists in the area 901 and other edge points exist in the surrounding areas 902 to 907, it is assumed that they are connected. The partial cleft line detecting means 50 searches for the longest line segment connected by such a relationship and assumes that it is a part of the cleft line. In FIG. 8, the line segment A7 is determined to be a part of the cleft line. Such a determination result is sent to the line segment search / connection means 60, 61 as edge connection information.

From the extracted line segment A7, line segment search / connection means 60 and 61 are searched for those connected to the left side and the right side, respectively.
The search is performed on the weighted range in the direction of the line segment. If a connecting line segment is found, further search is performed from the end of the line segment. Otherwise, it is determined that the end point of the cleft line has been reached, and the search ends.

In the example of FIG. 8, when searching in the right direction from the line segment A7, since the line segment A8 exists nearby, it is continued.
Since there is no edge near the right of the line segment A8, the search ends. Line segment A6, A5, A4, A3, A2, A in order from the line segment A7 to the left
The search is ended by connecting to 1. The result of the search is sent to the left and right cleft palate connecting means 70 as edge connection information.

Finally, the left and right cleft lines A8, A6-A5-A4 centered on the line segment A7
-A3-A2-A1 are connected by the left and right cleft palate connecting means 70 to obtain a cleft palate, and the image thereof is sent to the cleft palate image storage means 80.

As described in the action section, the parameters are only the size of the operator and the line segment search range, and the processing itself is very simplified.

The operator may use any operator as long as it can detect that the profile in the direction substantially parallel to the coordinate axis y ′ of the face is V-shaped instead of the one shown in the equation (2). . Further, the line segment search range may be weighted in the line direction of the partial cleft line.

The partial cleft line is also detected as "the longest line segment" in the embodiment, but it does not affect the present invention even if it is not so. For example, the line segment searching / connecting means may be sequentially applied to each edge line segment, and the cleft determination may be finally performed. This determination can be performed, for example, by the method described in the related art.

〔The invention's effect〕

According to the present invention, which performs processing in consideration of the structure around the mouth of a person, it is possible to detect a cleft line with simpler processing and a smaller number of parameters than in the past.

[Brief description of the drawings]

FIG. 1 is a diagram showing an embodiment of a cleft palate detection device of the present invention, FIG. 2 is an explanatory diagram of a face image to be processed, and FIG. 3 is a graph of a ΔG operator in a one-dimensional case. FIG. 4 is an explanatory view of the percentile method, FIG. 5 is a view showing a contour position on the y-axis in the prior art, and FIG. 6 is a graph showing a step function. FIG. 7 is an explanatory diagram of a result when the V-shaped edge detection operator shown in the embodiment of the present invention is applied, and FIG. 8 is an explanatory diagram of line segment search / join processing in the embodiment, FIG. FIG. 6 is a diagram showing a connection determination region between edge points in the example. 10: Face image storage means 20: Mouth portion image cut-out means 30: V-shaped edge detection means 40: Edge image storage means 50: Partial cleft line detection means 60, 61: Line segment search / connection means 70 …… Left and right cleft line connecting means 80 …… Cleft line image storage means

Claims (2)

(57) [Claims] 1. In an image of a normal face of a person, a pixel that is darker than any of the upper and lower pixels of the image is detected, another edge line is searched in the line direction from the detected edge line, and the corresponding edge line is detected. If there is a cleft, the cleft is detected from the image by connecting it, and the cleft detection method. 2. An image obtained by photographing a normal face of a person is input,
V for detecting a pixel that is darker than the upper and lower pixels of the image
The character edge detection means, the edge image storage means for storing the edge image output from the V-shaped edge detection means, and the connected edge lines are searched from the edge images stored in the edge image storage means. An edge line search means, another edge line in the direction of the edge line, and an edge line search / connection means that connects them if they are in the search range, and a mouth line that selects a cleft line from the connected edge lines. A cleft tear detection device comprising a tear line extraction means.