JPH1153525A - Facial organ detector and medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a facial organ detector by which a robust facial organ not to be affected by an image input environment such as the background, illumination conditions or object size can be segmented from an input image by using color information such as hue or saturation and to provide the medium therefor. SOLUTION: Based on the color information calculated by a color information calculating means 102 and smoothing filtering processing at a smoothing processing means 103, a figure area limiting means 104 extracts a figure area. Plural rectangular areas are selected from a figure area image by an initial extraction vector setting means 107 and based on the distribution of color information in each rectangular area, an evaluation vector generating means 108 generates the evaluation vector of each extraction vector. By comparing each evaluation vector with an evaluation template vector in an evaluation template dictionary 109, the suitability of each extraction vector is evaluated by an extracted area evaluating means 110, and in a recombining operating means 111, each rectangular area is recombined by genetic recombination operation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face organ detection device and a medium which can be used as a device for detecting a face organ region such as an eye, a nose, a mouth, and eyebrows from a natural image.

[0002]

2. Description of the Related Art As the opportunities for handling multimedia information increase, there is an increasing demand for extracting arbitrary objects, particularly human faces and their organs, from natural images. Examples of application of such technology include a personal matching technology using a face image and a technology for improving speech recognition accuracy using a lip.

Conventionally, as a technique for detecting a face organ from an image, there is a method of [1] detecting a face organ by performing binarization processing on an input image by shading and then detecting a connected region. [2] A method in which after an input image is binarized by shading, projections are calculated for the vertical and horizontal directions, and face divisions are detected on the projection pattern by threshold processing of the projection pattern. [3] A method of extracting an edge image from an input image and extracting a face organ based on template matching of a target organ shape and mutual positional relationship. [4] A method in which density information of an input image is represented by a compressed expression such as a mosaic expression, and a facial organ is detected by matching with a template. And the like.

[0004]

However, [1]
In the case of [2], it is necessary to dynamically change the optimal threshold value for performing binarization depending on the lighting conditions and the like, so that a large amount of noise is picked up on the image, or conversely, the face organ part is missing. Issues. In the case of [3], an image is captured when an edge emphasizing process is performed on an image including a face organ to extract a face contour, a contour shape of the face organ (eyes, nose, mouth, etc.) and a mutual positional relationship. There is a problem that it is often difficult to stably extract the edge itself due to the lighting conditions at that time. Furthermore, even if the edges are successfully extracted and the linear shape features of the face are obtained, it is still unclear how far the subtle differences between the face, facial organs and other objects can be expressed by the linear shape features It is also the fact that it is not. On the other hand, the method of [4] for compressing and expressing the light and shade pattern into a mosaic pattern has the advantage that simple processing is sufficient, but it is not possible to realize stable cutout unless certain constraints are satisfied. Reports have been made.

In the present invention, in consideration of such conventional problems, an input image is not affected by an image input environment such as a background, an illumination condition, and a size of an object by using color information such as hue and saturation. An object of the present invention is to provide a face organ detection device and a medium that can cut out a robust face organ.

[0006]

In order to solve the above-mentioned problems, a first facial organ detecting apparatus according to the present invention finds an inappropriate pixel group using color information and performs a smoothing process on a luminance value. Thus, first, a person candidate area is extracted. After that, pattern matching is performed between the evaluation vector generated from the color information distribution in each extraction candidate area selected from the person area and the evaluation template vector set generated from the previously prepared sample image set for extraction. Based on the result, the extraction candidate area is adjusted by a genetic algorithm.

Further, the second face organ detecting apparatus according to the present invention first extracts a person candidate area by finding an inappropriate pixel group using color information and performing a smoothing process on a luminance value, for example. . After that, pattern matching is performed between the evaluation vector generated from the color information distribution in each extraction candidate area selected from the person area and the evaluation template vector set generated from the previously prepared sample image set for extraction. Based on the result, the extraction candidate area is adjusted by a genetic algorithm. At this time, after performing adjustment in the vicinity of the extraction vector corresponding to the extraction candidate area and replacing it with the vector having the highest matching degree, the recombining operation of each extraction vector is performed using a genetic algorithm.

Further, the third face organ detecting apparatus according to the present invention first extracts a candidate person area by, for example, finding an inappropriate pixel group using color information and performing a smoothing process on a luminance value. . Pattern matching is performed between the evaluation vector generated from the color information distribution in each extraction candidate area and the evaluation template vector set generated from the prepared sample image set for extraction, and the genetic algorithm is used based on the result. Is performed to adjust the extraction candidate area. At this time, the extraction candidate area is divided into a plurality of groups based on the similarity between the evaluation vectors representing the respective extraction candidate areas, and crossover processing by a genetic algorithm in each group is prohibited, thereby identifying a target face organ. This is to detect only a plurality of persons at the same time.

The fourth face organ detecting apparatus according to the present invention first extracts a candidate person area by, for example, finding an inappropriate pixel group using color information and performing a smoothing process on a luminance value. . Pattern matching between an evaluation vector generated from the color information distribution in each extraction candidate area and a set of multiple organ evaluation template vectors generated from a set of sample images prepared for detecting a plurality of organs prepared in advance is also performed. In addition, the extraction candidate area is adjusted by a genetic algorithm to simultaneously extract a plurality of organs. At this time, first, adjustment is performed in the vicinity of the extraction vector corresponding to the extraction candidate area, and the vector is replaced with the vector having the highest matching degree. And multiple face organs (eyes, nose, mouth, ears, etc.) present in the input image are detected at the same time by prohibiting the crossover processing by the genetic algorithm in each group. is there.

[0010]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a configuration diagram of a face organ detection device according to a first embodiment of the present invention, and FIG. 2 is a configuration diagram of rearrangement operation means of the face organ detection device according to the first embodiment of the present invention. ,
FIG. 9 is a configuration diagram of a face organ detection device according to a second embodiment of the present invention, FIG. 11 is a configuration diagram of a face organ detection device according to a third embodiment of the present invention, and FIG. It is a lineblock diagram of a face part detecting device which is a 4th embodiment. In the drawings of the configuration, the same parts are denoted by the same reference numerals.

(First Embodiment) First, a first embodiment of the present invention will be described.
A face organ detection device according to the embodiment will be described.

In FIG. 1, reference numeral 101 denotes image input means for capturing image data to be identified by using a CCD camera or the like;
Reference numeral 102 denotes color information calculation means for calculating color information (saturation S, lightness V, hue H) at each pixel of the image input by the image input means 101, and reference numeral 103 denotes a person area based on the color information obtained in 102. After finding an inappropriate pixel group and setting the luminance value at that pixel to a value indicating that it is not a target, a smoothing processing unit 104 that applies a smoothing filter to the luminance. A person region extracting unit for extracting a region corresponding to a person; 105, a genetic algorithm processing unit for extracting a target face organ region from a 104 person region image using a genetic algorithm; 106, a genetic algorithm processing unit 106 Is a target area recording means for recording an area determined as a target facial organ area.

The genetic algorithm processing means 106 has 10
4, the center coordinates (x0 (i), y0 (i)) of the rectangular area necessary for extracting a target facial organ from the human area image, the number of horizontal pixels width (i), and the number of vertical pixels height (i ) Parameter sequence (extracted vector) v_c [i] = (x0 (i), y0
(i), width (i), height (i)) Initial extraction vector setting means 107 for randomly setting (i = 1, ..., K)
The K M-dimensional evaluation vector v_e [i] = (e (1, i), e (2,
i), ..., e (M, i)) (i = 1,2, ..., K) for comparison with the K evaluation vectors generated by the evaluation vector generating means 108,108 N evaluation template vectors T_e [j] = (te (1, j), te (2, j), ..., te (M , j)) (j = 1,2, ...,
N) is recorded in the evaluation template dictionary 109, 10
Evaluation vector v_e corresponding to each extraction region generated in step 8
Extraction area evaluation means 110 and 11 for comparing [i] with the template vector T_e [j] in the evaluation template dictionary 109 to evaluate the fitness “fitness (i)” of the extraction vector v_c [i].
It is composed of a recombining operation means 111 for performing a recombining operation of extracted vectors by a genetic algorithm based on the degree of conformity of each extracted vector at 0 and searching for an extracted rectangular area having a higher degree of conformity. As shown in FIG. 2, the rearrangement operation unit 111 outputs the current extracted vector v_c [i].
Of the extracted vectors based on the fitness degree [i] of each extracted vector, a candidate selecting means 201 for performing selective selection of the extracted vectors, and an intersection with the extracted vector set C obtained by the candidate selecting means 201 Cross-processing means 202 for executing processing,
It is configured by a mutation processing unit 203 that executes a mutation process on the extracted vector set C obtained by the crossover processing unit 202. Further, the candidate selection unit 201 selects a selection probability h (i) when selecting a certain extraction vector v_c [i] from the extraction vector set C and a selection range derivation unit 204 that derives the selection range I (i). , 1), a set of uniform random numbers g (i) G = (g
(1), g (2),..., G (K))
Extracted vector set C based on the result of random number generation means 205
And an extraction vector selection unit 206 for extracting an extraction vector to be selected from the above.

The operation of the face organ detecting apparatus according to the first embodiment configured as described above will be described. In the image input means 101, an input image is captured using a CCD camera or the like with a color signal of 256 pixels in the horizontal direction and 256 pixels in the vertical direction, each of which has 256 gradations of RGB. The color information calculation means 102 converts the color signal at each pixel in the input image captured in 101 into color information called saturation, lightness, and hue based on the HSV color system. In general, the hue H component is a value that represents the type of tint, is less likely to be affected by reflections and shadows due to lighting, etc., and is said to be effective for extracting areas where the hue is almost constant, such as a face. I have. In addition, the lightness V represents a value representing the brightness of the color, the saturation S represents the degree of vividness of the color, and the human face region is an object having relatively high saturation and high brightness, Relatively low chroma colors are used in office interior environments, including offices, where the invention is likely to be used primarily. The smoothing processing unit 102 puts a point on these two points, determines that a pixel group that does not correspond is not included in the person area, and resets the luminance value of that pixel to an exceptional value (for example, a large negative value). I do. In addition,
Here, the HSV color system shown in (Equation 1) is used, but any color system having a hue component and a saturation component can be similarly applied. In (Equation 1), r_xy, g_xy, and b_xy are color signals at pixel coordinates (x, y), H_xy is a hue, and V
_xy represents lightness, S_xy represents saturation, and Y_xy represents a luminance value.

[0016]

(Equation 1) Specifically, the above processing is performed by (i) saturation S_xy ≧ saturation threshold th_s (ii) lightness V_xy ≧ lightness threshold th_v (iv) 0.0 ≦ hue H_xy ≦ 60.0, or 300.0 ≦ H_xy ≦ 36.0 The luminance value Y_xy at the pixel coordinates (x, y) that are not satisfied is -1000
Reset to 0.0. As the luminance Y_xy at the satisfied (x, y), the value calculated from (Equation 2) is used as it is. In addition,
I_xy and Q_xy will be described later.

[0017]

(Equation 2) Then, based on the result, a smoothing filter as shown in FIG. 3 is used for smoothing the luminance value. This is a process for reducing the influence by performing a smoothing process because there is a possibility that a pixel that does not satisfy the brightness, saturation, and hue conditions may exist in the face organ.

Next, the person region extracting means 104 performs the following processing. First, with the vertical pixel y fixed as shown in FIG. 4, the pixel coordinates are set in the horizontal pixel x direction (0 ≦ x ≦ 255).
A pixel histogram Count (x) satisfying the luminance value Y_xy ≧ 0.0 at (x, y) is obtained, and an average value C_ave_x in the x direction is obtained. Then, scanning from x = 0, x = x_s where Count (x) exceeds C_ave_x first, and scanning from x = 255, C_ave_x
X = x_e that exceeds In a certain vertical pixel y, an area corresponding to x_s to x_e is regarded as a person area in the horizontal pixel x direction. Repeat this operation for all vertical pixels y (0 ≦ y ≦ 25
This is performed for 5) to determine a person area in the horizontal pixel x direction. Similarly, in the vertical pixel y direction (0 ≦ x ≦ 255), a pixel histogram Count (y) satisfying the luminance value Y_xy ≧ 0.0 is obtained, and an average value C_ave_y in the y direction is obtained. And y =
Scan from 0, Count (y) first exceeds C_ave_y y = y_s
And similarly scan from y = 255 and first y = above C_ave_y
Find y_e. In a certain horizontal pixel x, an area corresponding to y_s to y_e is not seen in the person area in the vertical pixel y direction, and this operation is performed on all the horizontal pixels x (0 ≦ x ≦ 255) to obtain a person in the vertical pixel y direction. The area is determined. X_s to x_e in the horizontal pixel x direction, vertical pixel y thus obtained
A process of extracting a rectangular area included in y_s to y_e in the direction as a person area is performed.

Further, in order to detect only a target facial organ from a human region, the genetic algorithm processing means 105 performs region extraction using a genetic algorithm as described below. Here, a vector T_e [j] (j = 1, j) in an evaluation template vector set G_T generated from a sample image set prepared in advance for extracting a target facial organ.
, N) and an evaluation vector v_e [i] (i = 1, 2,...) Obtained from K rectangular areas selected from the person areas extracted by the person area extracting means 104. K) is performed, and an optimal rectangular area is regarded as a target facial organ and cut out. At that time, the coordinates of the center point of the rectangular area (x0 (i), y0
(i)), the optimal values of the horizontal pixel number width (i) and the vertical pixel number height (i) are estimated by using a genetic algorithm (GA) having a large effect by combining a large number of parameters.

The genetic algorithm is an optimization method that simulates the evolution of the chromosome of a species of an organism while adapting to the environment.
"In Search Optimization and Machine Learning"("Genetic Algorithms in Search,
Optimization and Machine Learning ”(David E. Goldb
erg, Addison Wesley)). First, a parameter sequence vector including a plurality of parameters to be estimated is regarded as a chromosome q_k (k = 1, 2,..., K), and a problem to be optimized is regarded as an environment. In nature, individuals whose chromosomes are unsuitable for the environment die, and those that are suitable crosses with other chromosomes to produce offspring. The offspring combine the genes of their parents and have chromosomes not previously found in the population. A child born to the same parents can get better or worse depending on the combination of genes. Highly fit offspring increase offspring and low offspring die. Sometimes chromosomes mutate. By repeating this process, the population of chromosomes is gradually homogenized to have chromosomes with a high degree of fitness. In GA, a group of parameter column vectors to be determined is regarded as a group of chromosomes in an organism. First, a group of parameter column vectors is randomly generated. Each solution is evaluated by substituting it into a function to be optimized (referred to as an evaluation function), and those that are close to optimal are proliferated, and those that are not suitable are eliminated. Then, a part of the solution is exchanged with another chromosome. This is called crossover. Further, a mutation is caused at a certain probability, and a part of the parameter column vector is changed (see FIG. 5).
The process of generation alternation consisting of multiplication / selection, crossover, and mutation is repeatedly applied, and after a certain generation alternation, the best parameter column vector in the group is set as the optimal vector. GA
Is that a plurality of searches are simultaneously performed in a search space using a parameter column vector group. This reduces the possibility of falling into a local minimum. Another feature is that efficient search can be performed by creating a new parameter column vector by combining good points of a plurality of parameter column vectors. The GA optimizes a parameter column vector by repeatedly executing the above processing.

In the present invention, facial organs are cut out by applying GA to the person region cut out based on the color information in step 104. It should be noted that the lip is used as a target facial organ to be described. As shown in Fig. 6, the chromosome structure is the center coordinates of the rectangular area required to extract the facial organs.
(x0 (i), y0 (i)), horizontal pixel number width (i), vertical pixel number height (i)
(i = 1, 2,..., K). First, in the initial extraction vector setting means 107, a chromosome set D having K elements set at random is prepared. In this case, the chromosome is generated by using the coordinates of the rectangular area, the number of horizontal pixels, and the number of vertical pixels as they are, but the coordinates, the number of horizontal pixels, and the number of vertical pixels are converted into a binary bit string of 0 and 1 and arranged. It is also conceivable to use a bit string. The evaluation vector generating means 108 generates an evaluation vector representing the feature of the image in the rectangular area represented by each chromosome in order to evaluate the fitness of each chromosome q_i (i = 1, 2,..., K). . Although various methods can be considered for the evaluation of the chromosome q_i, the present invention uses the distribution of the hue H in the HSV color system in the rectangular area and the distribution of the I signal in the YIQ color system expressed as (Equation 2). And This is because (1) the hue H is less affected by fluctuations in lighting conditions, and (2) the I and Q signals corresponding to the so-called color difference in the YIQ color system used in normal NTSC color TV broadcasting. Is caused by having a large value in the lips compared to the surroundings. Note that it is naturally possible to use the distribution of the hue H of the HSV color system and the Q signal of the YIQ color system. In addition to these, many combinations can be considered as representing features for evaluating a rectangular area, which is an extraction candidate, and the distribution of hue H and lightness V in the HSV color system may be used as an example. This is because, in the HSV color system, the white and black systems cannot be determined only by the hue H and the brightness V needs to be used, and this is a device for surely finding the skin color.

First, in the hue H, from 0.0 degrees at which flesh color exists.
Divide 60.0 degrees and 300.0 degrees to 360.0 degrees into Hnum equal regions. Similarly, the I signal ranges from a minimum of -160.0 to a maximum of 16
Divide the 0.0 range into Inum equal regions. H division width
Assuming that Hstep = 120 / Hnum and the division width of I is Istep = 320 / Inum, the range I_h (jj) of the jj-th H is represented by (Equation 3), and the range I_i (kk) of the kk-th I is ( Equation 4) is obtained. In addition,
In Equation (4), the range of the kk-th Q, I_q (kk), is also described. This is a histogram obtained when the distribution ratio of the Q signal is used instead of the distribution ratio of the I signal as an evaluation feature. It indicates the area setting for creation, and the following I
What is necessary is just to perform the same processing as the signal processing.

[0023]

(Equation 3)

[0024]

(Equation 4) The number of pixels C_h (jj) included in the range I_h (jj) of H is obtained, and the ratio ratio_h (jj) (jj = 1, 2, to the total number of pixels in the entire region obtained by the person region limiting unit 104). .., Hnum)
Ask for. Similarly, the number of pixels C_i included in the range I_i (kk) of I
(kk) and the ratio of that value to Total ratio_i (kk) (kk =
1,2, .., Inum). By arranging these in order, the evaluation vector v_e corresponding to the extraction vector v_c [i]
[i] = (e (1, i), e (2, i), ..., e (M, i)) = (ratio_h (1), ..., rat
io_h (Hnum), ratio_i (1), ..., ratio_i (Inum)) (M = Hnum + I
num) is derived. The evaluation template dictionary 109 includes N_sample prepared in advance for extracting a target facial organ region.
An M-dimensional vector T_ed [nn] = (ted (1, n) having the distribution ratio of the hue H and Q signals in the image nn in the sample image set for
n), ted (2, nn), ..., ted (M, nn)) (nn = 1,2, ..., N_sample) Then, the created N_sample vectors T_
Vector quantization (Vector Quantization: VQ) is applied to ed [nn] to create an evaluation template vector set having N elements. VQ is a clustering method that divides the original vector space into multiple subspaces by arranging vectors representing each subspace (cluster) called reference vectors according to the data distribution of a large number of prepared vector sets For details, refer to the document "An algorithm for vector
or vector quantizer design "(IEEE Transaction of Co
mmunication, COM-28, No.1, pp.84-95,1980, Linde,
Y., Buzo, A. and Gray, RM)).
In the present invention, an evaluation vector generated from a set of N_sample extraction sample images prepared in advance using this VQ is represented by N
Then, an evaluation template vector set G_te is generated using the obtained N reference vectors T_e [j] (j = 1,..., N). This G_te corresponds to the evaluation template dictionary 109.

Next, the extraction area evaluation means 110
The fitness of each chromosome is calculated using the evaluation vector from the rectangular region corresponding to each chromosome q_i (i = 1, 2,..., K) generated in the above. Fitness function fitness (i) representing the fitness of each chromosome
In the present invention, the evaluation vector v_e [i] (i = 1,..., K) and the vector T_e [j] (j = 1, ..., N) and the maximum value of the correlation coefficient Sim (i, j)
ness (i). However, the evaluation vector v_e [i] (i = 1, ...,
Match the maximum value of the reciprocal of the square distance D (i, j) between (K) and the vector T_e [j] (j = 1, ..., N) in the prepared N evaluation template dictionaries It can also be regarded as degree fitness (i).
In GA, the optimal chromosome q_best
Is estimated.

Further, at 110, it is determined whether the extraction criterion is satisfied or the number of repetitions of the genetic algorithm processing does not exceed a preset upper limit. In addition,
Although various extraction criteria can be considered, here, the minimum fitness f_th is used as the extraction criteria. In this determination, [i] when the maximum fitness f_min is larger than the extraction criterion f_th, the area extraction is completed, and the optimal rectangular area is recorded by the extraction area recording unit 106.

[Ii] The maximum fitness f_min is the extraction criterion f_th
If the number is smaller and the chromosome set update count g_num is equal to or smaller than the preset maximum number of repetitions g_num_max, the process proceeds to the rearrangement operation unit 111.

[Iii] The maximum fitness f_min is the extraction criterion f_th
If smaller, and the chromosome set update count g_num exceeds a preset upper limit count g_num_max,
It is determined that the target face organ is not included in the input image, and the face organ detection processing ends. Is executed according to the three determination processes.

The operation of the rearrangement operation means 111 is as follows. First, chromosome selection is performed in the candidate selection means 201. In this case, as shown in FIG. 7, a roulette wheel selection method for selecting a chromosome with a probability proportional to the fitness is used.

(Roulette Selection Method) (1) The fitness degree fitnsee (i) of each chromosome q_i (i = 1,..., K) belonging to the chromosome set Pd and the sum f_total of the fitness degrees of all chromosomes are obtained.

(2) The selection probability h (i) that q_i is selected as a parent that creates the next generation chromosome is obtained as shown in (Equation 5).

[0032]

(Equation 5) In order to assign this probability to a chromosome, for example, the following method can be considered. (iii) The selection range L (i) of each chromosome is allocated to the section in [0, 1) using (Equation 6) and (Equation 7) as follows. That is,

[0033]

(Equation 6) Then, the selection range L (i) of q_i is

[0034]

(Equation 7) Is defined as Here, a set G of uniform random numbers g (i) in [0,1)
= (g (1), g (2), ..., g (K)). g (i) ∈I (j) (i, j =
A set of num (i) = j satisfying (1,2, ..., K) Num = (num (1), num
(2), ..., num (K)), a set of K chromosomes corresponding to this Num is selected.

The chromosome q_i in the current chromosome group P is selected by such a roulette wheel selection method.
First, the selection range deriving means 20 according to (Equation 5) to (Equation 7)
4 is the probability h (i) that each chromosome is selected and the selection range L (i)
Ask for. Then, the random number generation means 205 generates K uniform random numbers g between 0 and 1. The set of random numbers G obtained by the random number generation means 205 and the selection range L (i) obtained by the selection range derivation means 204 are sent to the chromosome selection means 206 and g (i)
A set Num of num (i) = k that satisfies ∈I (j) is obtained. As a result, the chromosome selection means 206 outputs a new chromosome population Pd composed of chromosomes designated by Num. Crossover processing means 202 performs crossover processing on the new chromosome population Pd obtained by the candidate selection means 201. There are various methods for the crossover processing. In this embodiment, a one-point crossover or a two-point crossover as shown in FIG. 5 is used. Further, the mutation means 203 is provided for
This is realized by adding a random number given within a certain range to a gene (an element of a coordinate vector in a multidimensional space) selected at a certain low probability to a new chromosome population obtained through 02. At that time, we tried to maintain more chromosomal diversity by changing the probability of mutation in half and the other half of the chromosome population. Here, the real-valued elements of the coordinate vector in the multidimensional space are arranged as they are and treated as chromosomes, but they may be converted into bit strings and treated. FIG. 8 illustrates an example of extraction of a lip that is a facial organ from a certain input image. FIG.
8A shows an input image, FIG. 8B shows a person extraction result, and FIG.
(C) shows an example of a face organ extraction result by GA (Genetic Algorithm). FIG. 8C shows a result obtained by applying a rectangular line to the lip region detected in the input image of FIG. 8A, and shows a result of detecting that there is a lip in the region inside the rectangular line. is there. In the extraction of the person region, the saturation threshold th_s = 25 and the lightness threshold th_v = 18. Chromosomes prepared per generation when extracting facial organs by GA
The number of q_i is assumed to be 100, and the result of repetitive estimation for 60 generations before the facial organ extraction is shown. As is clear from these, it can be seen that the facial organ of the person has been well cut out.

As described above, according to the present embodiment, after extracting a person region from an input image using color information, a region which seems to be most compatible with a template vector dictionary prepared in advance by a genetic algorithm is searched. Face organs can be extracted, and face organs can be automatically extracted independently of the background and the size of the face.

(Second Embodiment) Next, a second embodiment of the present invention will be described.
A face organ detecting device according to the embodiment will be described.
FIG. 9 is a configuration diagram of a face organ detection device according to a second embodiment of the present invention. In FIG. 9, reference numeral 901 denotes an extraction vector selection means for extracting a plurality of vector groups from the vicinity of each extraction vector, and 902 denotes an extraction vector selection means 90
This is an extraction vector neighborhood adjustment unit that replaces the vector having the highest degree of matching from the vector group obtained in step 1 with the original extraction vector. The operation of the face organ detecting apparatus according to the second embodiment configured as described above will be described. The input image input from the image input unit 101 is similar to the face organ detection device according to the first embodiment of the present invention, and the color information of the hue, saturation, and brightness obtained by the color information calculation unit 102 is used. , The luminance of the inappropriate pixel is set to a non-target value, and the person area is roughly cut out through luminance smoothing processing. Thereafter, the genetic algorithm processing means 105
First, the hue H in the rectangular area randomly cut out
The evaluation vector is obtained from the ratio of the distribution value of the distribution signal and the distribution value of the Q signal in the YIQ color system to the entire human area. Then, the matching degree of each extracted vector is determined by pattern matching between the evaluation vector and a template vector in the evaluation template dictionary 109 for facial organ extraction prepared in advance, and a genetic rearrangement operation is performed based on the matching degree. By repeating the adjustment of the extraction vector (rectangular area as an extraction candidate), an optimal face organ area is estimated and recorded by the extraction area recording unit 106. In the present embodiment, in addition to the above processing, local adjustment capability processing is added in order to improve the efficiency of optimization of the extraction vector. Before moving to the rearrangement operation means 111, any extracted vector v_c [i] (i = 1, 2,..., K) at the time is centered on a sphere in a multidimensional space within the vector norm len. Mm vectors v_cd [j] (j = 1, .., mm)
Select Then, an evaluation vector v_ed [j] is similarly obtained for the rectangular area represented by the mm vectors,
The fitness fitn is determined based on the evaluation with the evaluation template dictionary 109.
Find ess_d (j). The vector having the highest fitness in the total (mm + 1) fitness of fitness (i) between the fitenss_d (j) and the original extracted vector v_c [i] is again genetically rearranged by the rearrangement operation means 111. Is replaced by an extraction vector. By doing so, it is possible to compensate for the lack of local search capability, which has been a problem in searching for a genetic algorithm, and to detect a target facial organ at high speed.

As described above, according to the present embodiment, it is possible to detect a target facial organ in an input image at high speed and with high accuracy, and the effect is considered to be great.

(Third Embodiment) Next, a face organ detecting apparatus according to a third embodiment of the present invention will be described. FIG. 10 shows the configuration of a face organ detection device according to the third embodiment. Reference numeral 1001 denotes a vector similarity deriving unit that derives the similarity between the evaluation vectors with respect to each extracted vector obtained in 108, and 1002 denotes a vector similarity deriving unit 100.
Extracting vector set dividing means based on the similarity between the vector obtained in 1 divides the current extraction vector set into a plurality of groups, 1003 recombinant manipulation by the genetic algorithm with low similarity plurality of extraction vectors The group-specific rearrangement operation means 1004 performs the region extraction process when the number of repetitions of the rearrangement operation of the extraction vector exceeds the preset upper limit of the number of repetitions. This is a multiple target recording unit that records a target area represented by a satisfactory extraction vector.

The operation of the face organ detecting apparatus according to the third embodiment configured as described above will be described. The input image input from the image input unit 101 is similar to the face organ detection device according to the first embodiment of the present invention, and the color information of the hue, saturation, and brightness obtained by the color information calculation unit 102 is used. , The luminance of the inappropriate pixel is set to a non-target value, and the person area is roughly cut out through luminance smoothing processing. Thereafter, the genetic algorithm processing means 105
Then, an evaluation obtained from a template vector in the evaluation template dictionary 109 for facial organ extraction prepared in advance, the hue in a randomly cut rectangular area, and the distribution ratio of the Q signal in the YIQ color system to the entire human area. The matching degree of the current extracted vector is evaluated by pattern matching with the vector. The optimal face organ region is estimated using the recombination operation by the genetic algorithm based on the fitness evaluated here.Generally, in the genetic algorithm, as the diversity in the search candidate vector set is lacking, It has already been clarified that the rate of dependence on the search for an optimal solution due to mutation increases, and the search ability decreases. Therefore, in order to maintain the diversity in the search candidate vector set, it is necessary to avoid a re-combining operation between similar vectors. Therefore, the similarity between each of the extracted vectors is calculated, and the crossover processing, which is the re-combining operation in the genetic algorithm, is prohibited for a combination in which the similarity is larger than a preset similarity threshold. First, the extraction vector similarity deriving unit 1001 calculates the similarity sim (i, j) between the vectors v_c [i] and v_c [j] in the extraction vector set. There are many possible definitions of this similarity. For example, the norm between two vectors, the angle formed between two vectors, the sum of absolute values of differences between two vector elements, and the like. However, in order to avoid searching in the same region as much as possible, in the present embodiment, a value obtained by dividing the number of pixels where the rectangular regions represented by the two extracted vectors overlap by the smaller number of the two rectangular region pixels is used. did. In this way, the greater the overlap, the greater the similarity between the two vectors (FIG. 11). When the distribution ratio of the color information in the region to the number of pixels in the candidate region is used as the evaluation feature amount of the extraction candidate region as in the present invention, the partial region of the target facial organ often shows a fairly high degree of fitness. As the area search by the genetic algorithm progresses, the extracted vector representing the area close to the target face organ area and the extracted vector representing the area corresponding to a part of the face organ can be mixed in the same extracted vector set The nature becomes high. In this case, if, for example, the norm between two extracted vectors is used as the extracted vector similarity, the groups are clearly divided into different groups, and as a result, there is a possibility that multiple facial organs are detected in the same region. Occurs. However, as in the present invention, when a value obtained by dividing the number of pixels where the rectangular areas represented by two extraction vectors overlap by the smaller number of pixels of the two rectangular areas is used, when one is included in the other, Since the similarity between the extracted vectors representing the two regions is 1.0, the extracted vectors can be always classified into the same group, and the above-described problem can be avoided. However, in the first stage of the target area adjustment by the genetic algorithm,
No matter how much the number of pixels in the rectangular area is different, if one area is included in the other, it is considered to belong to the same group, which has a bad influence on the optimization of the extraction area, so the small extraction area The ratio rrr of the number of pixels in the large extraction region to the number of pixels of the area difference threshold th_check
If smaller, the two extracted vector similarities
multiply by rrr. The extracted vector similarity obtained in this way is a similarity threshold th_s prepared in advance.
Vectors larger than im are in the same group Gs_kk (kk = 1, ..., g
(num) by the extracted vector set dividing means 1002. The group-specific rearrangement operation means 1003
When performing the crossover process in 202, the candidate selection unit 20
In 1, if two of the same groups are selected from the group classified in 1002, only the mutation process in 203 is performed without performing the cross process in 202. On the other hand, when a group is selected from another group, after the crossover process of 202, the mutation process of 203 is performed. Then, a series of processing is performed until the rearrangement operation in the genetic algorithm processing means 105 satisfies a preset threshold value of the number of repetitions, and each group of extracted vectors at the time when the threshold value of the number of repetitions is satisfied. Then, a rectangular area represented by an extraction vector that satisfies the matching threshold value, which is a prepared extraction criterion, is recorded in the plural extraction recording means 1004, and the face organ detection of plural persons is completed.

As described above, in the present embodiment, after detecting a facial organ of one person, a repetitive operation is performed such that the region is masked and the facial organ of the next person is detected again using a genetic algorithm. Without this, it is possible to efficiently and simultaneously detect facial organs targeted for a plurality of persons existing in the input image, and the effect is great.

(Fourth Embodiment) Finally, the fourth embodiment of the present invention
A face organ detecting device according to the embodiment will be described.
FIG. 12 shows a configuration of a face organ detection device according to the fourth embodiment. A multiple organ evaluation template dictionary 1201 stores evaluation template vectors generated from a set of extraction sample images prepared in advance for each organ to extract a plurality of facial organs, and 1202 is set in advance in each group. A plurality of organ object recording means for recording an object area represented by an extraction vector satisfying the extraction criterion.

The operation of the face organ detecting apparatus according to the fourth embodiment configured as described above will be described. The processing from the image input unit 101 to the group-specific rearrangement operation unit 113 is the same as in the case of the face organ detection device according to the third embodiment of the present invention. An object of the present invention is not to extract the face organs of a plurality of persons present in an input image, but to detect a plurality of organs such as eyes, nose, mouth, and eyebrows, which are one face organ. . Therefore, a sample image set for creating a template dictionary 1201 for evaluating each extracted vector is prepared for each of a plurality of organs. Then, a multiple organ evaluation template dictionary 1201 is created for the evaluation vector obtained from each sample image by using a learning vector quantization technique. The learning vector quantization method is based on a vector T_r [k] = (tr (1, k), tr (2, k), ..., tr (M,
k)) Create a set G_Tr of (k = 1, 2,..., P). As a method of creating the vector set G_Tr, in the present invention, an LVQ for creating a vector set representing each category from a plurality of vectors v given in the following procedure is applied. [i] The closest representative vector v_c (n) is found for the vector v by Expression (Equation 8). Note that n represents the number of repetitions in LVQ, and v_p (n) represents a representative vector in the number n.

[0044]

(Equation 8) [ii] Here, the category to which v belongs is G_v, and the category to which v_c (n) belongs is G_vc. If G_v and G_vc are in the same category, v_c (n) is made closer to v as in (Equation 9), and if they are different categories, v_c (n) is moved away from v as in (Equation 10). Note that the representative vectors other than v_c are not updated.

[0045]

(Equation 9)

[0046]

(Equation 10) alpha (n) is a monotonically decreasing function (0 <alpha <1). LVQ
Is a self-organizing neural network method developed from VQ, in which a reference vector representing each cluster in VQ is made to correspond to the coupling coefficient between neurons, and an appropriate reference vector is obtained by using supervised learning. is there. In LVQ, the distance between vectors belonging to the same category is reduced, and the distance between vectors belonging to different categories is increased to optimize the boundary between the categories.
In the present invention, using LVQ to the evaluation vector obtained from a sample image set for detection of a plurality of organs given in advance, to create a reference vector set representing each organ to be detected,
It is prepared as an evaluation template dictionary 1201. In each group of the extracted vector set when the number of repetitions in the genetic algorithm processing unit 105 satisfies a preset threshold value of the number of repetitions, 1202 indicates a rectangular area indicated by the extracted vector satisfying the extraction criterion. Record it. In this case, the extraction judgment criterion may be the lowest matching degree, and may be a uniform value. However, it is wise to set individually according to the degree of difficulty in detecting each facial organ.

As described above, the present embodiment is intended to simultaneously and efficiently detect a plurality of facial organs possessed by a person in an input image, and can be used in many fields including personal verification using images. The effect is great.
Note that, in the genetic algorithm processing means 105 of the face organ detecting apparatus according to the first to fourth embodiments of the present invention, the center coordinates (x0 (i), y0 (i)) of the rectangular area required for extraction,
A parameter column vector consisting of the number of horizontal pixels width (i) and the number of vertical pixels height (i) (i = 1,2, ..., K) was applied as a chromosome. By considering the axis and setting the tilt angle α with the left rotation of the symmetry axis being positive, and adding this to the parameter column vector, it can be applied to the extraction of a tilted face area. In that case, when calculating the fitness of each extracted rectangular area, matching with the evaluation template dictionary can be performed by rotating the image in the rectangular area by an angle -α with respect to the center of gravity of the rectangular area. . Although the crossover processing unit 202 in the face organ detecting apparatus according to the first to fourth embodiments of the present invention has been described using one-point or two-point crossover as shown in FIG. A method of arbitrarily selecting two points on a straight line connecting the two selected vectors is also conceivable.

Further, in the creation of an evaluation template dictionary in the face organ detecting apparatus according to the first to third embodiments of the present invention, a vector quantization technique is applied to an evaluation vector obtained from a sample image set prepared in advance. However, it is also possible to use a learning vector quantization method having a learning function using a neural network in this vector quantization method. By doing this, instead of mechanically creating a face organ template focusing solely on the vector distribution in the evaluation vector space, the face organs belonging to the same person must be learned while learning so that they always belong to the same category. Organ template vectors can be created. By doing so, it is considered that the feature in the color information of the facial organs possessed by each individual can be better extracted, and the detection efficiency of the target facial organ from the input image can be further improved.

As described above, the first to fourth face organ detecting devices according to the present invention relate to a technique for extracting only a face organ such as a mouth without being affected by the environment when an image is input. Therefore, stable region extraction can be realized with a simple structure without being affected by the background, the size of the face organ in the image, the lighting conditions, and the like.

It should be noted that the same effects as in the above case can be obtained by using a medium on which a program for causing a computer to execute the functions of all or some of the means described in the above embodiments is used. I do.

The processing operation of each means of the above-described embodiment may be realized as software by using a computer and operating a program, or the processing operation may be performed without using a computer. The circuit configuration may be implemented as hardware.

[0052]

As is apparent from the above description, the present invention has an advantage that the detection of the face organ can be performed more stably than in the conventional case.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a face organ detection device according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a rearrangement operation unit of the face organ detection device according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating a filter used in a brightness value smoothing process according to the embodiment;

FIG. 4 is a conceptual diagram of a method for extracting a person region from an input image according to the present embodiment;

FIG. 5 is a conceptual diagram of crossover / mutation processing performed by the rearrangement operation means according to the embodiment;

FIG. 6 is a conceptual diagram illustrating a relationship between an extracted rectangular region and a chromosome structure in a genetic algorithm according to the present embodiment.

FIG. 7 is a diagram for explaining a roulette selection method used for selection.

FIG. 8 shows an example of lip extraction according to the present embodiment.
The halftone image displayed on the display is output from a printer, and is a photograph replacing a drawing.
(A) is a photograph replacing a drawing showing an input image, (b) is a photograph replacing a drawing showing a person image extracted by saturation and hue,
(C) is a photograph instead of a drawing showing the lip region extracted by the genetic algorithm.

FIG. 9 is a block diagram illustrating a configuration of a face organ detection device according to a second embodiment of the present invention.

FIG. 10 is a block diagram illustrating a configuration of a face organ detection device according to a third embodiment of the present invention.

FIG. 11 is a conceptual diagram related to group classification of a set of extraction candidate regions.

FIG. 12 is a block diagram illustrating a configuration of a face organ detection device according to a fourth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 Image input means 102 Color information calculation means 103 Smoothing processing means 104 Person area limitation means 105 Genetic algorithm processing means 106 Extraction area recording means 107 Initial extraction vector setting means 108 Evaluation vector generation means 109 Evaluation template dictionary 110 Extraction area evaluation means 111 Recombination operation means 201 Candidate selection means 202 Crossover processing means 203 Mutation processing means 204 Selection range derivation means 205 Random number generation means 206 Extraction vector selection means 901 Extraction vector neighborhood selection means 902 Extraction vector neighborhood adjustment means 1001 Extraction vector similarity derivation means 1002 Extracted vector set division means 1003 Group-specific reordering operation means 1004 Multiple object recording means 1201 Multiple organ evaluation template dictionary 1202 Multiple organ object recording means

Claims (9)

[Claims] An image input unit for inputting an image; a color information calculation unit for calculating color information of the input image; and a pixel not belonging to a person area is extracted from the color information obtained by the color information calculation unit. Smoothing processing means for performing a smoothing process on luminance; Person area limiting means for limiting a person area based on the smoothing processing means; and setting of a group of extraction vectors for specifying an initial extraction area from within the person area Initial extraction vector setting means for performing evaluation vector generation means for generating an evaluation vector used for each area evaluation from a distribution of color information in an extraction area specified by the set extraction vector; and An evaluation template dictionary for storing evaluation template vectors generated based on a sample image set for extraction prepared in advance; Extracting area evaluation means for comparing the obtained evaluation vector with the evaluation template vector in the evaluation template dictionary and evaluating the degree of fitness, and an extraction criterion in which the degree of fitness obtained by the extraction area evaluating means is set in advance. A recombining operation means for performing a symbolic recombining operation of the extracted vector based on the degree of suitability when the satisfaction is not satisfied; A face organ detecting apparatus, comprising: a target area recording unit that records an organ. 2. An image input means for inputting an image, a color information calculation means for calculating color information of an input image, and a pixel which does not belong to a person area is extracted from the color information obtained by the color information calculation means. Smoothing processing means for performing a smoothing process on luminance; Person area limiting means for limiting a person area based on the smoothing processing means; and setting of a group of extraction vectors for specifying an initial extraction area from within the person area Initial extraction vector setting means for performing evaluation vector generation means for generating an evaluation vector used for each area evaluation from a distribution of color information in an extraction area specified by the set extraction vector; and An evaluation template dictionary for storing evaluation template vectors generated based on a sample image set for extraction prepared in advance; Extracting area evaluation means for comparing the obtained evaluation vector with the evaluation template vector in the evaluation template dictionary and evaluating the degree of fitness, and an extraction criterion in which the degree of fitness obtained by the extraction area evaluating means is set in advance. Extracting vector neighborhood selecting means for extracting a plurality of vector groups from the vicinity of each extracted vector when not satisfied; and a vector having the highest fitness and the original extracted vector among the vector groups obtained by the extracted vector neighboring selecting means Extracting vector neighborhood adjusting means for replacing the extracted vector, a recombining operation means for performing a symbolic rearranging operation of the extracted vector replaced by the extracted vector neighboring adjusting means based on the degree of conformity, and when the extraction criterion is satisfied. The target area recording means for ending the area extraction processing and recording the optimal extraction area as a face organ. A facial organ detection device characterized by being formed. 3. Image input means for performing image input processing; color information calculation means for calculating color information of an input image; and pixels which do not belong to a person area are extracted from the color information obtained by the color information calculation means. Together with a smoothing processing means for performing a smoothing processing on the luminance, a person area limiting means for limiting a person area based on the smoothing processing means, and a group of extraction vectors for specifying an initial extraction area from within the person area. Initial extraction vector setting means for setting; evaluation vector generating means for generating an evaluation vector for use in evaluating each area from a distribution of color information in the extraction area specified by the set extraction vector; Extraction vector similarity deriving means for deriving the similarity between, and an evaluation template generated from a set of extraction sample images prepared in advance for extracting a face organ An evaluation template dictionary for storing a vector, an extraction vector evaluation unit for comparing an evaluation vector corresponding to each extraction region obtained by the evaluation vector generation unit with an evaluation template vector in the evaluation template dictionary, and evaluating the degree of conformity. Means, an extracted vector set dividing means for dividing an extracted vector into a plurality of groups based on the extracted vector similarity, and a group for performing a rearrangement operation of a plurality of extracted vectors having a low similarity based on the fitness. A separate recombination operation means, an extraction vector which satisfies a predetermined extraction determination criterion in each group when the number of repetitions of the extraction vector rearrangement operation exceeds a preset maximum number of repetitions, and terminates the region extraction processing. Characterized by comprising a plurality of target recording means for recording a target area represented by Organ detection device. 4. Image input means for performing image input processing; color information calculation means for calculating color information of an input image; and pixels which do not belong to a person area are extracted from the color information obtained by the color information calculation means. Together with a smoothing processing means for performing a smoothing processing on the luminance, a person area limiting means for limiting a person area based on the smoothing processing means, and a group of extraction vectors for specifying an initial extraction area from within the person area. Initial extraction vector setting means for setting; evaluation vector generating means for generating an evaluation vector for use in evaluating each area from a distribution of color information in the extraction area specified by the set extraction vector; Extraction vector similarity deriving means for deriving a similarity between the images; and extracting a plurality of facial organs from an extraction sample image set prepared in advance for each organ. A plurality of organ evaluation template dictionaries storing the generated evaluation template vectors, and comparing an evaluation vector corresponding to each extraction region obtained by the evaluation vector generating means with an evaluation template vector in the plural organ evaluation template dictionary. Extraction region evaluation means for evaluating the degree of fitness, extracted vector neighborhood selection means for selecting a plurality of vector groups from the vicinity of each extracted vector, and a vector having the highest degree of fitness in the selected vector group and the original vector. Extraction vector neighborhood adjustment means for replacing the extraction vector; extraction vector set division means for dividing the extraction vector into a plurality of groups based on the extraction vector similarity; and a plurality of low similarity degrees based on the fitness. Group-based rearrangement operation means for performing rearrangement of extracted vectors, and rearrangement of extracted vectors When the number of operation repetitions exceeds a preset upper limit number of repetitions, the region extraction process is terminated, and a plurality of organ objects for recording a target region represented by an extraction vector satisfying a preset extraction criterion in each group A facial organ detecting device comprising a recording unit. 5. The recombining operation means uses a genetic algorithm for performing a genetic operation process by crossover, mutation, and selection.
A facial organ detection device according to claim 1. 6. The face organ detecting apparatus according to claim 3, wherein said group-specific rearranging operation means uses a genetic algorithm for performing a genetic operation process by crossover, mutation, and selection. apparatus. 7. The face organ detecting apparatus according to claim 1, wherein said evaluation vector is created from a distribution histogram of hue and lightness in each area. 8. The face organ detecting apparatus according to claim 1, wherein said evaluation vector is created from a distribution histogram of hue and color difference in each area. 9. A medium in which a program for causing a computer to execute the functions of all or a part of each means described in any one of claims 1 to 8 is recorded.