JP2008123399A - Feeling recognition device, electronic equipment, feeling recognizing method, control program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a feeling recognition device, electronic equipment, a feeling recognizing method, a control program and a recording medium capable of easily recognizing feeling from a face image without performing large scale study processes or creation of a dictionary. <P>SOLUTION: A visual guide field, which is a "field" distributing over the perimeter of the face image, of which intensity depends on the distance from the image, and of which value gets larger as closer to the image is determined. The degree of complexity of a closed curved surface of an equipotential line is determined from the guide field. A function which is a correspondence of the degree of complexity and a potential value is acquired as distribution information of the guide field, and the feeling corresponding to the face image is distinguished based on the function. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to an emotion recognition device, an electronic device, an emotion recognition method, a control program, and a recording medium that can recognize an emotion from a face image.

2. Description of the Related Art Conventionally, an emotion recognition device that extracts facial feature values from a face image and determines which emotion the feature value corresponds to based on dictionary data obtained by a sample survey in advance is known (for example, Patent Documents). 1 to 4). This type of emotion recognition device employs a method that uses facial parts such as eyes, nose, mouth, and eyebrows as facial features, and a method that uses quantity values such as image analysis results obtained by frequency analysis or filters. There is something.
JP 2004-513462 A JP 2005-512248 A JP 2005-234686 A JP 2005-293539 A

However, in the conventional configuration, the process of obtaining the correspondence between the obtained feature quantity and emotion, that is,
Learning is hard. Learning is difficult for the entire pattern recognition, but in the case of facial images, each facial image is indexed manually, and the corresponding feature parameter, for example, a neural network, is used. The enormous amount of work that is determined by using it becomes necessary.
Especially in the case of emotions, for example, it is often impossible to clearly distinguish one type of emotion, such as a bitter smile, so it is necessary to prepare a large number of facial image samples so that they can be distinguished even when multiple emotions are mixed There is an enormous amount of work for creating an identification dictionary.
In addition, it takes time to match the feature quantity obtained from pattern recognition itself with the dictionary for identification, and in the case of emotion recognition, since multiple emotions are mixed, it takes a lot of time to collate. There are many issues to consider, which is one of the obstacles to system configuration.

The present invention has been made in view of the above-described circumstances, and an emotion recognition device, an electronic device, an emotion recognition method, and the like that can easily recognize an emotion from a face image without performing a large-scale learning process or dictionary creation, A control program and a recording medium are provided.

In order to solve the above-described problems, the present invention provides an emotion recognition apparatus including an input unit for inputting a face image and a “field” distributed around the face image, the strength of which is determined by the distance from the face image. And an emotion discriminating means for obtaining a visual guidance field having a larger value as it is closer to the face image and discriminating an emotion corresponding to the face image based on distribution information of the guidance field. According to the present invention, the visual guidance field of the face image is obtained, and the emotion corresponding to the face image is determined based on the distribution information of the guidance field, so that we do not perform large-scale learning processing or dictionary creation. Emotions can be discriminated in a way that is close to how you see and feel the object.

In the above configuration, the emotion discrimination means obtains the complexity of a closed surface of an equipotential line from the visual guidance field of the face image, and curve information that is a correspondence relationship between the complexity and the equipotential value is obtained as the guidance. It is preferable to acquire as field distribution information and discriminate emotion based on the curve information. According to this configuration, the curve information that is the correspondence relationship between the complexity and the potential value is acquired as the distribution information of the induction field, so that information that quantifies the distribution of the induction field can be easily obtained.

In the above-described configuration, the emotion discriminating unit is configured by comparing a function calculating unit that calculates a function that approximates the curve information, a function calculated by the function calculating unit, and a function corresponding to each emotion obtained in advance. It is preferable to have emotion determination means for determining emotion. According to this configuration, since the emotion is determined by comparing the function calculated by the function calculation means and the function corresponding to each emotion obtained in advance, it is possible to easily determine which emotion is the emotion. become.

In this case, it is preferable that the function specifying unit calculates a sigmoid function. According to this configuration, since it is approximated by a sigmoid function suitable for showing the visual guidance field distribution of the face image, it is possible to improve the determination accuracy of emotion recognition.

で定義されるシグモイド関数であり、前記感情判定手段は、前記シグモイド関数のパラ
メータに基づいて感情を判定することが好ましい。この構成によれば、シグモイド関数の
パラメータに基づいて感情を容易に判定することができる。 Further, the sigmoid function has a complexity C, a potential value p, a range a,
When the offset value is b and the parameters are T and p0,

It is preferable that the emotion determination unit determines an emotion based on a parameter of the sigmoid function. According to this configuration, emotion can be easily determined based on the parameters of the sigmoid function.

In the above configuration, it is preferable that the emotion determination unit determines an emotion by comparing the sigmoid function calculated by the function calculation unit with a sigmoid function corresponding to each emotion obtained in advance. According to this configuration, it is possible to easily determine which emotion is based on a comparison between the sigmoid function calculated by the function calculation unit and the sigmoid function corresponding to each emotion obtained in advance.
In this case, the emotion determination unit calculates one or a plurality of sigmoid functions similar to the sigmoid function calculated by the function calculation unit among the sigmoid functions corresponding to each emotion obtained in advance, and the similar sigmoid function If there is a single sigmoid function, it is determined as an emotion corresponding to that single sigmoid function, and if there are multiple similar sigmoid functions, it is determined that multiple emotions corresponding to the multiple sigmoid functions are mixed. You may make it do. According to this configuration, a state between emotions can be identified.

In the above configuration, the face image is preferably a binary image. According to this configuration, the amount of calculation can be reduced as compared with the case of emotion recognition of a multi-value image, and the determination speed can be increased. In addition, the present invention can be widely applied to electronic devices including an emotion recognition device.

Further, the present invention provides a visual guidance that is a “field” distributed around the face image in the emotion recognition method, the strength of which depends on the distance from the face image and has a larger value as the face image is closer. A field is obtained, and an emotion corresponding to the face image is recognized based on distribution information of the guidance field. According to the present invention, since the visual guidance field of the face image is obtained and the emotion corresponding to the face image is determined based on the distribution information of the guidance field, without performing a large-scale learning process or dictionary creation,
Emotions can be discriminated in a way that is close to how we see and feel our things.

In addition to being applied to the emotion recognition apparatus, electronic device, and emotion recognition method described above, the present invention makes it possible to download a control program for carrying out the present invention via an electric communication line, or to download such a program. The present invention can also be implemented in such a manner that it is stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, or a semiconductor recording medium and distributed.

According to the emotion recognition device, the electronic device, the emotion recognition method, the control program, and the recording medium according to the present invention, the visual guidance field of the face image is obtained, and the emotion corresponding to the face image is obtained based on the distribution information of the guidance field. Because we discriminate, we do not have to do large-scale learning processing and dictionary creation,
Emotion can be discriminated in a form close to how it is felt.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to an emotion recognition apparatus that recognizes emotion from facial expression is described as an example. In this emotion recognition device 10, in order to recognize emotions using the concept of “visual guidance field”, the visual guidance field will be described first.
The visual induction field is a psychological concept that explains the visual perception phenomenon such as pattern recognition, assuming a field like an electrostatic field around the figure. “Psychology of shape” by Yoshimasa Yokose (Nagoya University) Publication (
1986)) (hereinafter referred to as a reference paper).
In this reference paper, the distribution of visual guidance fields (hereinafter simply referred to as induction fields) is related to how we see and feel our objects, such as the similarity of letters and the interpretation of optical illusions. Yes. In this reference paper, since the target is a figure composed of straight lines and arcs, an induction field for an arbitrary digital image is not required. Here, a method for calculating an induction field in a black and white binary digital image (hereinafter referred to as a binary image) will be described first.

Since the induction field can be basically interpreted as a Coulomb potential, the pixels constituting the outline of the pattern are assumed to be point charges, and the distribution of the induction field in the digital image is calculated from the accumulation of the Coulomb potential generated by them.

FIG. 1 is a diagram showing a pixel arrangement of a digital image. As shown in FIG. 1, it is assumed that an induction field is formed at an arbitrary point P by a curve f (s) composed of n point sequences. This curve f
(S) corresponds to a line segment of a line figure or an outline of an image figure. Then, each point p1, p2,..., Pi,..., Pn constituting the curve f (s) is assumed to be a point charge having a positive charge 1, and the curve f (s) is scanned from the point P. N points p1, p2,..., Pi,.
..., Where pn is found and the distance to each point on the curve f (s) found by scanning is ri, the strength Mp of the induction field at the point P is defined as follows.

この式（２）を用いることにより、任意のディジタル画像の誘導場を求めることができ
る。また、曲線が複数ある場合、点Ｐにおける誘導場の強さは個々の曲線が点Ｐにつくる
誘導場の和になる。なお、式（２）は点Ｐから発した光が直接当たる部分のみ和をとると
いう制約条件がつく。例えば、点Ｐに対して、曲線ｆ１（ｓ），ｆ２（ｓ），ｆ３（ｓ）
が図２に示すように存在しているとすると、点Ｐから見えない部分、つまり、この場合、
曲線ｆ１（ｓ）に遮蔽されて点Ｐから見えない範囲Ｚに存在する部分の和はとらない。こ
の図２の例では、曲線ｆ３（ｓ）のすべてと曲線ｆ２（ｓ）の一部の和はとらないことに
なる。これを、ここでは遮蔽条件という。

By using this equation (2), the induction field of an arbitrary digital image can be obtained. When there are a plurality of curves, the strength of the induction field at the point P is the sum of the induction fields created by the individual curves at the point P. It should be noted that Equation (2) has a constraint that only the portion directly irradiated with light emitted from the point P is summed. For example, for the point P, the curves f1 (s), f2 (s), f3 (s)
Is present as shown in FIG. 2, the part that cannot be seen from the point P, that is, in this case,
The sum of the portions present in the range Z that is shielded by the curve f1 (s) and cannot be seen from the point P is not taken. In the example of FIG. 2, the sum of all of the curve f3 (s) and a part of the curve f2 (s) is not taken. This is referred to herein as a shielding condition.

FIG. 3A shows an example of the induction field calculated by the above-described equation (1) assuming that all the pixels are point charges of charge 1 for the letter “A”. A thin line distributed in contour lines on the map around the letter “A” in FIG. 3A is an equipotential line drawn by connecting the equipotential values in the induction field. The strength (potential value) becomes weak and eventually approaches 0 (zero).
The characteristics in the shape and strength of the distribution of the induction field in FIG. 3A, particularly the characteristics near the apex of “A” are sharper than others, are related to the vicinity of the corner of the figure such as a rectangle or a triangle according to the aforementioned reference paper. It agrees with the result of psychological experiment on the distribution of the induction field.

FIG. 3B shows an induction field in which all the pixels are assumed to be a point charge of charge 1 without the above-described shielding condition (the sum of the portions existing in the range Z invisible from an arbitrary point P is not taken). For example,
The distribution of the induction field is rounded as a whole, which is different from the result of the psychological experiment by the reference paper mentioned above. Thus, the shielding condition is important in characterizing the induction field.

In this way, a visual guidance field for a certain figure can be obtained. Examples of techniques using such visual guidance fields include, for example, “Michihiro Nagaishi:“ Easy-to-read Japanese proportional display using visual guidance fields ”, Journal of the Institute of Image Media and Technology, Vol. 52, No.
. 12, pp. 1865-1872 (1998) (hereinafter referred to as the first paper), Masazumi Miyoshi, Yoshifumi Shimoshiro, Hiroaki Koga, Ken Ideguchi: “Design of character layout based on sensibility using the visual induction field theory”, Electronics IEICE Transactions, 82-A, 9, 1465-1473 (1999)
”” (Hereinafter referred to as the second paper). Incidentally, the author of the first paper mentioned above is the inventor of the present invention.

In this embodiment, by using such a visual guidance field, it is possible to evaluate the sensitivity of images that have been relied on human intuition and manual work, and more specifically, visual guidance of facial images. By evaluating the field distribution, it is possible to automate emotion recognition from facial expressions.
More specifically, in this embodiment, the complexity indicating the degree of unevenness of the closed surface of the equipotential line is obtained from the induction field of the face image to be emotion recognition. This complexity can be obtained by the following equation (3) when the complexity of a closed curve having an equipotential value i is represented by Ci.

ここで、Ｌｉは、等ポテンシャル値ｉの等ポテンシャル面の周囲長であり、Ｓｉは面積
である。なお、周囲長Ｌｉは、等ポテンシャル面の輪郭を構成するドット数と考えること
ができ、面積Ｓｉは、等ポテンシャル面に存在するドット数と考えることができる。

Here, Li is the perimeter of the equipotential surface having the equipotential value i, and Si is the area. The peripheral length Li can be considered as the number of dots constituting the contour of the equipotential surface, and the area Si can be considered as the number of dots existing on the equipotential surface.

According to this equation (3), the longer the circumference length Li and the smaller the area Si, the larger the value of the complexity Ci, that is, the greater the equipotential line unevenness, the greater the value of the complexity Ci.
Then, a characteristic curve is shown by graphing the complexity Ci and the equipotential value i (hereinafter, the potential value that is not limited to the equipotential value i). Can be charted.

In the case of a face image, the characteristic curve that is “distribution of the visual induction field of the face image” (distribution information) is approximately a monotonically increasing function, and this function, that is, “the distribution of the visual induction field of the facial image” The curve shown can be approximated by a sigmoid function.
FIG. 4 shows an example of the sigmoid function. In particular, the sigmoid function has the symbol α
And the entire curve including the rising portion is very similar to the tendency of the characteristic curve indicating the correspondence between the complexity C and the potential value p in the case of a face image.
For this reason, in the present embodiment, by obtaining a sigmoid function as curve information indicating the correspondence relationship between the complexity C and the potential value p, that is, as “the distribution of the visual induction field of the face image” (distribution information), Information obtained by quantifying the “distribution of the visual induction field of the face image” can be easily obtained.

Next, an outline of a method for calculating the “distribution of the visual induction field of the face image” for each emotion will be described. Here, the case where the sigmoid functions of “anger”, “normal”, and “joy” are obtained from basic emotions will be described as an example.
FIG. 5A shows an example of a face image of an angry expression, and FIG. 5B is a diagram showing a guidance field of the face image. FIG. 5C shows an example of a face image having a normal expression, and FIG. 5D shows a guidance field of the face image. FIG. 5E shows an example of an angry face image, and FIG. 5F shows a guidance field of the face image. Note that these face images show extremely simplified image examples of only the face parts of the eyes, nose and mouth for easy understanding.
When calculating the sigmoid functions of “anger”, “normal”, and “joy”, first, FIG.
(C) For the face images of “anger”, “normal” and “joy” shown in (D), FIG.
) (F) As shown in (F), each induction field is calculated, and for each induction field, the complexity Ci is obtained for each equipotential value i, and the relationship between the complexity C and the potential value p is calculated using the least square method. And
It is obtained by approximating each with a sigmoid function expressed by the following equation (4).

ここで、ａはレンジであり、ｐ０及びＴはパラメータであり、ｂはオフセット値である
。

Here, a is a range, p0 and T are parameters, and b is an offset value.

The face image data used for the above calculation is obtained by taking an edge of image data (image having a plurality of colors and a plurality of gradations such as a color image or a gray scale image) at the time of shooting as an original image, And binarized image data binarized into black and black.
6A and 6B show face images obtained by binarizing the original image with different threshold values. Specifically, at the time of the above binarization, the threshold value is set to such a degree that main face parts such as the eyes, nose and mouth of the face are not lost, and as shown in FIG. It is preferable to convert the image into a binary image from which parts other than the facial parts are removed as much as possible. This binarized image is further subjected to noise removal processing and isolated point removal processing, and finally, as shown in FIG. 6C, a binary image in which only eyebrows, eyes, nose, and mouth are extracted. Is converted to The binary image may be an image in which only eyes, nose, and mouth can be distinguished. Thereby, a binary image obtained by extracting main face parts is obtained.

An example of the sigmoid function of the face image of the six basic emotions (normal, sadness, disgust, joy, anger, surprise) obtained in this way is shown in FIG. In FIG. 7, a curve h1 (p) represents a sigmoid function of a face image having a “normal” expression, and a curve h2 (p) represents “sadness”.
The sigmoid function in the case of, the curve h3 (p) indicates the sigmoid function in the case of "disgust", the curve h4 (p) indicates the sigmoid function in the case of "joy", and the curve h5 (p) "
The sigmoid function in the case of “anger” is shown, and the sigmoid function in the case where the curve h6 (p) is “surprise” is shown.

Considering the curve h1 (p) in the case of “normal” as a reference, the rises of the strong emotional curves h5 (p) and h6 (p) such as “anger” and “surprise” are steep, and the curve of “joy” h4 (p
) Complexity C is slightly larger, negative emotional curve h3 such as “disgust” or “sadness”
It can be seen that the characteristic (shape) of the curve differs depending on the emotion such that the complexity C of (p) and h2 (p) is small.
In this way, in view of the fact that the sigmoid function indicating “the distribution of the visual induction field of the face image” is different for each emotion, in this embodiment, the “distribution of the visual induction field of the face image” is approximated by a sigmoid function. The emotion can be easily estimated by comparison with a sigmoid function indicating the “distribution of the visual induction field of the face image” corresponding to each emotion obtained in advance.

For example, “anger” indicates that the above-described parameter T is a value around 0.04, and the above-described range a
Can be specified such as 300 or more, "anger" depending on whether this condition is met
It is possible to determine whether or not it is an emotion. Actually, the sigmoid function curves may be slightly different even for the same emotion, so the range of the sigmoid function for each emotion can be calculated by calculating the induction field of face images of multiple types of emotions in advance and plotting them for complexity. Is preferably obtained. By this work, for example, the range of a certain emotion I is determined as conceptually shown in FIG. 8A, and the parameter range of the emotion I is determined as shown in FIG. 8B. Similarly,
Other emotional II parameter ranges can also be defined. In this embodiment, the parameter range for each of a plurality of emotions is stored in the internal memory, and the emotions can be accurately discriminated by determining which parameter range they belong to.

Further, in this configuration, as shown in FIG. 8A, since the range for each emotion is specified by a two-dimensional plane of complexity C and parameter p, the range is divided around an area having a high appearance density. Thus, it can be easily classified into a sufficiently accurate range.
On the other hand, generally, in pattern recognition, it is necessary to partition an n-dimensional space (n = 60 or more). Therefore, it is necessary to collect a large number of samples without bias and to determine the partitioning well by machine learning. The work becomes complicated and the learning process takes time. Therefore, in this configuration, the division work of the range for each emotion is overwhelmingly simplified compared to the case of general pattern recognition, and the range for each emotion is easily and in a short time without learning processing. Can be determined.

FIG. 9 is a block diagram illustrating a functional configuration of the emotion recognition apparatus 10 according to the present embodiment. This emotion recognition device 10 is a device that calculates the induced field of an electronic face image, determines emotion based on the distribution of the induced field, and displays the result.
More specifically, the emotion recognition apparatus 10 recognizes a face image from a face image input unit (input means) 11 for inputting an electronic image to be determined and an image input to the face image input unit 11. The face region extraction unit (region extraction unit) 12 to be extracted, the emotion discrimination unit (emotion discrimination unit) 13 that discriminates emotions from the face area extracted by the face region extraction unit 12, and the discrimination results of the emotion discrimination unit 13 The display part (output means) 14 which displays is provided.

The face image input unit 11 inputs images of a plurality of colors and gradations (hereinafter referred to as multi-value images),
The image is converted into a binary image by performing the above-described binarization processing that binarizes with a threshold value such that major facial parts such as the eyes, nose, and mouth of the face are not lost. As an image input method to the face image input unit 11, the image may be input by wireless or wired communication, or may be input by reading image data recorded on a recording medium. Further, the face image input unit 11 may have a shooting function and directly input image data obtained by shooting.
Note that the face image input unit 11 may directly input a binary image binarized with a threshold value such that main face parts such as the eyes, nose, and mouth of the face are not lost.

The face area extraction unit 12 recognizes a face area using a known face recognition technique such as a skin color utilization method or a face part detection method. The skin color utilization method is a method for recognizing a skin color part as a face area based on the color distribution by utilizing the fact that the face image is a skin color, and the face part detection method is based on the edge in the image. This is a method of detecting a facial part (eye, nose, mouth, etc.) and recognizing an area with the facial part as a facial area.

The emotion determination unit 13 includes a guidance field calculation unit (calculation unit) 21, a guidance field distribution evaluation unit 22, and a determination unit (determination unit) 23. The guidance field calculation unit 21 calculates the guidance field of the face area extracted by the face area extraction unit 12.
The guidance field distribution evaluation unit 22 obtains and evaluates “visual guidance field distribution” from the guidance field calculated by the guidance field calculation unit 21. Specifically, the induction field distribution evaluation unit 22 obtains the complexity Ci for each equipotential value i by the equation (3) for the visual induction field, and approximates the relationship between the complexity C and the potential value p. Each sigmoid function is calculated. Thus, the guidance field distribution evaluation unit 22 functions as a function calculation unit that calculates a sigmoid function (face image function) indicating “visual guidance field distribution”.

The determination unit 23 determines the emotion by comparing the sigmoid function calculated by the induction field calculation unit 21 and the sigmoid function corresponding to each of the emotions obtained in advance (for example, the six basic emotions described above). It functions as a determination means. As described above, the guidance field calculation unit 21, the guidance field distribution evaluation unit 22, and the determination unit 23 included in the emotion determination unit 13 can be configured by a calculation unit that performs the calculation process described above. Therefore, in actuality, each unit of the emotion discrimination unit 13 may be configured by one or a plurality of semiconductor integrated circuits that perform the above arithmetic processing by hardware processing, or a CPU, ROM, or RAM that performs software processing. It may be configured by a general-purpose computer such as, or relatively heavy calculation processing such as visual guidance field calculation processing is performed by hardware processing, and relatively light calculation processing such as emotion determination is performed by software processing. You may comprise.

Next, the processing flow of the emotion recognition apparatus 10 will be described with reference to the flowchart shown in FIG. First, when a face area is extracted from the input binary image by the face area extraction unit 12, the guidance field calculation unit 21 calculates a guidance field of the face area (step S11). The induction field distribution evaluation unit 22 starts a process of calculating the complexity C for each equipotential surface of the calculated induction field.
Here, in the calculation definition formula of the induction field defined in the above formula (2), when the minimum pixel distance is 1, the field strength is in the range of 0 (zero) to 1. As the range of the potential value p for calculating the complexity C increases, the approximation accuracy of the later sigmoid function increases. From the viewpoint of shortening the calculation time,
It is preferable to keep the minimum range useful for emotion recognition.
Therefore, in the present embodiment, when calculating the complexity C, the potential value p is an appropriate range in a section of 0 (zero) or more and less than 1, for example, the minimum value p1 of the potential value p is set to 0.03.
And the amount of calculation is reduced by setting the resolution value Δp to 0.01 steps.

Specifically, the induction field distribution evaluation unit 22 first sets the potential value p to the minimum value p1 (step S12), and extracts an equipotential surface of the potential value p (= p1) (step S13). The perimeter length Li and the area Si of the closed curved surface having the equipotential value i (= p1) are obtained, and the complexity Ci is calculated by the equation (3) (step S14).
Subsequently, if the potential value p is less than the maximum value “1” (step S5: YES), the induction field distribution evaluation unit 22 adds the resolution value Δp to the potential value p (step S16).
By repeating the equipotential surface extraction process of step S3 and the calculation process of complexity C of step S14, the potential value p is complex in units of resolution Δp in the range from the minimum value p1 to the maximum value “1”. Calculate degree C. Note that the maximum value may be changed to a value equal to or less than the value “1” without being limited to the case where the complexity c is calculated up to the range of the maximum value “1”.

When the potential value p reaches the maximum value “1” (step S15: NO), the induced field distribution evaluation unit 22 determines the obtained complexity C and the potential value p corresponding to each complexity C. From these, parameters (range a, offset value b, parameters p0, T) for determining the sigmoid function shown in Expression (4) are determined using the least square method, and an approximate sigmoid function is obtained (step S17). In the process of step S17, the error variance, that is, the approximate error is obtained by calculating the sum of squares of the error (residual) from the theoretical value.

In this case, the approximation error corresponds to information indicating whether the image is a face-like image. For this reason, if the approximation error is large, that is, the correlation of the function is very low and it is difficult to approximate with a sigmoid function, it is considered that the target is not a face, or the facial expression cannot be recognized because the image is very deteriorated. It is done. The specific threshold may be determined experimentally using an actual sample. In such a case, the determination unit 23 notifies the user of a warning by displaying on the display unit 14 that recognition is difficult.

On the other hand, when the approximation error is small, the determination unit 23 calculates one sigmoid function similar to the approximated sigmoid function among the sigmoid functions obtained for each emotion in advance, that is, the parameter (range) of the approximate sigmoid function. a, offset value b, parameters p0, T, etc.)
Emotion is determined by determining which of the parameter ranges for each emotion stored in the internal memory corresponds (step S18), and the determination result is displayed as an image or message on the display unit 14
It is possible to notify the user of the emotion (expression) of the face image.

As described above, according to the present embodiment, the visual guidance field of the face image is obtained, the distribution information of this guidance field is approximated by a sigmoid function, and the emotion corresponding to the face image based on the approximated sigmoid function Because it is possible to discriminate emotions (facial expressions) in facial images in a way that is close to how we see and feel our objects, and it is also possible to easily recognize emotions from the approximate sigmoid function. Can be made faster (shorter time).
Moreover, since it approximates with the sigmoid function suitable for showing the distribution of the visual induction field of the face image,
The determination accuracy of emotion recognition can be improved.

In this case, the emotion is determined by comparing the approximated sigmoid function and the sigmoid function corresponding to each emotion obtained in advance, and more specifically, the parameters of the sigmoid function belong to any emotion parameter range. Since the emotion is discriminated based on whether or not it is, the task of calculating the correspondence between the parameter and the emotion is a two-dimensional plane identification task. For this reason, the number of samples does not have to be enormous, and even when the calculation operation is performed using a neural network or the like, the processing time can be significantly reduced.
Thereby, in this embodiment, compared with general pattern recognition, since it is not necessary to perform a large-scale learning process and dictionary creation, there is an effect that less pre-recognition work is required.

In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention. For example, in the above embodiment,
Although the case where the present invention is applied to the case where emotion recognition is performed on a face image of a binary image has been described, the present invention is not limited thereto, and the present invention can also be applied to the case where emotion recognition is performed on a face image of a multi-value image.

Hereinafter, the case where emotion recognition is performed on a face image of a multi-valued image will be described. In general, many digital devices adopt R (red), G (green), and B (blue) as basic colors, and the color is expressed by a combination of RGB. Note that each of RGB changes from 0 to 255, and a color is expressed by a combination thereof. By the way, black is R
= G = B = 255, white is a combination of R = G = B = 0, and a combination of R = G = B having an intermediate value thereof is an achromatic color (gray). In this way, not only colors but also gradations can be expressed by RGB.
In this case, the calculation is performed as follows by applying the technique of Japanese Patent Application Laid-Open No. 2004-171115 (hereinafter referred to as a reference technical document). More specifically, in FIG. 1, each point p1, p2,
.., Pi,..., Pn are affected by the gradations of R, G, B (for example, 0 to 255). Mp is defined as in equation (5).

Here, Qi (R, G, B) is an independent function (Qi (R), Qi (G
), Qi (B)), and in the case of a binary image, Qi (R = 0, G = 0, B = 0)
= 1, and in the case of a multi-valued image, Qi (R, G, B) is larger than 1 (Qi> 1). These Qi (R), Qi (G), and Qi (B) are curves that draw a substantially S-shaped curve that saturates when the gradation (density) reaches a certain value according to the above-mentioned reference technical literature. It is also known that R (red) is the most sensitive to changes in gradation (density), followed by B (blue) and G (green).
This is consistent with the fact that, for example, when a warning sign is displayed on a traffic sign or the like, colors are used in the order of red and blue, and green is often not used. The magnitude of such a degree of attention is considered to be the strength and energy of the induction field. On the basis of this, the difference in change due to the color of Qi is consistent with the traffic sign example described above. . Therefore, the function for obtaining Qi used in this equation (5) can be determined by psychological experiments or the like.

Therefore, the induced field of the multi-valued image can be calculated by using Expression (5). And if this induction field is decided, the complexity C is calculated by the process similar to the above-mentioned embodiment,
The correspondence relationship between the obtained plurality of complexity C and potential value p is approximated by a sigmoid function, and emotion is determined from the face image based on approximation error and parameters (range a, offset value b, parameters p0, T). be able to.
As described above, when emotion recognition is performed on a face image of a multi-valued image, it is not necessary to convert the multi-valued image into a binary image. Therefore, the determination accuracy can be improved as there is no information loss during such conversion. .

However, when emotions are recognized from a multi-value image, the amount of calculation increases and the calculation time becomes longer. For this reason, when priority is given to determination speed, emotion recognition is performed by converting to a binary image, and when priority is given to determination accuracy, which emotion recognition is to be performed, such as emotion recognition with a multi-value image. It may be selectable.

In the above-described embodiment, as a function for a face image indicating the distribution of the visual induction field of the face image,
Although the case of using the sigmoid function has been described, the present invention is not limited to this, and other functions that can express the visual induction field of the face image may be applied.
In the above-described embodiment, a case has been described in which one emotion is determined by calculating one sigmoid function that is similar to the approximate sigmoid function among sigmoid functions corresponding to each emotion obtained in advance. However, the present invention is not limited to this, and when there are a plurality of sigmoid functions similar to the approximate sigmoid function, a plurality of emotions corresponding to the plurality of sigmoid functions are mixed, for example, `` anger '' and `` surprise '' You may make it determine with the state which mixed. In this case, among the sigmoid functions corresponding to the emotions obtained in advance, whether or not there is one or a plurality of sigmoid functions similar to the approximated sigmoid function may be determined by parameter comparison. According to this configuration, a state between emotions can be identified.

In addition, the present invention creates a control program in which the processing procedure for carrying out the present invention described above is described, and makes this control program downloadable via a telecommunication line, The present invention can also be implemented in such a manner that the program is stored and distributed in a computer-readable recording medium such as a recording medium, an optical recording medium, or a semiconductor recording medium.
It should be noted that the emotion recognition apparatus according to the present embodiment can be implemented in any electronic device such as a camera, scanner, projector, television, or printer. For example, a camera equipped with the emotion recognition device described above recognizes an emotion corresponding to the face image to be captured, and performs image correction based on the recognized emotion, such as performing bright color correction when it is recognized as a pleasant emotion. It becomes possible to execute. In addition, a printer including the emotion recognition device described above recognizes an emotion corresponding to a face image included in the image to be printed, and if it is recognized as a pleasant emotion, automatically adds a pleasant frame to the image. Thus, it is possible to process and print an image based on the recognized emotion.

It is a figure which shows the pixel arrangement | sequence of the digital image for demonstrating the visual induction field. It is a figure explaining the shielding conditions at the time of calculating | requiring the intensity | strength of a visual induction field. (A) is a figure which shows the case where the visual guidance field of character "A" is calculated | required in consideration of shielding conditions, (B) is the figure which shows the case where the visual guidance field is calculated | required without considering shielding conditions. It is. It is a figure which shows an example of a sigmoid function. (A) is a figure which shows an example of the face image of an angry expression, (B) is a figure which shows the induction field of (A), (C) is a figure which shows an example of the face image of a normal expression. Yes, (D) is a diagram showing the guidance field of (C), (E) is a diagram showing an example of a facial image of a joyful expression, and (F) is a diagram showing the guidance field of (E). is there. (A) is a figure which shows the binary image of the face from which the part except main face parts was not removed much, (B) is the face binary image from which the part except main face parts was removed as much as possible. (C) is a figure which shows the binary image which extracted the main face components from the figure of (B). It is a figure which shows an example of the sigmoid function of the face image for every emotion. (A) is a figure which shows the range for every emotion, (B) is a figure which shows the parameter range for every emotion. It is a block diagram which shows the function structure of the emotion recognition apparatus which concerns on this embodiment. It is a flowchart which shows the processing flow of an emotion recognition apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Emotion recognition apparatus, 11 ... Face image input part (input means), 12 ... Face area extraction part (area extraction means), 13 ... Emotion discrimination part (emotion discrimination means), 14 ... Display part (output means), 21 ... guidance field calculation part (calculation means), 22 ... guidance field distribution evaluation part (function calculation means), 23 ... determination part (determination means).

Claims (12)

An input means for inputting a face image;
A “field” distributed around the face image, the strength of which depends on the distance from the face image,
An emotion recognition apparatus comprising: an emotion discriminating means for obtaining a visual guidance field having a larger value as it is closer to a face image, and discriminating an emotion corresponding to the face image based on the distribution information of the guidance field. The emotion recognition apparatus according to claim 1,
The emotion discriminating means obtains the complexity of the closed surface of the equipotential line from the visual induction field of the face image, and obtains curve information that is a correspondence relationship between the complexity and the equipotential value as distribution information of the induction field. And recognizing emotion based on the curve information. The emotion recognition apparatus according to claim 2,
The emotion discrimination means includes
Function calculating means for calculating a function approximating the curve information;
An image recognition apparatus comprising: an emotion determination unit that determines an emotion by comparing a function calculated by the function calculation unit and a function corresponding to each emotion obtained in advance. The emotion recognition device according to claim 3,
The emotion recognition apparatus, wherein the function specifying means calculates a sigmoid function. The emotion recognition device according to claim 4,
When the complexity is C, the potential value is p, the range is a, the offset value is b, the parameters are T, and p0,

Is a sigmoid function defined by
The emotion recognition apparatus characterized in that the emotion determination means determines an emotion based on a parameter of the sigmoid function. The emotion recognition apparatus according to claim 5,
The emotion recognition apparatus, wherein the emotion determination unit determines an emotion by comparing the sigmoid function calculated by the function calculation unit with a sigmoid function corresponding to each emotion obtained in advance. The emotion recognition apparatus according to claim 6,
The emotion determination unit calculates one or a plurality of sigmoid functions similar to the sigmoid function calculated by the function calculation unit among the sigmoid functions corresponding to each emotion obtained in advance, and there is one similar sigmoid function. If there is only one sigmoid function, it is determined that the emotion corresponds to that one sigmoid function, and if there are multiple similar sigmoid functions, it is determined that a plurality of emotions corresponding to the plurality of sigmoid functions are mixed. A feature emotion recognition device. The emotion recognition apparatus according to any one of claims 1 to 7,
The emotion recognition apparatus, wherein the face image is a binary image.   An electronic device comprising the emotion recognition device according to claim 1. A “field” distributed around the face image, the strength of which depends on the distance from the face image, and a visual induction field having a larger value is obtained as the face image is closer. An emotion recognition method characterized by recognizing an emotion corresponding to the face image. Computer
An input means for inputting a face image;
A “field” distributed around the face image, the strength of which depends on the distance from the face image,
A control program for obtaining a visual induction field having a larger value as it is closer to a face image, and functioning as an emotion determination unit that determines an emotion corresponding to the face image based on distribution information of the induction field. Computer
An input means for inputting a face image;
A “field” distributed around the face image, the strength of which depends on the distance from the face image,
A computer in which a control program for obtaining a visual guidance field having a larger value as it is closer to the face image and functioning as an emotion discrimination means for discriminating an emotion corresponding to the face image based on the distribution information of the guidance field is recorded A readable recording medium.

Images