KR20150003585A - Method for estimating emotion by measuring micro movement of human body - Google Patents

Abstract

The present invention relates to an emotion classifying method according to human body micro movement analysis. The present invention includes the steps of: photographing the body of a subject; detecting the human body micro movement quantity signal of the subject from images captured on a specific frequency band; and specifying the emotion state of the subject by analyzing the micro movement quantity signal. The emotion classifying method of the present invention detects the micro movement quantity of the human body on the specific frequency band and determines the emotion state of the subject by using the micro movement quantity signal of a specific part.

Description

[0001] The present invention relates to a method for estimating emotion by using human micro-

The present invention relates to a sensibility determination and analysis method for extracting a signal capable of specifying human emotion from a minute motion of the human body, that is, a slight motion of the human body.

To date, various physiological signals such as electrocardiography (ECG), photoplethysmography, galvanic skin response (GSR), skin temperature (SKT), and electroencephalography (EEG) Was applied [1].

Generally, when using ECG or PPG, the heart rate can be determined by analyzing consecutive intervals between pulses [2]. The amplitude levels of GSR and SKT are also parameters that measure the relative skin response and skin temperature. In particular, EEG data can be interpreted using various methods in the time domain or the frequency domain.

However, the above-described method must take the inconvenience of measurement caused by the attachment of sensors or the like. It also causes negative emotions that can act as a noise factor in correct emotional measurements.

N. Wu, H. Jiang, and G. Yang, "Emotion recognition based on physiological signals," Lec-ture Notes in Computer Science, vol. 7366, pp. 311-320, 2012. F. C. Chang, C. K. Chang, C. C. Chiu, S. F. Hsu, and Y. D. Lin, "Variations of HRV anal- ysis in different approaches," In Proc. of Computers in Cardiology, pp. 17-20, Oct. 2007. http://www.livescience.com/15469-cardiocam-mirror-mit-siggraph.html, accessed on 17th March, 2013. MJ Gregoski, M. Mueller, A. Vertegel, A. Shaporev, BB Jackson, RM Frenzel, SM Sprehn, and FA Treiber, "Development and Validation of a Smartphone Acquisition Application for Health Promotion and Wellness Telehealth Application, International Journal of Telemedicine and Applications, vol. 2012, Article ID 696324, 7 pages, Oct. 2012. P. Han and J. Liao, "Face detection based on Adaboost," In Proc. of Apperceiving Compu-ting and Intelligence Analysis, pp. 373-340, Oct. 2009. http://opencv.org/ (Accessed on 31st Mar. 2013) S. Park, D. Ko, M. Whang, E. C. Lee, " Vision based body dither measurement for estimat- ing human emotion parameters, "Lecture Notes in Computer Science, vol. 8008, pp. 346-352, 2013.

The present invention proposes a sensibility recognition method capable of determining a sensibility state of a subject from the human motion.

The emotion recognition method according to the present invention:

Imaging a body of a subject;

Detecting a subject's anamorphic signal from images of the subject in a specific frequency band from the captured image;

And analyzing the asymmetric signal to specify the emotional state of the subject.

According to an embodiment of the present invention, the body moving amount signal can be detected from the upper body of the subject.

According to another embodiment of the present invention, an asymmetry signal for a specific region in the body can be detected.

According to another embodiment of the present invention, the Adaboost method can be applied in finding the asymmetry signal for a specific part of the body.

According to still another embodiment of the present invention, an asymmetric frequency of another specific frequency band may be removed from the asymmetric signal of the specific frequency band.

According to another embodiment of the present invention, an asymmetric signal in the specific frequency band is generated according to the unconscious motion of the subject, and the signal of the other specific frequency is generated according to conscious movement of the subject.

The present invention does not apply the attachment type sensor in the conventional method in classifying the emotional state. Therefore, it eliminates the physical and psychological burden of subjects with sensor attachment. Further, since the emotional state can be classified in a state in which the subject of the present invention is unconscious, a more accurate emotional state can be classified or specified. Therefore, the present invention can be applied to a specific method and apparatus for emotional evaluation, because the emotional state of the subject can be determined regardless of the intention of the subject.

Fig. 1 illustrates various fine motions defined by human body parts.
Fig. 2 shows an example of detection of an asymmetric measurement area from a subject according to the present invention.
FIG. 3 illustrates a method of extracting an asymmetric amount in each frequency band by capturing a continuous image (image) obtained from a human body in a plurality of frequency bands.
FIG. 4 shows an asymmetric signal for various frequency bands extracted according to the present invention.
FIG. 5 shows the results of fine detection in body-specific regions in the asymmetric detection according to the present invention.
FIG. 6 is a graph comparing the average of the asymptotic amounts detected from intimate groups and non-intimate groups according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, various embodiments of the emotional determination method according to the present invention and the background art of the present invention will be described with reference to the accompanying drawings.

Recently, a physiological data acquisition method based on a camera image free from a general bio signal sensor has been proposed. Cardio-Cam has been proposed for human heart rate measurement by color channel analysis based on Independent Component Analysis (ICA) without any sensor attachment [3].

For the same purpose, a variety of smartphone aplications have been provided that can measure heart rate in real time using a built-in rear camera and white light [4]. In such an application, a change in the brightness level of successive images was observed because the degree of illumination reflections varies continuously and continuously by the blood flow. Although this method is meaningful in that it does not require sensor attachment, only heart rate decryption is possible by this method.

A variety of fine movements of the human body, that is, fine movements, represent a very important characteristic that a physiological signal can be obtained without attaching a sensor. The present invention defines a measurement of human motion that can identify the independence or dependency of the motion of each part of the body as shown in Fig.

As shown in FIG. 1, a hierarchical model expresses the dependency or dependency of fine movement between body parts. For example, the facial fine movement can be calculated by the summation of the facial fine movement and the facial fine movement.

In the hierarchical model shown in FIG. 1, the size of the motion is defined as Full body> (upper body (Bust)> arm = (eye = mouth)). In other words, the entire body is larger than any other, and the smaller upper body is larger than the arm, and the arms are the same as the face. And face is big compared to eyes and mouth, eyes are like mouth. It is not appropriate to measure the fine movement of the entire body using the camera. Therefore, the fine motion information of the upper body "bust", which is relatively small in movement but relatively higher than the other, is used as a parameter for evaluating the social sensibility pursued by the present invention Or < / RTI >

In order to measure the fine movement, ROI (Region of Interest) is first defined in the upper body image frame captured based on the facial part detected by the Adaboost (adaptive boosting) method [5].

Then, the amount of fine motion is calculated by subtracting two consecutive images. Here, depending on the interval (interval) of two image frames, the amount of fine motion per frequency band can be obtained. The results of the feasibility test for comparisons between intimate groups and non-intimate groups showed that the movements in the intimate group cases were less than those in the non-intimate groups.

Hereinafter, a detailed description will be given of a method which has been actually performed according to a specific measuring method according to the present invention, as an example.

First, the face region is detected in the first frame of the upper body image using a face detector (program) using the Adaboost method. At this time, the detection of the face area in all the frames is not necessary. This is because, according to the present invention, the difference between the luminance values of pixels at the same position in two adjacent frames is obtained and the amount of fine movement is extracted.

The Adaboost method uses a strong classifier generated by combining many weak classifiers to detect faces in the input image [5]. The algorithm requires a lot of learning time, but it has advantages such as short time required for detection and good quality detection result. This required an average of 29 msec (milliseconds) per image to detect the facial area in a decimated image of 1/4. Fig. 2 shows the face detection in the rectangular area "B" as a sample.

After detection of the facial region (red square) as shown in FIG. 2, a selected region (Candidate region (green square in FIG. 2) for subtracting the image to calculate the upper body fine movement is 160 Pixel.

In order to measure the same amount, a camera image analysis program was applied to the analysis. The captured color image was converted to grayscale in black and white. This is because the color information of the image is not important for the behavior analysis. In calculating the fine average amount "O" _{, the} average amount of micro movement at frequency "F" Band in the band where the frequency is arbitrary value "F" (O _FHz ) can be calculated by the following equation .

In Equation (1), W and H are the horizontal and vertical lengths (size or distance) of the upper body in the ROI. In ( i, j ) is a pixel value of the i-th column and the j-th row of the n-th image frame. R is the frame rate of the camera used is 15.

In the analysis method of the present invention, the concept of fine extraction according to the present invention is illustrated in Fig.

Referring to FIG. 4, the continuous average averages (O _{15 Hz} , O _{5 Hz} , O 3 _Hz , O _{1 Hz} , O _{0.5 Hz} ) extracted for each frequency band of _{15 Hz} , _{5 Hz} , _{1 Hz} , (Amplitude) signal on the one-dimensional time axis.

These signals can be analyzed using general signal analysis methods. In addition, the fine motion information can be analyzed for various frequency bands. The frequency analysis according to the present invention uses a method different from the previously proposed method [7]. For example, an intended large operation can be extracted in a low frequency band, and these areas have a role as a mask or filter region that can remove or reject fine measurement in a high frequency band .

Although the prior art background extraction method for object detection has a problem in continuous change of the background or complex background modeling, the present invention uses only the image of the last two frames, so it is independent of the change of the background.

In this invention, the asymmetric analysis can be applied to any body part classified in Fig. If a detection method for a specific body part is performed, its asymmetry or muscle movement can be measured and analyzed. For example, when the facial detection method is applied based on the above-described Adaboost, only the fine motion of the facial region can be analyzed. FIG. 5 is a diagram for testing the feasibility of fine detection, and shows various fine detection results by a program according to the Adaboost method. In Fig. 5, the bright region represents the fine-grained region. Specifically, in FIG. 5, (a) shows a state in which there is almost no motion, and therefore, all pixels are dark without any region having a high value. FIG. 5 (b) shows fine lines in the entire upper body. In the entire image, the upper body including the head and the chest shows a bright part, that is, a high pixel value. FIG. 5 (c) shows a state in which only the muscles of the forehead portion of the subject show fine motion. . FIG. 5 (d) shows a state in which fine motion is observed only at the mouth portion of the subject, FIG. 5 (e) shows the subject's fine motion due to eye blinking, and FIG. 5 (f) Respectively.

In the present invention, as shown in FIG. 5, only the fine motion of a specific part of the subject is detected and the emotional state of the subject can be classified using the same. This invention is based on the fact that the human mind is regarded as an intrinsic eigen response of social sensibility and allows social interaction, for example, conversation, at this time to be free from sensor attachment, The relationship between the subject group and the non-subject group can be compared and classified.

In order to verify the feasibility of the present invention, the following experiment was conducted. For this purpose, a group of subjects was asked to talk for twenty - five minutes with two groups of four groups. Here, the two sets have an intimate relationship, while the other two sets are unfamiliar (unfamiliar) relationships without social relations.

During the conversation, the body image was captured in 15 frames per second with a commonly used 640 * 480 resolution and 15 frames per second webcam.

The captured image frames were analyzed so that the tilts were detected in five frequency bands such as 0.5, 1, 3, 5, and 15 Hz (see FIG. 3).

The average fine quantities in the various frequency bands extracted through these experiments are shown in FIG. As shown in FIG. 6, the amount of fine tangent in the intimate social relation group was lower than that of the non-intimate group except for the 0.5 Hz band.

Also, it can be seen that more differences between the two groups appear in relatively high frequency bands. From these results, it could be assumed that the non - intimate social interaction induces a larger fine rhythm of the human body.

The present invention proposes a method of estimating social emotions by analyzing human movements. In addition, the present invention allows social sympathy without the burden of measurement such as sensor attachment, and provides an advantage of observing social emotions. According to the present invention, the fine motion of the body can classify the intimacy into two levels.

In carrying out the method according to the present invention, the fine motion of the human upper body was successfully analyzed using a continuous image frame captured using a conventional webcam. For this purpose, the upper body mass was measured by subtraction of two adjacent image frames. Since the measured successive upper body asymptotes (values) are in the form of one-dimensional time-axis intensity signals, all common time signal processing methods can be applied.

As a result, as described above, the similarity in the case of the intimate relationship was lower than that in the non-intimate emotional relationship. In light of these results, it would be possible to confirm the relationship between the meandering of body parts and various general physiological signaling responses. For example, after detecting an ECG signal and an upper body tilting for a particular temporal emotion, an analysis of the interrelation between the pulse interval and the upper body anomaly amount will be possible. It is also possible that the high frequency band signal is partially subjected to rejection filtering by masking of the large gesture part extracted from the low frequency band.

The present invention as described above is summarized as follows.

In the first step, it is necessary to continuously photograph the upper body or upper body of a subject lying in a particular emotional state. The upper body can be photographed by capturing the whole body and then capturing only the upper body part.

In the second step, the image obtained by continuous shooting is captured at a constant frequency to obtain a difference between two adjacent images. At this time, the difference may be obtained only for a specific region in the captured image. As described above, this difference is used to determine the difference in luminance between two adjacent images among the images continuously captured on a frame-by-frame basis. The difference in luminance of the same pixels in both images, as summarized in Equation 1, And accumulates for all the pixels to obtain a signal of a specific frequency band as shown in Fig. Here, it is possible to filter intentional asymmetry independent of the emotional state of the subject. As described above, the intentional dynamic amount occurs in a high frequency band, and therefore filtering to remove it from a signal in a low frequency band is desirable.

In the third step, the signal is analyzed to classify the emotional state of the subject. In sorting the emotional states, for example, the average of the asymptotic quantities is obtained as shown in FIG. 6, and the two-stage emotional state, for example, intimate or non-intimate, is classified based on a predetermined threshold value.

The present invention as described above is to obtain a characteristic signal appearing according to a human emotional state from a human human motion in a specific frequency band which is unconsciously generated, and it is possible to analyze various emotional states using the human signal. The present invention eliminates the sensor attached to the human body in order to specify the emotional state of the human being, so that the subject of emotional state analysis is freed from such burden, and the measurement can naturally take place without any burden on the task of emotional state analysis . In addition, the present invention can also be applied to a specific purpose, for example, for the purpose of forcibly judging the authenticity of a specific person's status, such as a polygraph, and in this case, Can proceed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention. Therefore, the true scope of protection of the present invention should be defined only by the appended claims.

Claims (6)

Imaging a body of a subject;
Detecting a subject's body motion amount signal from captured images in a specific frequency band from a captured image;
And analyzing the asymmetric signal to identify the emotional state of the subject. The method according to claim 1,
And the body motion amount signal is detected from the upper body of the subject. 3. The method of claim 2,
Wherein the detection signal is detected from a specific region selected from the body. The method of claim 3,
Wherein face detection by the Adaboost method is applied in detecting an asymmetry signal for a specific part of the body. 5. The method according to any one of claims 1 to 4,
And removing an analogous frequency of another specific frequency band from the analog signal of the specific frequency band. 6. The method of claim 5,
Wherein the signal of the specific frequency occurs according to the unconscious movement of the subject and the signal of the other specific frequency occurs according to the conscious movement of the subject.

Images