KR20190124470A - method for determining competition and cooperation by social interaction based on body movement by non-contact sensing technique - Google Patents

Abstract

Disclosed are a method and an apparatus for evaluating a cooperation-competition relationship which is a social relationship. The method comprises the steps of: photographing faces of two subjects in a state where the two subjects can communicate using a video camera; extracting fine motion data of the subject by detecting a change amount between frames in the face image; detecting fine motion data for each frequency band by the magnitude of the fine motion of the subject from the fine motion data; and evaluating a cooperation-competition relationship between the two subjects using fine motion data for each frequency band.

Description

Method for determining competition and cooperation by social interaction based on body movement by non-contact sensing technique}

The present invention relates to a method for evaluating competition and cooperation according to social interaction by contactless sensing technology, and more particularly, to a method for evaluating competition and cooperation according to social interaction using human body movement.

With the development of IT technology, users are easily accessing and using various information and contents. In the past, offline content such as newspapers and books was used. Today, online content using IT devices takes the place. Similarly, the forms of communication between people are expanding online as well as offline.

Human communication activities can be divided into simple information transfer and emotional communication. In addition, in the past, the information to be conveyed was passive and displayable information. In the present, the two-way interactive sympathy and emotional information are included. In particular, there is an increasing number of researches that exchange emotional and emotional information online and recognize and evaluate it. The existing research on emotion recognition has been conducted simply to recognize one's own emotion. However, as social relations become more wide-ranging and issues through online, research on emotional perception according to social relations is increasing.

Recently, research on the recognition of Social Emotion caused by the interaction with others has been conducted. Social sensibility is formed through various interaction activities between two people, and it means high-level emotions generated in the relationship. Such social sensibility is a representative example of intimacy, empathy and participation of two people [11, 12]. Emotion recognition methods were inferred using contact sensors to collect quantitative biometric data. However, there is a limit in collecting quantitative data of biological responses because wearing sensors does not allow free interaction and communication. In order to solve this problem, emotional recognition method using non-contact sensing technology is being studied.

Up to now, the emotional recognition technology using the external response of facial expressions is representative. However, emotion recognition using only facial expression can recognize only Ekman's six basic emotions, and there are some limitations in recognizing complex social emotions. Therefore, there is a need for a non-contact sensing technology based on an internal biological response.

Recently, a technique for sensing a biological response in a non-contact form has been studied. Previous studies have reported a study of inferences of heart reactions through facial skin changes or head movements. The reason why the biological response can be inferred in the non-contact form is because of the anatomical principles of the human body. From a neurotechnology perspective, the heart and brain are connected. The response of the heart is a response of the human body that cannot be controlled by the human body. The response of the heart is transmitted to the brain through afferent, and the response of the brain is to transfer controllable cognitive information to the human body through efferent. Maintain relationships. Anatomically, the vestibular organ (Vestibular organ) is involved in the internal physiological changes to the appearance of the external body movement response.

Specifically, the vestibular organ regulates the sense of balance of the human body. It is an organ that keeps the human body in balance by organically transmitting information to all body organs according to the human body situation such as redirection and standing. This organ is located in the neuropath that sends and receives autonomic and central nervous system information and directly affects the cardiovascular and respiratory system. It uses the vestibular reflexes to exchange information of various nervous systems. At this time, as the body maintains a sense of balance including a physiological response, minute movements are generated and the movements are expressed differently according to emotion and psychological state. Such movements are represented by very small tremors that are invisible to the human eye.

<Preceding technical literature>

[1] Y. S. Cha, J. H. Kim, J. H. Kim, S. Y. Kim, D. K. Kim & M. Whang, “Validity analysis of the social emotion model based on relation types in SNS”, J. of the Korea Emotion and Sensibility, Vol. 15, No. 2, pp. 283-296, 2012.

[2] S. Shin, M. Shin, Y. Jeong & J. Lee, “An investigation of Social Commerce Service Quality on Consumer's Satisfaction”, J of IT Convergence Society for SMB, Vol. 5, No. 2, pp. 27-32, 2015.

[3] H. Kim & P. Jang, “Differences in Sentiment on SNS: Comparison among Six Languages”, J of the Digital Convergence, Vol. 14, No. 3, pp. 165-170, 2016. [4] E Jung, J. Kim, H. J & K. Chung, Social Network based Sensibility Design Recommendation using {User? Associative Design} Matrix ", J of the Digital Convergence, Vol. 14, No. 8, pp. 313-318,2016.

[5] J. Ahn & S. Kim, “Comparison the Difference of User Experience for Mobile Facebook and Instagram Using Nonparametric Statistics Methods-Focused on Emotional Interface Model-", J of the Digital Convergence, Vol. 14, No. 11, pp. 481-488,2016.

[6] D. K. Lee & Y. D. Yoon, “Study of Emotional Communication Strategy of Storytelling through Social Media-Based on the" Bear and Hare "Commercial of John lewis", J of the Korea Contents Association, Vol. 16, No. 11, pp. 29-37, 2016.

[7] Y. J. Kim, Motivation to use according to the relationship formation of social network service: focusing on Twitter and Me2DAY. Hongik University, Master. thesis, 2010.

[8] Y. J. Kim, “A Study of Online Relationship”, J. of the Broadcasting and Telecommunication Studies, Vol. 23, No. 5, pp. 45-82, 2009.

[9] Y. Bae, “Social Relation in Cyberspace-Focused on the Making and Maintenance of Relation with Personal Media”, J. of the Korea Socialogy, Vol. 39, No. 5, pp. 55-82, 2005.

[10] J. H. Park, “Exploring Factors Influencing User's Continuance Intention in Social Networking Sites,” J. of the Korea Society for Information Management, Vol. 25, No. 4, pp. 205-226, 2008. [11] S. J. Lim, The Integrated Model of Social Emotion Based on Both Emotional Adjectives and Face Expression. Sangmyung University, MS. thesis, 2015.

[12] J. E. Cho, S. Park, M. J. Won, M. J. Park & M. Whang, “Research on Micro-Movement Responses of Facial, Muscles by Intimacy, Empathy, Valence”, J. of the Korea Contents Association, Vol. 17, No. 2, pp. 439-448, 2017.

[13] J. H. Park, A Study on Emotion Evaluation by Change of the Facial Micromobility, Sangmyung University, MS. thesis, 2012.

[14] K. Scherer & P. Ekman, Expression and the nature of emotion, Approaches to Emotion, 1984.

[15] Wu. H. Y, Rubinstein. M, Shih. E, Guttag. J. V, Durand. F & Freeman. W. T, "Eulerian video magnification for revealing subtle changes in the world", J of the ACM Transactions on Graphics, Vol. 31, No. 4, pp. 1-8, 2012.

[16] Poh. M. Z, McDuff. D. J & Picard. R. W, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation", J. of the Optics express, Vol. 18, No. 10, pp. 10762-10774, 2010.

[17] M. Whang & S. Park, "Brain-Heart Neural Connectivity and Emotion" J. of the Architecture and Building Science, Vol. 58, No. 9, pp. 59-62, 2014.

[18] T. K. Lee & W. H. Chung, "Vestibular Influences on Autonomic Regulation", J. of the Korea Balance Society, Vol. 5, No. 2, pp. 329-335, 2006.

[19] E. C. Lee, M. Whang, D. Ko, S. Park & S. T. Hwang, A New Social Emotion Estimating Method by Measuring Micro-Movement of Human Bust, In Industrial Application of Affective Engineering, 2014.

[20] S. Park, M. J. Won, S. Mun, E. C. Lee and M. Whang, "Does visual fatigue from 3D displays affect autonomic regulation and heart rhythm?", J. of the Psychophysiology, Vol. 92, No. 1, pp. 42-48, 2011.

[21] V. Muller & U. Lindenbeerger, "Cardiac and Respiratory Patterns Synchronize between Persons during Choir Singing", J. of the PloS one, Vol. 6, No. 9, pp. e24893, 2011.

[22] D. Tag, "A Study on The Influence of Convergence Benefit of Facebook Fan Page in Brand Attachment and Brand Commitment", J. of the Korea Convergence Society, Vol. 6, No. 5, pp. 199-206, 2015.

[23] L. S. Kim, "Convergence of Information Technology and Corporate Strategy", J. of the Korea Convergence Society, Vol. 6, No. 6, pp. 18-26, 2015.

[24] M. Lee, "A Game Design for IoT environment", J. of the Korea Convergence Society, Vol. 6, No. 4, pp. 134-137, 2015

The present invention extracts human microscopic movements (MM, Micro-movement) as biometric response information using non-contact image processing technology, and uses it for social sensitivity to competition and cooperation that can be induced in social interaction. Present ways to assess perception.

Process according to the invention:

Photographing the faces of two subjects in a sympathetic state using a video camera;

Extracting fine motion data of a subject by detecting a change amount between frames in the face image;

Detecting fine motion data for each frequency band by the magnitude of the fine motion of the subject from the fine motion data; And

And evaluating a cooperative-competition relationship between the two subjects using the frequency band fine motion data.

According to a specific embodiment of the present invention, the evaluation of the cooperative-competition relationship may use any one of the amount of motion for each frequency band, the amount of change in motion, or the synchronization of motion.

According to a specific embodiment of the present invention, the frequency band may include at least one band of 0.5Hz, 1Hz, 3Hz, 5Hz or 15Hz.

According to a specific embodiment of the present invention, the movement change amount is a fine movement amount (O _FHZ ) defined by the following Equation.

<Expression>

In the above formula, W and H are horizontal and vertical lengths (size or distance) of upper body fines in the region of interest, i.e., region of interest (ROI), and In ( i, j ) is the i th of the nth image frame The pixel values of the column and Jth row, R, are the frame rate of the camera used.

According to a specific embodiment of the present invention, the two subjects may face each other on-line or off-line.

Apparatus for evaluating competition-cooperation relationship according to the present invention for performing the above method:

A video camera photographing the subject;

An image processor which processes an image from the video camera; And

And an analyzer configured to evaluate a cooperative-competition relationship between the subjects using the information from the image processor.

1 shows an experimental environment according to the present invention.
Figure 2 shows the key arrangement of the keyboard used in the experiment according to the present invention.
3 shows a screen presented in a co-competition experiment according to the present invention.
4 is an interface screen used for extracting human body movements in a co-competition experiment according to the present invention.
Fig. 5 shows fine motion for each frequency band obtained by the experiment of the present invention.
6 is a flowchart showing an experimental procedure of the present invention.
Figure 7 shows the task flow in the experimental procedure of the present invention.
8 shows a pattern of microscopic amounts as a result of the experiment according to the present invention.
Figure 9 shows the results of the statistical analysis of small amounts as experimental results according to the present invention.
10 shows the microscopic change pattern as a result of the experiment according to the present invention.
Figure 11 shows the statistical analysis of the microscopic change as an experimental result according to the present invention.
12 shows the results of group synchronization correlation analysis as an experimental result according to the present invention.
Figure 13 shows a group synchronization pattern as a result of the experiment according to the present invention.
14 is a block diagram of a system used in the experiment of the present invention.
Figure 15 is a graph showing the rule base obtained by the experiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail a specific embodiment of the method and apparatus for evaluating the cooperation / competition relationship between the subjects using the human body movement according to the present invention.

First, an experiment for verifying a cooperation / competition relationship evaluation method according to the present invention will be described.

<Subject>

Subjects could be paired to perform the task, and the gender ratio was recruited considering gender differences. The two subjects were selected as face-to-face subjects to avoid awkwardness and were able to perform aggressive tasks. Twelve college students (6 males, 6 females, average age: 25.72 ± 1.26) participated in the experiment. These subjects recruited those with no history of cardiovascular and nervous systems. Before the experiment, it was recommended to avoid ingestion of caffeine, smoking, and drinking that may affect sleep and cardiovascular system. Subjects were given an informed consent for their voluntary intention after explaining the approximate details of the experiment, except for the purpose of the study. The intention to participate in the experiment was raised by paying a predetermined amount in exchange for participating in the experiment.

Experimental Environment

The experimental environment includes a system that can perform pattern matching tasks, a large 42-inch monitor that can see the stimulus screen, a keyboard for task execution, and a web camera that can simultaneously photograph the subject.

The two subjects, as shown in Figure 1, sit in the seats of Subject 1 (S1) and Subject 2 (S2) to perform the competition and cooperation tasks provided by the system.

The experiment was conducted between the same sex to eliminate the difference between the opposite sex, and the subjects were classified as leader and follower. The leader performed the task by directly looking at the suggested facial expressions displayed on the monitor of the stimulation system behind the other party, and the follower conducted the facial expression experiment by directly looking at the leader's face. The two subjects performed four repetitions with different roles. In order to remove the effects of the order during the role of the leader and the follower, they were performed randomly. Task performance was performed according to the presented expression, and the experimental procedure is as shown in FIG.

<Experimental Stimulation>

The experimental stimuli needed to perform the pattern matching task consisted of three coins (gold, silver and copper).

Target (Target) is a combination of the three coins to produce a pattern randomly displayed.

In order to minimize the influence on the movement, the keyboard is arranged so that the task execution can be controlled only by the right hand, and the arranged key arrangement is shown in FIG. 2.

In the competition, the stimulus presented each screen that can match the pattern by each subject, and presented a target pattern that should be centered on the screen, and accumulated a score for each subject by setting an unlimited pattern for a limited 3 minutes. In cooperation, the stimulus performs the pattern matching task in the same way as the competition. On the left side, the computer screen is displayed by showing the computer screen on the left side and the subject 1 and the subject 2 cursors on the team screen on the right side. In this competition, the team actually presented tasks to encourage cooperation.

After the last 3 minutes, additional rewards were given to subjects who scored a lot during the competition, and additional rewards were given to the team as soon as the task was completed. As shown.

<Data collection and processing>

Image data collection enabled both subjects to photograph the upper body from the front. The web camera used Logitech's C920. The camera settings were set to 1280 × 720,30fps and all the auto-adjustment functions were turned OFF to reduce errors in the data when acquiring image data such as white balance and focus.

14 is a schematic structural diagram of a system to be applied in the present invention.

Cameras photographing the two subjects S1 and S2, respectively, are video cameras, for example webcams or small video cameras 110a and 110b. The moving image from the cameras 110a and 110b is extracted through the image processing unit 120 and a specific image is processed by the processing apparatus 130. The processing device 130 has software for performing the method as described above and a hardware system supporting the same. In FIG. 14, cameras are arranged for each subject, but according to another exemplary embodiment of the present disclosure, two subjects may be photographed in one camera as illustrated in FIG. 1.

Such processing device 130 may be a computer-based device, for example a general purpose computer or dedicated device including software containing the methods or algorithms described above and hardware on which the software can run. The processing result from the processing device 130 as described above is displayed by the display device 130. The system as described above may further include a general external interface device including a general input device, for example, a keyboard, a mouse, and the like.

In this system, OpenCV and Visual C ++ were used to extract the power value of the frequency band by averaging the information of the movement difference after setting the ROI (Region of Interest) based on the center of the face. As it is.

Hereinafter, a detailed data processing method will be described.

In order to extract fine movements of the human body from the head using the image data input using the image camera, image information is separated through face recognition using OpenC.

Open Computer Vision is OpenCV. Originally developed by, it can be used on several platforms, including. This OpenCV is a library focused on real-time image processing.

The OpenCV based facial recognition system refers to a computer-assisted application that automatically identifies each person through a digital image. This is done by comparing the facial database with the selected facial features appearing in the live image.

The facial region is detected in the first frame of the upper body image using a facial recognition system or a detector (program) using the Adaboost method. At this time, detection of the face area in all the frames is not necessary. This is because, according to the present invention, the amount of fine movement is extracted by obtaining a difference of (luminance) values of pixels at the same position in two adjacent frames.

The Adaboost method uses a Strong Classifier created by combining many Simple Weak Classifiers to detect faces in the input image. Algorithms require a lot of learning time, but this has the short time required for detection and the advantages of good detection results. This took an average of 29 msec (milliseconds) per image to detect the facial area in a decimated image of 1/4.

After facial region detection, the selected region for subtracting the image (green square in FIG. 2) to expand the upper body fine motion extends to a predetermined number of pixels, for example 160 pixels, in the horizontal direction of the facial region. Let

The facial recognition program calculates the amount of motion changed between frames by calculating the difference value of each frame of each image, and extracts current and previous motion information into raw data of 30 Hz per second. From the raw data, we extract the frequency components according to the size of the motion.

Specifically, the average fine movement amount (O _FHZ ) was extracted for each 30 Hz frequency band of 0.5 Hz, 1 Hz, 3 Hz, 5 Hz, and 15 Hz frequency bands.

Specifically, in order to measure the average amount of fine movement for each frequency band, a camera image analysis program was applied to the analysis. The captured color image was converted to gray scale of black and white. This is because the color information of the image is not important for behavior analysis. In calculating the fine movement average amount "O" _{, the} average amount of micro movement at frequency "F" band (F F) may be calculated by the following equation.

<Equation 1>

In Equation 1, W and H are horizontal and vertical lengths (sizes or distances) of upper body fines in a region to be measured for microscopic movements, that is, a region of interest (ROI). In ( i, j ) are pixel values of the i th column and the J th row of the n th image frame. R is the frame rate of the camera used.

In Fig. 4, the screen of (1) is a screen displaying a real-time face image of the subject, and the screen of (2) is an area displaying only information on which the actual subject's movement occurred, and the data is displayed as shown in (4) screen. . (3) The screen is a screen that displays the raw data of the amount of motion that appears when extracting minute motion data that is hard to see.

Fig. 5 is a graph of fine motion obtained in the above data processing.

In the case of large movements visible to the human eye, small movements appear in the properties of low frequency components, and small movements invisible have very small movement components in high frequency components.

Therefore, in the experiments of the present invention, the frequency component extraction method proposed in the previous study was selected to extract the data of the corresponding frequency components by selecting a high frequency 15 Hz band capable of sampling the most and viewing the smallest data. The mean derivation algorithm is applied.

The motion data of the frequency component was extracted, and the total motion in the frame was converted to an average and stored.

The experimental procedure was performed in the order shown in FIG. 6, with two subjects sitting in front of the stimulation system capable of performing the task.

(1) Subject was allowed to proceed after setting camera and keyboard position in accordance with experiment.

(2) The competition and cooperation were explained in advance to help understand the task performance.

(3) In order to minimize the error of the operation mistake, the experiment was conducted after fully learned through the pre-test of the task.

(4) When the knowledge of the operation panel was completed, the task on the experiment was performed.

(5) When the experiment was completed, the measurement and setting were finished.

Task performance of the experiment was divided into competition and cooperation, and two subjects performed the task of competing and cooperating with each other twice.

In addition, to minimize the running effect of the physiological response according to repeated experiments, the reference and rest time were given before and after the start of the task for 3 minutes and 5 minutes. Is as shown in FIG.

<Data analysis>

Fine motion data was extracted and analyzed by the above-mentioned method from the image data measured during the task of competition and cooperation, and three indicators of the amount of fine movement, the amount of change in fine movement, and the coordination of fine movement were derived and analyzed.

As the indicators extracted in the present invention, the micro-movement amount refers to the amount of fine movement generated when each subject performs a task. In addition, the change amount of fine motion refers to the degree of change performed by the normalization process of the generated fine motion. In addition, coarseness of fineness is an indicator to analyze the correspondence of the movements of two subjects.

These indicators were analyzed by reflecting the methodology and concept of physiological responses such as HRC (Heart rhythm coherence) or HRP (Heart rhythm pattern) in the microscopic indicators. The amount of fluctuation and change data did not satisfy the normality, so the nonparametric test (Mann-whitney) was performed.

The coherence was confirmed by the correlation coefficient derived after the correlation analysis and then the independent T-test, which is a parameter test, was performed.

In order to secure statistical significance, G Power (ver 3.1.7) was used to check whether the number of samples was met, and then statistical data was analyzed using SPSS ver 17k.

<Mist analysis>

The motion data of the 15 Hz component was extracted in the section in which the task of competition and cooperation was performed.

As a result of comparing the patterns according to the amount of microscopic movement of the extracted data, the competition pattern was higher than that of cooperation, and is shown in FIG. 8.

As a result of comparing the mean value of the amount of unanimity, competition ( M = 0.656, SE = 0.020) and cooperation ( M = 0.602, SE = 0.020) showed statistically significant differences as shown in FIG. 9 ( Z (40) =- 2.272, p <0.05).

<Slow change analysis>

The variation pattern of the amount of microscopic movement was compared through the frequency component of 15 Hz extracted from the same section.

As a result, almost all the subjects showed a higher pattern when performing the competing task than when performing the cooperative task, as shown in FIG.

As a result of comparing the average amount of change in the amount of micromovement, as shown in FIG. 11, competition ( M = 0.278, SE = 0.015) and cooperation ( M = 0.149, SE = 0.018) showed a statistically significant difference ( Z (40)). = -4.680, p <0.001)

<Competitive / Collaborative Evaluation Rule-Base Setting>

Competitive / Collaborative Evaluation The evaluation rule-base was set based on the results of the analysis of the micro-movement variation. As a result of plotting the change analysis result of each subject according to the conditions of the micro movement rate of change (Variation) and the movement distance (Micro-movement of distance, Amount) as shown in FIG. 15.

Based on these plotting results, we obtained a rule-base of the following first-order equations that distinguish cooperation and competition. The rule-base below shows the statistically significant micro-movement of distance and amount. Two-dimensional evaluation rule bases were derived using two variables (micro-movement rate of change, variation).

Y = 0.15909X + 0.11239

Among the data of each subject, the point having the largest discernment to distinguish the variation and the distance was set as a threshold. As a result, competition occurs when the actual variation increases by more than about 10-15% compared to the reference value Y of the variation corresponding to the movement distance. Set to rule-based to judge.

In other words, when a subject's movement distance (Amount) is substituted into the rule base of the equation above as an X value, if the value is larger than the Y value corresponding to the X value, it is a competitive relationship and vice versa. In this case, the above threshold is determined based on 20-25% of Y for a competitive or cooperative relationship, and in the middle, it may be regarded as a neutral relationship.

The present invention as described above has confirmed the social sensitivity of competition and cooperation important for emotional interaction in the virtual space using the human body movement, a non-contact sensing technique.

The experiment was carried out through 12 tasks of 12 male and female college students through the task of inducing competition and cooperation. A subject group consisting of 12 male and female college students was selected under the conditions described above.

After competing and collaborating with each other, each game was won by giving additional rewards to induce emotional responses through active participation in competition and cooperation. Data analysis classifies 15 Hz frequency component data obtained from human body motion by each competition and cooperative task.

The collected data were analyzed for the amount of fluctuation, the amount of fluctuation, and the synchronism of the fluctuations between the two subjects. .

In addition, the coordination of movement between the two subjects was higher in coordination than in competition, and statistically significant difference was observed.

This can be seen that the amount of change and the degree of change are increased unconsciously when the arousal and tension are increased to overcome the opponent when the subject is in a competitive state. Concentrating on breathing on the other side can be interpreted as a match between the movements of the two subjects.

Therefore, it seems that the team's focus and breathing aim to win is highly synchronized with the movement.

Experimental results of the present invention can be interpreted that the synchronism for movement is the same as the synchronism for respiration and heart rate occurs when performing the task as suggested in the previous study. We evaluated the feasibility of emotional convergence technology for competition and cooperation by using human body motion, which is a non-contact sensing method that minimizes the burden and inconvenience of wearing sensors.

In the non-contact method, we confirmed that social sensitivity, which cannot be recognized in Ekman's six basic emotions, is recognizable.

In addition, the recognition method based on anatomical principles and physiological reactions, not the method of recognizing with artificial facial expressions, etc., has been confirmed through the evaluation of new indicators in emotional recognition.

It allows users to easily evaluate competition and cooperation by using the reactions of users interacting in a virtual space with a simple webcam between remote users.

Currently, the communication means mainly used in online games, etc. are mainly achieved through language means through chat or voice conversation. However, previous studies have found that emotional attachment affects brands more than functional benefits provided by simple information in brand evaluation on SNS, which suggests that emotional non-verbal communication means are more important than verbal communication methods. It is shown. In addition, information technology is a very important factor for corporate competitiveness in corporate strategy.

Accordingly, emotional information such as competition and cooperation can be used as an element of major information technology in corporate strategy. Recently, it is trying to include information about big data and human in developing game contents based on IoT. At present, human habits and patterns of thinking are used as information, but in the future, it is expected to reflect information about human emotional thinking, and non-verbal communication methods such as immigration are used for emotional recognition and other related fields. Is expected to be high. The experiment of the present invention confirmed the evaluation of competition and cooperation using the human body movement technology, which is a non-contact sensing technology.

This invention is one of the earliest studies through the non-contact sensing method. It is a study on the influence of external factors such as the surrounding environment and light, and the verification and improvement of accuracy on new indicators through systemization and mutual comparison with quantitative indicators. Further research related to stabilization is needed.

With the development of technologies such as virtual and coexistence reality, the constraints of time and space have disappeared, and activities in virtual space are expanding. There is also an increasing number of studies on communication tools and emotion recognition technologies. Therefore, it is meaningful to suggest a non-contact sensing method and a quantitative social emotion recognition method for competition and cooperation.

Although the present invention has been described with reference to the embodiments illustrated in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true scope of protection of the present invention should be defined only by the appended claims.

Claims (12)

Photographing the faces of two subjects in a sympathetic state using a video camera;
Extracting fine motion data of a subject by detecting a change amount between frames in the face image;
Detecting fine motion data for each frequency band by the magnitude of the fine motion of the subject from the fine motion data; And
And evaluating a cooperative-competitive relationship between the two subjects using the frequency band-specific microscopic data. The method of claim 1,
The fine data for each frequency band is any one of the amount of motion by frequency band, the amount of change of motion or the synchronization of motion. The method of claim 2,
Wherein said frequency band comprises at least one of 0.5 Hz, 1 Hz, 3 Hz, 5 Hz or 15 Hz. The method of claim 1,
The frequency band includes at least one of 0.5 Hz, 1 Hz, 3 Hz, 5 Hz, or 15 Hz. The method of claim 4, wherein
The variation amount of movement is a cooperative-competition relationship evaluation method, characterized in that the fine movement amount (O _FHZ ) defined in the following equation.
<Expression>

Where W and H are horizontal and vertical lengths (sizes or distances) of upper body fines in the area subject to microscopic measurement, and In ( i, j ) are the i th column and the J th The pixel value of (Row), R, is the frame rate of the camera used. The method of claim 1,
The fine motion data for each frequency band is a cooperative-competition relationship determination method, characterized in that the fine motion amount (O _FHZ ) defined in the following Equation.
<Expression>

Where W and H are horizontal and vertical lengths (sizes or distances) of upper body fines in the area subject to microscopic measurement, and In ( i, j ) are the i th column and the J th The pixel value of (Row), R, is the frame rate of the camera used. The method of claim 6,
The frequency band is a cooperative-competition relationship determination method characterized in that it comprises at least one band of 0.5Hz, 1Hz, 3Hz, 5Hz or 15Hz A competition-cooperation relationship evaluation apparatus for performing the method according to claim 1, comprising:
A video camera photographing the subject;
An image processor which processes an image from the video camera; And
And an analyzer configured to evaluate a cooperative-competition relationship between the subjects using the information from the image processing unit. The method of claim 8,
The fine data for each frequency band is any one of the amount of movement of each frequency band, the amount of change of movement or the synchronization of movement. The method according to claim 8 or 9,
And said frequency band comprises at least one of 0.5 Hz, 1 Hz, 3 Hz, 5 Hz or 15 Hz. The method of claim 10,
The apparatus for evaluating cooperation-competition relationship according to claim 1, wherein the movement change amount is a micro-movement amount (O _FHZ ) defined in the following Equation.
<Expression>

Where W and H are horizontal and vertical lengths (sizes or distances) of upper body fines in the area subject to microscopic measurement, and In ( i, j ) are the i th column and the J th The pixel value of (Row), R, is the frame rate of the camera used. The method of claim 9,
The variation amount of movement is a cooperative-competition relationship evaluation method, characterized in that the fine movement amount (O _FHZ ) defined in the following equation.
<Expression>

Where W and H are horizontal and vertical lengths (sizes or distances) of upper body fines in the area subject to microscopic measurement, and In ( i, j ) are the i th column and the J th The pixel value of (Row), R, is the frame rate of the camera used.

Images