KR101583774B1 - System and method for fear mentality analysis - Google Patents

Abstract

A fear psychological analysis system and method are disclosed. The fear psychological analysis system includes an EEG acquisition device for acquiring brain waves and generating and outputting EEG data, a first camera device for simultaneously acquiring and outputting a visible light image and a thermal image for the face region, and acquiring an eye image for the eye region A second camera device for outputting the brain wave data, a brain wave data, a visible ray image, a thermal image, and a snow image, and analyzing the brain wave, the temperature of the face, and the eye change to estimate a fear psychological state .

Description

{System and method for fear mentality analysis}

The present invention relates to a fear psychological analysis system and method.

A method of grasping the fear psychology and state of a person includes acquiring and analyzing the user's brain wave using the brain wave device, and grasping the fear psychology and the state. However, since the EEG acquires an EEG accompanied by noise caused by hair, motion, and the like, it has an inaccurate disadvantage in grasping the fear psychology and state.

Another method for grasping a person's fear psychology and state is to analyze the temperature change of a person's face region using a thermal image to grasp the fear psychology and state. However, in this method, when the thermal imaging camera acquires a face image of a person in a short distance, loss of facial feature point location information such as face, nose, eye, and mouth occurs and the face region can not be detected accurately, There is a problem that the position of the facial feature point in the image image is incorrectly detected. Accordingly, there is a problem that the accuracy of the fear psychology and the state grasp according to the change of the face temperature at the face recognition and facial feature point positions through the thermal image of the face image is degraded.

In the past, the image of the eye region was analyzed, and the psychological state and the state were grasped according to the eye blinking, the pupil size change, and the like. However, this is user dependent and is highly influenced by the surrounding environment.

The present invention intends to more accurately estimate the fear psychological state by integrally analyzing brain waves, temperature changes of the face, and eye changes.

According to an aspect of the invention, a fear psychological analysis system is disclosed.

The fear psychological analysis system according to the embodiment of the present invention includes an EEG acquisition device for acquiring brain waves and generating and outputting EEG data, a first camera device for simultaneously acquiring and outputting visible light images and thermal images for the face region, A second camera device for acquiring and outputting an eye image for a region, and a second camera device for collecting and analyzing the brain wave data, the visible light image, the thermal image, and the eye image, And a fear psychological analysis device for estimating a fear psychological state.

The brain wave acquisition apparatus is a wearable wireless brain wave apparatus formed in the form of a headset that can be worn on the head and includes a plurality of brain wave sensors for receiving brain waves from the brain.

The first camera device is a device combining a visible light camera and a thermal imaging camera.

Wherein the second camera device includes a high-speed camera and a near-infrared illumination device, wherein the near-infrared illumination device irradiates near-infrared light to the eye, and the high-speed camera photographs the eye area irradiated with the near- This clear eye image is obtained.

Wherein the fear psychological analysis apparatus comprises a first analyzing unit for receiving and analyzing the brain wave data and calculating first fear psychological state estimation data according to an EEG, a second analyzing unit for receiving and analyzing the visible light ray image and the thermal image, A third analyzing unit for receiving and analyzing the eye image and calculating third fear psychological state estimation data according to the eye change, and a second analyzing unit for calculating second fear psychological state estimation data according to the temperature change of the eye, 1 horror psychological state estimation data, the second horror psychological state estimation data, and the third horror psychological state estimation data to estimate a fear psychological state.

The first analysis unit receives a plurality of brain wave data acquired by a plurality of brain wave sensors, converts the brain wave data into a frequency domain to calculate brain wave data in a frequency domain, classifies brain wave data in the frequency domain by band, And calculates the first fear psychological state estimation data indicating the degree of fear.

Wherein the second analyzing unit detects facial feature points in the visible light ray image, maps the detected facial feature points to the thermal image, detects temperature values of the mapped facial feature points in the thermal image, The temperature change of the face is analyzed using the temperature value, and the second fear psychological state estimation data indicating the degree of fear sensitivity is calculated.

Wherein the third analyzing unit detects the eye region in the eye image, detects the pupil in the detected eye region, calculates the center and size of the detected pupil, and calculates the center and the size of the detected pupil, And calculates the third fear psychological state estimation data indicating the degree of fear.

The estimator estimates the fear psychological state using the first to third fear psychological state estimation data obtained from the users in advance and the subjective evaluation data of the users obtained before and after the users touch the fear medium .

According to another aspect of the present invention, a fear psychological analysis apparatus performs fear psychological analysis using brain wave data, simultaneously acquired visible light image, thermal image, and eye image.

The fear psychological analysis method according to an embodiment of the present invention includes inputting and analyzing the brain wave data, calculating first fear psychological state estimation data according to brain waves, inputting and analyzing the visible light image and the thermal image, Calculating second fear psychological state estimation data according to a temperature change of a face, inputting and analyzing the eye image, calculating third fear psychological state estimation data according to eye changes, Estimation data, the second fear psychological state estimation data, and the third fear psychological state estimation data to estimate a fear psychological state.

Calculating the first fear psychological state estimation data includes receiving a plurality of brain wave data acquired by a plurality of brain wave sensors, converting the brain wave data into a frequency domain to calculate EEG brain wave data, Analyzing the EEG data in the frequency domain by band and analyzing the EEG data to calculate the first fear psychological state estimation data indicating the degree of fear.

Wherein the step of calculating the second fear psychological state estimation data comprises the steps of: detecting facial feature points in the visible light image; mapping the detected facial feature points to the thermal image; Analyzing the temperature change of the face using the detected temperature value, and calculating the second fear psychological state estimation data indicating the degree of fear sensitivity.

Wherein the step of calculating the third fear psychological state estimation data comprises the steps of: detecting an eye region in the eye image; detecting a pupil in the detected eye region; calculating a center and a size of the detected pupil; And analyzing the eye change using the center and the size of the calculated pupil to calculate the third fear psychological state estimation data indicating the degree of fear.

The present invention can more accurately estimate the fear psychological state by integrally analyzing the brain waves, the temperature change of the face, and the eye change.

1 schematically illustrates a configuration of a fear psychological analysis system;
Figure 2 illustrates the use of a fear psychological analysis system.
3 is a flowchart showing a fear psychological analysis method using brain wave data.
4 is a flowchart illustrating a fear psychological analysis method using a visible light image and a thermal image.
5 is a flowchart showing a fear psychological analysis method using eye images.
6 is a diagram illustrating an image acquired by a visible light camera, a thermal imaging camera, and a high-speed camera;

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals (e.g., first, second, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another.

Also, in this specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate a thorough understanding of the present invention, the same reference numerals are used for the same means regardless of the number of the drawings.

FIG. 1 schematically illustrates a configuration of a fear psychological analysis system, and FIG. 2 illustrates an application of a fear psychological analysis system. Hereinafter, a fear psychological analysis system will be described with reference to FIG. 1, but with reference to FIG.

Referring to FIG. 1, a fear psychological analysis system includes a fear psychological analysis apparatus 100, an EEG acquisition apparatus 200, a first camera apparatus 300, and a second camera apparatus 400.

The EEG acquiring apparatus 200 acquires an EEG to generate and output EEG data. Here, the outputted brain wave data is transmitted to the fear psychological analysis apparatus 100. For example, as shown in FIG. 2, the brain wave acquisition device 200 may be a wearable wireless brain wave device 210 formed in the form of a headset that can be worn on the head, A plurality of EEG sensors for receiving the EEG signals.

The first camera device 300 simultaneously acquires and outputs the visible light image and the thermal image to the face region. Here, the output visible light ray image and the thermal image are transmitted to the fear psychological analysis apparatus 100. For example, as shown in FIG. 2, the first camera device 300 is a device 310 that combines a visible light camera and a thermal imaging camera, and displays a visible light image and a thermal image for a human face region simultaneously Can be obtained.

The second camera device 400 acquires and outputs eye images of the eye region. Here, the output eye image is transmitted to the fear psychological analysis apparatus 100. For example, as shown in FIG. 2, the second camera device 400 may include a high-speed camera 410 and a near-infrared illumination device 420. Here, the near-infrared illumination device 420 irradiates the near-infrared light to the eye, and the high-speed camera 410 photographs the eye area irradiated with the near-infrared light, so that the second camera device 400 can recognize the eye Images can be acquired.

The fear psychoanalytic apparatus 100 includes brain wave data from the brain wave acquiring apparatus 200, the first camera apparatus 300 and the second camera apparatus 400, a visible light image and a thermal image for the face region, The eye image is input and analyzed to estimate the fear psychological state according to the brain wave, the temperature change of the face and the eye change.

1, the fear psychological analysis apparatus 100 includes a first analysis unit 10, a second analysis unit 20, a third analysis unit 30, and an estimation unit 40.

The first analyzer 10 receives and analyzes brain wave data from the brain wave acquisition apparatus 200, and calculates the fear psychological state estimation data according to brain waves. This will be described with reference to FIG.

3 is a flowchart illustrating a fear psychological analysis method using EEG data.

In step S310, the first analyzer 10 receives the brain wave data from the brain wave acquiring apparatus 200. For example, the first analysis unit 10 may receive a plurality of brain wave data acquired by a plurality of brain wave sensors of the brain wave acquisition apparatus 200.

In step S320, the first analyzer 10 converts the brain wave data into the frequency domain. For example, the first analyzer 10 may calculate EEG data in a frequency domain using a Fourier transform for converting a plurality of brain wave data into a frequency domain.

In step S330, the first analyzer 10 analyzes the computed EEG brain wave data. For example, the first analyzer 10 can analyze the EEG data in the frequency domain according to bands such as a seta wave, a gamma wave, an alpha wave, and a beta wave to thereby analyze the user's fear level, Data can be calculated. Here, the first analyzer 10 compares the EEG data of the calculated frequency domain with the EEG data in the initial state or the normal state and the EEG data in the state of feeling fear, thereby indicating the degree of fear according to the EEG of each band The fear psychological state estimation data can be calculated.

The second analyzer 20 receives and analyzes the visible light image and the thermal image of the face region from the first camera device 300, and calculates the fear psychological state estimation data according to the temperature change of the face. This will be described with reference to FIG.

4 is a flowchart illustrating a fear psychological analysis method using a visible ray image and a thermal image.

In step S410, the second analysis unit 20 receives the visible ray image and the thermal image of the face region from the first camera device 300. [

In step S420, the second analyzer 20 detects facial feature points in the visible light image of the face region. For example, the second analyzing unit 20 may detect the facial feature points such as face region, nose, eye, eyebrow, cheek, forehead, ear, up robust features, Camshift (adaptive mean shift), Adaptive Template Matching, SIFT (scale-invariant feature transform), Adaboost (adaptive boosting), Rapid Eye Detection, The face region and the facial feature point can be detected.

In step S430, the second analysis unit 20 maps the facial feature points detected in the visible light image to the thermal image. For example, the second analyzing unit 20 maps the facial feature points such as the face region, the face region, the nose, the eye, the eyebrows, the cheeks, the forehead, the ear, the neck, . At this time, the second analyzer 20 can map the facial feature points of the visible light image to the thermal image using a calibration-based geometric transform method or the like.

In step S440, the second analyzer 20 detects the temperature value of the facial feature point mapped to the thermal image. For example, the second analyzing unit 20 detects the temperature of the regions where the facial feature points or the facial feature points are located based on the positions of the facial feature points mapped to the thermal image as temperature values such as an average value, a maximum value, and a minimum value can do.

For example, FIG. 6 is a diagram illustrating an image acquired by a visible light camera, a thermal imaging camera, and a high-speed camera. 6, the 610 image of FIG. 6 is an image obtained by the visible light camera, the yellow box area is the area where the wearable wireless brain wave device 210 is located, the red box area is the face area, nose, Cheeks, and necks. The 620 image of FIG. 6 is an image obtained by the thermal imaging camera and is a thermal image obtained by mapping the facial feature points detected in the visible ray image, as described above. In this manner, the temperature value of the mapped region can be detected and analyzed in the thermal image in which the facial feature points detected in the visible light image are mapped.

In step S450, the second analyzer 20 analyzes the temperature change of the face using the detected temperature values. For example, the second analyzer 20 may analyze the temperature change of the areas where the facial feature points or the facial feature points are located, and the temperature change of the areas where the facial feature points are located may be an average temperature value, Temperature values and the like.

For example, when the user's fear is increased, the pulse rate is rapidly increased or the temperature in the specific region of the face is increased or decreased. Thus, the second analyzing unit 20 calculates the fear psychological state estimation data indicating the degree of fear of the user by using the features such as the ear, neck, can do. That is, the second analyzing unit 20 compares the temperature change of the face with the temperature change of the face in the initial state or the normal state and the face temperature in the state of feeling fear, The fear psychological state estimation data can be calculated.

The third analyzer 30 receives and analyzes the eye image of the eye region from the second camera apparatus 400, and calculates the fear psychological state estimation data according to the eye change. This will be described with reference to FIG.

5 is a flowchart illustrating a fear psychological analysis method using an eye image.

In step S510, the third analyzer 30 receives the eye image of the eye area from the second camera device 400. [ For example, the high-speed camera 410 may acquire an eye image of the eye region at about 70 frames per second. Here, the high-speed camera 410 can acquire an eye image of an eye region irradiated with near-infrared light by the near-infrared light illuminating device 420.

In step S520, the third analyzer 30 detects an eye area on the input eye image. For example, the third analyzing unit 30 can detect the eye region by searching the eye image using RED (Rapid Eye Detection). Referring back to FIG. 6, the 630 image of FIG. 6 is an image acquired by a high-speed camera, and both eye regions are detected using an eye detection algorithm.

In step S530, the third analyzer 30 detects the pupil in the detected eye area. For example, the third analyzer 30 may detect pupil in the eye region by performing binarization, morphology, labeling, and ellipse boundary detection on the detected eye region.

In step S540, the third analyzer 30 calculates the center and size of the detected pupil.

In step S550, the third analyzer 30 analyzes the eye change using the center and the size of the calculated pupil. For example, the third analyzer 30 can analyze changes in the center of the pupil, eye blinking according to the pupil size, pupil movement, pupil size change while the eye is open, and the degree of fear of the user Can be calculated. That is, the third analysis unit 30 calculates the fear psychological state estimation data indicating the degree of fear according to the eye change by comparing the eye change with the eye change in the initial state or the eye change in the normal state and the fear in the state of feeling fear can do.

The estimating unit 40 receives the fear psychological state estimation data according to the changes in the brain waves, the temperature of the face, and the eye changes from the first analyzing unit 10, the second analyzing unit 20 and the third analyzing unit 30 Combine, categorize and analyze to estimate the ultimate horror state.

For example, the estimating unit 40 may be configured to estimate the user's subjective evaluation data, brain waves, facial temperature changes, and fear psychological state data according to eye changes obtained before touching the normal state or the fear medium, The more accurate the fear psychological state can be estimated through the continuous learning based on the user's subjective evaluation data, the brain waves, the temperature change of the face, and the estimation data of the fear psychological state according to the eye change.

For example, the estimating unit 40 may use a method such as support vector machines, linear discriminant analysis, multi-layer perceptron, Fuzzy Based Fusion Method, Score Level Fusion, By combining the psychological state estimation data to reduce the size, and by weighting the weight values of each data, it is possible to classify the fear psychological state estimation data and classify the fear psychological state of the user accurately based on the subjective survey evaluation of the user Can be estimated.

Here, the estimation result of the fear psychological state can be calculated as a level indicating the degree of fear. The estimating unit 40 calculates a fear psychological state estimated value obtained by quantifying the degree of fear from the estimated data of the fear psychological state according to the brain wave, the temperature change of the face, and the eye change, and calculates a level having the range including the calculated fear psychological state estimated value It can be calculated as the estimation result of fear psychological state. For example, the levels can be divided as shown in Table 1 below.

level Contents One I do not feel frightened 2 Feeling a bit frightened 3 Feeling frightened 4 I feel a lot of fear or a lot of fear 5 Too scary ... ...

Meanwhile, the fear psychological analysis method according to an embodiment of the present invention may be implemented in a form of a program command that can be performed through various electronic means for processing information, and may be recorded in a storage medium. The storage medium may include program instructions, data files, data structures, and the like, alone or in combination.

Program instructions to be recorded on the storage medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of software. Examples of storage media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, magneto-optical media and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The above-mentioned medium may also be a transmission medium such as a light or metal wire, wave guide, etc., including a carrier wave for transmitting a signal designating a program command, a data structure and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as devices for processing information electronically using an interpreter or the like, for example, a high-level language code that can be executed by a computer.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

100: Fear psychological analyzer
10: First analysis section
20: Second analysis section
30: Third analysis section
40:
200: EEG acquisition device
210: Wearable wireless brain wave device
300: first camera device
310: Device combining a visible light camera and a thermal imaging camera
400: second camera device
410: High Speed Camera
420: near-infrared light device

Claims (13)

An EEG acquisition device for generating and outputting a plurality of EEG data acquired by a plurality of EEG sensors;
A first camera device for simultaneously acquiring and outputting a visible light image and a thermal image for the face region;
A second camera device for acquiring and outputting a snow image for the eye region, the second camera device including a high-speed camera and a near infrared ray illumination device; And
And a fear psychological analyzer for receiving and analyzing the brain wave data, the visible light image, the thermal image, and the eye image to estimate a fear psychological state according to brain waves, temperature change of eyes, and eye changes,
The fear psychological analysis apparatus comprises:
A first analyzing unit for receiving the EEG data and analyzing the EEG data by bands and calculating first fear psychological state estimation data according to brain waves;
A face image detecting unit for detecting a face feature point in a visible light image by first receiving the visible light image and the thermal image, detecting a temperature value of a facial feature point of the thermal image mapped to the detected face feature point, A second analysis unit for calculating second fear psychological state estimation data according to a temperature change, the face feature point being at least one of an ear, a neck and a ball;
The method of claim 1, further comprising the steps of: receiving an image of the eye region irradiated with near-infrared illumination from the near-infrared illumination device using the high-speed camera; A third analyzing unit for calculating state estimation data; And
And an estimator for estimating a fear psychological state by classifying the first fear psychological state estimation data, the second fear psychological state estimation data, and the third fear psychological state estimation data,
The estimating unit estimates the subjective evaluation data of the user obtained before touching the fear medium, the fear wave psychological state estimation data according to the brain wave, the temperature change of the face, and the eye change, the subjective evaluation data of the user obtained after contacting the fear medium, , A fear psychological analysis system that estimates the fear psychological state through continual learning based on the estimated data of the fear psychological state according to the temperature change of the face and the eye change.
The method according to claim 1,
Wherein the brain wave acquisition device is a wearable wireless brain wave device formed in the form of a headset that can be worn on the head and includes a plurality of brain wave sensors for receiving brain waves from the brain.
delete delete delete delete delete delete delete A method for performing a fear psychological analysis using a brain wave data, a simultaneously obtained visible light ray image, a thermal image, and an eye image,
Generating and outputting a plurality of brain wave data acquired by a plurality of brain wave sensors;
Acquiring and outputting a visible light image and a thermal image for the face region simultaneously;
Acquiring and outputting an eye image for the eye region; and a second camera apparatus for acquiring an eye image for the eye region, the apparatus comprising: a high-speed camera and a near infrared ray illumination device; And
And analyzing the received EEG data, the visible light image, the thermal image, and the eye image, and estimating a fear psychological state according to an EEG, a temperature change of a face, and an eye change,
Wherein the step of estimating the fear psychological state comprises:
Analyzing the EEG data by band and analyzing them, and calculating first fear psychological state estimation data according to brain waves;
Detecting facial feature points in the visible light image by receiving the visible light image and the thermal image;
Detecting a temperature value of a facial feature point of the thermal image mapped to the detected facial feature point and calculating second fear psychological state estimation data according to a temperature change of the face, And at least one of balls;
The method of claim 1, further comprising the steps of: receiving an image of the eye region irradiated with near-infrared illumination from the near-infrared illumination device using the high-speed camera; Calculating state estimation data; And
Classifying the first fear psychological state estimation data, the second fear psychological state estimation data, and the third fear psychological state estimation data to estimate a fear psychological state,
The step of classifying the first fear psychological state estimation data, the second fear psychological state estimation data, and the third fear psychological state estimation data to estimate the fear psychological state
The subjective evaluation data of the user obtained before touching the fear medium, the estimated data of the fear psychological state due to the change of the temperature of the face, the temperature of the face, and the eye change and the subjective evaluation data of the user obtained after contacting the fear medium, And estimating the horror psychological state through continuous learning based on the horror psychological state estimation data according to the temperature change and the eye change. delete delete delete

Images