CN114209322A - Method for detecting depression based on video analysis - Google Patents

Method for detecting depression based on video analysis Download PDF

Info

Publication number
CN114209322A
CN114209322A CN202111522156.2A CN202111522156A CN114209322A CN 114209322 A CN114209322 A CN 114209322A CN 202111522156 A CN202111522156 A CN 202111522156A CN 114209322 A CN114209322 A CN 114209322A
Authority
CN
China
Prior art keywords
value
difference
determining
pupil
pupil area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111522156.2A
Other languages
Chinese (zh)
Other versions
CN114209322B (en
Inventor
齐中祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Womin High New Science & Technology Beijing Co ltd
Original Assignee
Womin High New Science & Technology Beijing Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Womin High New Science & Technology Beijing Co ltd filed Critical Womin High New Science & Technology Beijing Co ltd
Priority to CN202111522156.2A priority Critical patent/CN114209322B/en
Publication of CN114209322A publication Critical patent/CN114209322A/en
Application granted granted Critical
Publication of CN114209322B publication Critical patent/CN114209322B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/11—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Educational Technology (AREA)
  • Developmental Disabilities (AREA)
  • Child & Adolescent Psychology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

The invention relates to a depression detection method based on video analysis, which comprises the following steps: acquiring vegetative nerve signals, facial videos and pupil areas of a user when receiving emotional stimulation; determining an emotion coefficient based on the autonomic nervous signal; determining an emotion coefficient based on the pupil area; and carrying out depression detection on the facial video based on the emotional coefficient and the mood coefficient to obtain a depression detection result. According to the method provided by the invention, whether the user suffers from the depression is detected through the vegetative nerve signals, the facial video and the pupil area when the user receives emotional stimulation, so that the automatic detection of the depression is realized.

Description

Method for detecting depression based on video analysis
Technical Field
The invention relates to the technical field of psychological evaluation, in particular to a depression detection method based on video analysis.
Background
At present, depression is the second largest disease of humans after cardiovascular diseases, about 80 ten thousand people suicide for depression every year, and the onset of depression has begun to trend toward the development of a low age (university, or even a group of primary and secondary school students). However, the medical treatment and prevention of the depression in China are still in the situation of low recognition rate, hospitals above grade market receive related drug treatment for patients with recognition rate less than 20% and less than 10%, so that the detection of the depression is vital to the medical prevention work of the depression.
Disclosure of Invention
Technical problem to be solved
In view of the above-mentioned shortcomings and drawbacks of the prior art, the present invention provides a method for depression detection based on video analysis.
(II) technical scheme
In order to achieve the purpose, the invention adopts the main technical scheme that:
a method of depression detection based on video analytics, the method comprising:
s101, acquiring vegetative nerve signals, facial videos and pupil areas of a user when receiving emotional stimulation;
s102, determining an emotion coefficient based on the autonomic nervous signals;
s103, determining an emotion coefficient based on the pupil area;
and S104, carrying out depression detection on the facial video based on the emotion coefficient and the emotion coefficient to obtain a depression detection result.
Optionally, the S102 includes:
s102-1, forming the vegetative nerve signals into a signal set; each element in the signal set corresponds to an vegetative nerve signal value acquired at a moment, and the elements in the signal set are arranged from far to near according to the acquisition moments;
s102-2, determining a difference value between every two adjacent elements in the signal set to form a signal difference set, wherein the difference value is a value of a next element-a value of a previous element;
and S102-3, determining the emotion coefficient according to the signal difference set.
Optionally, the S102-3 includes:
s102-3-1, determining standard deviation sigma of all elements in the signal difference setΔ；
S102-3-2, determining the element with the maximum value in the signal difference set
Figure BDA0003407925380000021
Element with minimum sum
Figure BDA0003407925380000022
S102-3-3, determining the element a with the largest value in the signal setmaxElement a with the smallest summinAnd the time t corresponding to the element with the largest valuemaxTime t corresponding to the element with the smallest valuemin；
S102-3-4, determining the emotional coefficient
Figure BDA0003407925380000023
Optionally, the S103 includes:
s103-1, forming a pupil area set by the pupil areas; each element in the pupil area set corresponds to a pupil area acquired at a moment, and the elements in the pupil area set are arranged from far to near according to the acquisition moments;
s103-2, determining a difference value between every two adjacent elements in the pupil area set to form a pupil area difference set, wherein the difference value is a value of a next element-a value of a previous element;
s103-3, determining an emotion coefficient according to the pupil area difference set.
Optionally, the S103-3 includes:
s103-3-1, determining the mean value avg of all elements in the pupil area difference setΔ；
S103-3-2, determining the mean value avg of all elements in the pupil area set;
s103-3-3, according to the avgΔDetermining a positive pupil area set and a negative pupil area set by the avg and the pupil area set;
s103-3-4, determining an emotion coefficient according to the positive pupil area set and the negative pupil area set.
Optionally, the S103-3-3 includes:
determining a difference of each element in the set of pupil areas and avg;
the difference is a positive number, and the absolute value of the difference is greater than avgΔForm a set of positive pupillary area；
The difference is negative and the absolute value of the difference is greater than min { avg }ΔThe elements of 2.5 form a negative set of pupil areas.
Optionally, the S103-3-3 includes:
determining a difference of each element in the set of pupil areas and avg;
the difference is a positive number, and the absolute value of the difference is greater than avgΔMeanwhile, elements with corresponding element values larger than the element mean value in the signal difference set form a negative pupil area set;
the difference is negative and the absolute value of the difference is greater than min { avg }ΔAnd 2.5, and simultaneously, forming a positive pupil area set by elements of which the corresponding element values in the signal difference set are larger than the element mean values in the signal difference set.
Optionally, a positive number for any difference, and the absolute value of the difference is greater than avgΔOr, negative for any difference, and the absolute value of the difference is greater than min { avg [ ]Δ2.5} whose corresponding element value in the set of signal differences is:
if any element is obtained from the last element in the pupil area set, determining the value of the last element in the signal difference set as the corresponding element value in the signal difference set;
if any element is not obtained from the last element in the pupil area set, determining the acquisition time t of any elementiCorresponding to the t-th signal difference in the signal difference seti+1Temporal autonomic nerve signaling and tiThe values of the elements of the difference between the vegetative nerve signals at a time are determined as their corresponding element values in the set of signal differences.
Optionally, the S103-3-4 includes:
determining a maximum element value in the set of positive pupil areas
Figure BDA0003407925380000041
Element with minimum sum
Figure BDA0003407925380000042
And
Figure BDA0003407925380000043
corresponding time of day
Figure BDA0003407925380000044
And
Figure BDA0003407925380000045
corresponding time of day
Figure BDA0003407925380000046
Determining a maximum element value in the set of negative pupil areas
Figure BDA0003407925380000047
Element with minimum sum
Figure BDA0003407925380000048
And
Figure BDA0003407925380000049
corresponding time of day
Figure BDA00034079253800000410
And
Figure BDA00034079253800000411
corresponding time of day
Figure BDA00034079253800000412
Determining the maximum element value in the pupil area difference set
Figure BDA00034079253800000413
Corresponding element value b in the set of pupil areas1Element with minimum sum
Figure BDA00034079253800000414
Corresponding element values in the set of pupil areasb2(ii) a Wherein, b is1For obtaining from the set of pupil areas
Figure BDA00034079253800000415
Of the two elements of (a) has the largest value, b2For obtaining from the set of pupil areas
Figure BDA00034079253800000416
The element with the largest value of the two elements;
determining positive coefficients
Figure BDA00034079253800000417
Negative coefficient of friction
Figure BDA00034079253800000418
Figure BDA00034079253800000419
Determine emotion coefficient I2 ═ max { I ═ I+，I-}。
Optionally, the S104 includes:
s104-1, identifying the micro expression of each frame in the facial video;
s104-2, determining the change degree among the micro expressions of each frame;
s104-3, determining the maximum number of continuous frames with the change degree not greater than a change threshold;
s104-4, determining the maximum value I1I 2; wherein I1 is an emotion coefficient, and I2 is an emotion coefficient;
s104-5, if the detection value is larger than a depression threshold value, determining that depression is detected.
(III) advantageous effects
Acquiring vegetative nerve signals, facial videos and pupil areas of a user when receiving emotional stimulation; determining an emotion coefficient based on the autonomic nervous signal; determining an emotion coefficient based on the pupil area; and carrying out depression detection on the facial video based on the emotional coefficient and the mood coefficient to obtain a depression detection result. Whether the user suffers from the depression is detected through the vegetative nerve signals, the facial videos and the pupil areas when the user receives emotional stimulation, and the automatic detection of the depression is realized.
Drawings
Fig. 1 is a schematic flow chart of a method for depression detection based on video analysis according to an embodiment of the present invention.
Detailed Description
For the purpose of better explaining the present invention and to facilitate understanding, the present invention will be described in detail by way of specific embodiments with reference to the accompanying drawings.
At present, depression is the second largest disease of humans after cardiovascular diseases, about 80 ten thousand people suicide for depression every year, and the onset of depression has begun to trend toward the development of a low age (university, or even a group of primary and secondary school students). However, the medical treatment and prevention of the depression in China are still in the situation of low recognition rate, hospitals above grade market receive related drug treatment for patients with recognition rate less than 20% and less than 10%, so that the detection of the depression is vital to the medical prevention work of the depression.
Based on the method, the invention provides a depression detection method based on video analysis, which is used for acquiring the vegetative nerve signals, the facial video and the pupil area of a user when receiving emotional stimulation; determining an emotion coefficient based on the autonomic nervous signal; determining an emotion coefficient based on the pupil area; and carrying out depression detection on the facial video based on the emotional coefficient and the mood coefficient to obtain a depression detection result. Whether the user suffers from the depression is detected through the vegetative nerve signals, the facial videos and the pupil areas when the user receives emotional stimulation, and the automatic detection of the depression is realized.
In a specific implementation, the user may be provided with stimuli of different emotions, such as an angry emotion, a sentiment, and the like. While the user receives stimulation of different moods, whether the user has depression tendency is detected by the method as shown in fig. 1, and whether the user has depression is determined.
Referring to fig. 1, the implementation process of the method for depression detection based on video analysis provided in this embodiment is as follows:
s101, acquiring vegetative nerve signals, facial videos and pupil areas of a user when receiving emotional stimuli.
When receiving emotional stimulation, a user can obtain the current vegetative nerve signals, the face video and the pupil area of the user once in real time, and the acquisition time is recorded.
In specific implementation, the current vegetative nerve signals and the pupil area are acquired at the same frequency, but the frequency may be different from that of the facial video, data recording is performed at the acquisition time according to the frequency of vegetative nerve signals and the pupil area, and the facial video only needs to record the video frame number between the current acquisition time and the previous acquisition time.
For example: the data collected are shown in table 1.
TABLE 1
Collection mark 0 1 2 3 4
Vegetative nerve signal value S0 S1 S2 S3 S4
Video frame numbering F0-F23 F24-F47 F48-F71 F72-F95 F96-F119
Pupil area A0 A1 A2 A3 A4
Time of acquisition t1 t2 t3 t4 t5
And S102, determining the emotion coefficient based on the vegetative nerve signals.
In particular, the method comprises the following steps of,
s102-1, forming the vegetative nerve signals into a signal set.
Each element in the signal set corresponds to the vegetative nerve signal value acquired at a moment, and the elements in the signal set are arranged from far to near according to the acquisition moments.
Taking the data shown in table 1 obtained in step 101 as an example, the signal set S includes 5 elements, and the set S ═ S0,S1,S2,S3,S4}。
And S102-2, determining the difference value between every two adjacent elements in the signal set to form a signal difference set.
Where the difference is the value of the next element-the value of the previous element.
Still taking the example in step S102-1 as an example, the set of signal differences Δ S includes 4 elements, and the set Δ S ═ S1-S0,S2-S1,S3-S2,S4-S3}。
If it is to be S1-S0Is marked as a0，S2-S1Is marked as a1，S3-S2Is marked as a2，S4-S3Is marked as a3Then Δ S ═ a0,a1,a2,a3}。
That is, any element in the set Δ S (e.g., a)j) Having a value of Sj+1-SjI.e. aj＝Sj+1-Sj。
And S102-3, determining the emotion coefficient according to the signal difference set.
For example, the emotion coefficients are determined by the following scheme:
s102-3-1, determining standard deviation sigma of all elements in the signal difference setΔ。
I.e. determining the standard deviation sigma of all elements in the set of signal differences deltasΔ。
For example, if Δ S ═ a0,a1,a2,a3}, then
Figure BDA0003407925380000071
S102-3-2, determining the element with the largest mean value in the signal difference set
Figure BDA0003407925380000072
Element with minimum sum
Figure BDA0003407925380000073
I.e. to determine
Figure BDA0003407925380000074
Wherein max { } is a maximum value solving function, and min { } is a minimum value solving function.
S102-3-3, determining the element a with the largest value in the signal setmaxElement a with the smallest summinAnd the time t corresponding to the element with the largest valuemaxTime t corresponding to the element with the smallest valuemin。
I.e. determine amax＝max{S0,S1,S2,S3,S4}，amin＝min{S0,S1,S2,S3,S4}。
amaxCorresponding to a collection time tmax，aminCorresponding to a collection time tmin。
S102-3-4, determining the emotional coefficient
Figure BDA0003407925380000075
The clinical manifestations of depression are that the mood is bad and the life is too happy, the mood is low for a long time and is dull, from sultriness at first to sadness at last, the feeling is life every day, the people feel apprehensive and hurting, negative, escape and even have suicide attempt and behavior.
Patients suffering from depression do not actively interact emotionally with the outside, i.e., respond relatively slowly to external stimuli. The autonomic nervous signal value can reflect the user's response to the current emotional stimulus, so the longer the maximum response is compared to the minimum response (e.g., t)max-tminValue of (d) the greater the likelihood of having depression. The smaller the difference between the maximum response and the minimum response (e.g. a) in the same time periodmax-aminValue of) indicating that it is not sensitive to external stimuli, it has a possibility of depressionThe greater the sex.
In addition, patients with depression may also develop an overstimulation, σΔThe degree of dispersion of the change of the vegetative nerve signal values in two times is characterized, if sigmaΔThe larger the number of the patients, the more obvious the mood fluctuation is, and the more possible the patients have depression.
Figure BDA0003407925380000081
The difference between the maximum and minimum degree of change is characterized, and a larger difference indicates a more drastic reaction and a greater likelihood of suffering from depression.
And S103, determining the emotion coefficient based on the pupil area.
The pupillary area represents the response of the user to external stimuli, when sympathetic nerves are excited, if the sympathetic nerves show panic and uneasy pain, the pupils can be expanded, and the emotional response of the user to the current emotional stimuli can be known through the pupillary area.
In particular, the method comprises the following steps of,
s103-1, forming the pupil areas into a pupil area set.
Each element in the pupil area set corresponds to the pupil area acquired at a moment, and the elements in the pupil area set are arranged from far to near according to the acquisition moments.
Taking the data shown in table 1 obtained in step 101 as an example, the pupil area set a includes 5 elements, and the set a ═ a0,A1,A2,A3,A4}。
S103-2, determining the difference value between every two adjacent elements in the pupil area set to form a pupil area difference set.
Where the difference is the value of the next element-the value of the previous element.
Still taking the example in step S103-1 as an example, the pupil area difference set Δ a includes 4 elements, and the set Δ a ═ a1-A0,A2-A1,A3-A2,A4-A3}。
If A is to be1-A0Is marked as b0，A2-A1Is marked as b1，A3-A2Is marked as b2，A4-A3Is marked as b3If Δ a ═ b, then0,b1,b2,b3}。
That is, any element in the set Δ A (e.g., b)i) Having a value of Ai+1-AiI.e. bi＝Ai+1-Ai。
S103-3, determining the emotion coefficient according to the pupil area difference set.
For example, the emotion coefficient is determined by the following scheme:
s103-3-1, determining the average value avg of all elements in the pupil area difference setΔ。
For example,
Figure BDA0003407925380000091
s103-3-2, determining the average value avg of all elements in the pupil area set.
For example,
Figure BDA0003407925380000092
s103-3-3, according to avgΔAvg and the set of pupil areas determine a set of positive pupil areas and a set of negative pupil areas.
The implementation schemes of this step can be various, only 2 of them are described below, and one of them can be selected optionally when the implementation is specific.
A first implementation:
1.1 determine the difference of each element in the set of pupil areas from avg.
For example, determine A0-avg，A1-avg，A2-avg，A3-avg，A4-avg。
For convenience of description, A will be0Avg is denoted c0A is1Avg is denoted c1A is2Avg is denoted c2A is3Avg is denoted c3A is4Avg is denoted c4。
1.2 will be poorIs a positive number and the absolute value of the difference is greater than avgΔForms a positive set of pupil areas. The difference is negative and the absolute value of the difference is greater than min { avg }ΔThe elements of 2.5 form a negative set of pupil areas.
Wherein, min { avgΔ2.5} is avgΔAnd the smaller value of 2.5, min { } is the minimum function.
That is, if avgΔ<2.5, then min { avg }Δ,2.5}＝avgΔIf avg isΔ>2.5, then min { avg }Δ2.5} 2.5 if avgΔWhen the value is 2.5, avg is takenΔOr 2.5.
If c is0Is a positive number, and | c0|>avgΔThen c is0Are elements of the positive pupil area set.
If c is1Is a positive number, but | c1|≤avgΔThen c is1Not an element of the set of positive pupil areas, and, in addition, c1Nor are elements in the negative pupil area set.
If c is2Is a negative number, and | c2|>min{avgΔ2.5}, then c2Are elements in the negative pupil area set.
If c is3Is a negative number, and | c3|≤min{avgΔ2.5}, then c3Not being an element of the negative set of pupil areas, and c3Nor are the elements in the positive pupil area set.
If c is4Is a positive number, and | c4|>avgΔThen c is4Are elements of the positive pupil area set.
At this time, the positive pupil area set includes 2 elements, respectively c0And c4. The negative pupil area set includes only 1 element, i.e. c2。
A second implementation:
2.1 determine the difference of each element in the set of pupil areas from avg.
For example, determine A0-avg，A1-avg，A2-avg，A3-avg，A4-avg。
For convenience of description, A will be0Avg is denoted c0A is1Avg is denoted c1A is2Avg is denoted c2A is3Avg is denoted c3A is4Avg is denoted c4。
2.2 the difference is positive and the absolute value of the difference is greater than avgΔMeanwhile, the elements whose corresponding element values in the signal difference set are greater than the mean value of the elements in the signal difference set form a positive pupil area set. The difference is negative and the absolute value of the difference is greater than min { avg }Δ2.5, while elements in the set of signal differences that correspond to element values greater than the mean of the elements in the set of signal differences form a set of negative pupil areas.
Wherein, any difference is a positive number, and the absolute value of the difference is larger than avgΔOr, negative for any difference, and the absolute value of the difference is greater than min { avg [ ]Δ2.5} whose corresponding element value in the set of signal differences is:
and if any element is obtained from the last element in the pupil area set, determining the value of the last element in the signal difference set as the corresponding element value in the signal difference set.
If any element is not obtained from the last element in the pupil area set, determining the acquisition time t of any elementiCorresponding t in the signal difference seti+1Temporal autonomic nerve signaling and tiThe value of an element of the difference between the vegetative nerve signals at a time is determined as its corresponding element value in the set of signal differences.
If c is0Is a positive number, and | c0|>avgΔThen due to c0＝A0-avg, which is represented by A0Thus obtaining the product. Meanwhile, the pupil area set a ═ { a ═ a0，A1,A2,A3,A4In (A)0Not the last element, and c is therefore0Not from the last element in the set of pupil areas, at which point c is determined0At the acquisition time tiIs A0At the acquisition time t1Corresponding t in the signal difference set1+1Time of day (i.e. t)2Time) vegetative nerve signal and t1The element of the difference between the vegetative nerve signals at a given time (i.e., S)1-S0＝a0) Is determined as its corresponding element value in the signal difference set. That is, c0The corresponding element value in the signal difference set is a0If a has a value of0Not greater than the set of signal differences Δ S ═ a0,a1,a2,a3Mean of elements in (i.e. mean of elements in)
Figure BDA0003407925380000111
) Then c is0Elements of the set of non-positive pupil areas, and, in addition, c0Nor are elements in the negative pupil area set.
If c is1Is a positive number, but | c1|≤avgΔThen c is1Not an element of the set of positive pupil areas, and, in addition, c1Nor are elements in the negative pupil area set.
If c is2Is a negative number, and | c2|>min{avgΔ2.5}, then due to c2＝A2-avg, which is represented by A2Thus obtaining the product. Meanwhile, the pupil area set a ═ { a ═ a0,A1,A2,A3,A4In (A)2Not the last element, and c is therefore2Not from the last element in the set of pupil areas, at which point c is determined2At the acquisition time tiIs A2At the acquisition time t3Corresponding t in the signal difference set3+1Time of day (i.e. t)4Time) vegetative nerve signal and t3The element of the difference between the vegetative nerve signals at a given time (i.e., S)4-S3＝a3) Is determined as its corresponding element value in the signal difference set. That is, c2The corresponding element value in the signal difference set is a3If a has a value of3Greater than signal difference set Δ S ═ a0,a1,a2,a3Mean of elements in (i.e. mean of elements in)
Figure BDA0003407925380000112
) Then c is2Are elements in the negative pupil area set.
If c is3Is a negative number, and | c3|≤min{avgΔ2.5}, then c3Not being an element of the negative set of pupil areas, and c3Nor are the elements in the positive pupil area set.
If c is4Is a positive number, and | c4|>avgΔThen due to c4＝A4-avg, which is represented by A4Thus obtaining the product. Meanwhile, the pupil area set a ═ { a ═ a0,A1,A2,A3,A4In (A)4Is the last element, therefore c4The signal difference set delta S is obtained from the last element in the pupil area set, and is { a ═ a }0,a1,a2,a3The last element a in3Is determined as c4In the signal difference set Δ S ═ a0,a1,a2,a3The corresponding element value in (b), if a3Greater than signal difference set Δ S ═ a0,a1,a2,a3Mean of elements in (i.e. mean of elements in)
Figure BDA0003407925380000121
Figure BDA0003407925380000122
) Then c is4Are elements of the positive pupil area set.
At this time, the set of positive pupil areas includes only 1 element, i.e., c4. The negative pupil area set includes only 1 element, i.e. c2。
S103-3-4, determining the emotion coefficient according to the positive pupil area set and the negative pupil area set.
In particular, the method comprises the following steps of,
determining the maximum element value in the positive pupil area set
Figure BDA0003407925380000123
Element with minimum sum
Figure BDA0003407925380000124
And
Figure BDA0003407925380000125
corresponding time of day
Figure BDA0003407925380000126
And
Figure BDA0003407925380000127
corresponding time of day
Figure BDA0003407925380000128
Determining a maximum element value in a set of negative pupil areas
Figure BDA0003407925380000129
Element with minimum sum
Figure BDA00034079253800001210
And
Figure BDA00034079253800001211
corresponding time of day
Figure BDA00034079253800001212
And
Figure BDA00034079253800001213
corresponding time of day
Figure BDA00034079253800001214
Determining the maximum element value in the pupil area difference set
Figure BDA00034079253800001215
Corresponding element value b in the set of pupil areas1Element with minimum sum
Figure BDA00034079253800001216
In the collection of pupil areasCorresponding element value b of2。
Wherein, b1Obtained from a set of pupil areas
Figure BDA00034079253800001217
Of two elements (e.g., if the largest element in the pupil area difference set has a value of b1A value of (i), i.e
Figure BDA00034079253800001218
Then b1＝max{A1,A2I.e. b1Is A1And A2The one with the largest median value), b2Obtained from a set of pupil areas
Figure BDA00034079253800001219
Of the two elements (e.g., if the smallest element in the set of pupil area differences has a value of b3A value of (i), i.e
Figure BDA00034079253800001220
Then b2＝max{A3,A4I.e. b2Is A3And A4The one with the largest median).
Determining positive coefficients
Figure BDA0003407925380000131
Negative coefficient of friction
Figure BDA0003407925380000132
Figure BDA0003407925380000133
Determine emotion coefficient I2 ═ max { I ═ I+，I-}。
Figure BDA0003407925380000134
The difference between the maximum area and the minimum area of the pupil is represented, | b1-b2I represents the maximum pupil area of the largest variation andmaximum pupil area of minimum variation, by
Figure BDA0003407925380000135
And | b1-b2The maximum value of | characterizes the maximum mood swings of the user for the mood stimuli, and a larger value indicates a larger mood swing and a higher probability of suffering from depression.
Figure BDA0003407925380000136
The time difference between the maximum and minimum pupillary area is characterized, and the shorter this time indicates the more pronounced the mood swings, the greater its probability of suffering from depression.
And S104, carrying out depression detection on the facial video based on the emotional coefficient and the mood coefficient to obtain a depression detection result.
In particular, the method comprises the following steps of,
s104-1, identifying the micro expression of each frame in the facial video.
The existing micro expression recognition scheme is adopted in the step, and details are not repeated here.
S104-2, determining the variation degree among the micro expressions of each frame.
The existing micro-expression analysis method is also adopted, and the expression change between the previous frame and the next frame is determined by the method.
The degree of change can be represented by various factors, for example, the degree of change is the number of changed micro-expression feature points, or the degree of change is the number of changed micro-expression feature points/total number of micro-expression feature points, or the degree of change is the average distance of the changed micro-expression feature points, wherein the average distance is the position difference of each micro-expression feature point in the two frames before and after.
S104-3, determining the maximum number of continuous frames with the change degree not larger than the change threshold.
Wherein the variation threshold is an empirical value and can be set in advance. Or training through sample data.
In addition, the continuous frames include all frames involved in a degree of change not greater than the change threshold. That is, the degree of change is derived from the difference between two frames, which are both the frames they refer to.
In this step, the variation of two adjacent frames is calculated in sequence. And determining the relationship between each degree of change and the change threshold value respectively.
For example, as shown in table 2:
TABLE 2
Figure BDA0003407925380000141
In the data shown in Table 2, there are 3 consecutive frames having a degree of change not greater than the change threshold, the first segment being D0Corresponding frame F0And F1The second stage is D2And D3Corresponding frame F2、F3And F4And the third section is D5、D6And D7Corresponding frame F5、F6、F7And F8。
Then the maximum number is 4 (i.e., F)5、F6、F7And F8)。
The maximum number of consecutive frames characterizes the maximum number of frames that the user is unresponsive to, by which the duration of the reaction is longer, i.e. the maximum time that the user is unresponsive to receiving a emotional stimulus, the greater the likelihood that he will suffer from depression, since the frames are also chronologically present, i.e. how many frames per minute in the video are fixed.
And S104-4, determining the maximum value I1I 2.
Wherein, I1 is the emotion coefficient, and I2 is the emotion coefficient.
And S104-5, if the detection value is larger than the depression threshold value, determining that the depression is detected.
Where the depression threshold is an empirical value, it may be set in advance. Or training through sample data.
According to the method provided by the embodiment, the vegetative nerve signals, the face video and the pupil area of a user are obtained when the user receives emotional stimulation; determining an emotion coefficient based on the autonomic nervous signal; determining an emotion coefficient based on the pupil area; and carrying out depression detection on the facial video based on the emotional coefficient and the mood coefficient to obtain a depression detection result, thereby realizing automatic depression detection.
In order to better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions.
Furthermore, it should be noted that in the description of the present specification, the description of the term "one embodiment", "some embodiments", "examples", "specific examples" or "some examples", etc., means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, the claims should be construed to include preferred embodiments and all changes and modifications that fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include such modifications and variations.

Claims (10)

1. A method for depression detection based on video analytics, the method comprising:
s101, acquiring vegetative nerve signals, facial videos and pupil areas of a user when receiving emotional stimulation;
s102, determining an emotion coefficient based on the autonomic nervous signals;
s103, determining an emotion coefficient based on the pupil area;
and S104, carrying out depression detection on the facial video based on the emotion coefficient and the emotion coefficient to obtain a depression detection result.
2. The method according to claim 1, wherein the S102 includes:
s102-1, forming the vegetative nerve signals into a signal set; each element in the signal set corresponds to an vegetative nerve signal value acquired at a moment, and the elements in the signal set are arranged from far to near according to the acquisition moments;
s102-2, determining a difference value between every two adjacent elements in the signal set to form a signal difference set, wherein the difference value is a value of a next element-a value of a previous element;
and S102-3, determining the emotion coefficient according to the signal difference set.
3. The method according to claim 2, wherein the S102-3 comprises:
s102-3-1, determining standard deviation sigma of all elements in the signal difference setΔ；
S102-3-2, determining the element with the maximum value in the signal difference set
Figure FDA0003407925370000011
Element with minimum sum
Figure FDA0003407925370000012
S102-3-3, determining the element a with the largest value in the signal setmaxElement a with the smallest summinAnd the time t corresponding to the element with the largest valuemaxTime t corresponding to the element with the smallest valuemin；
S102-3-4, determining the emotional coefficient
Figure FDA0003407925370000013
4. The method according to claim 3, wherein the S103 comprises:
s103-1, forming a pupil area set by the pupil areas; each element in the pupil area set corresponds to a pupil area acquired at a moment, and the elements in the pupil area set are arranged from far to near according to the acquisition moments;
s103-2, determining a difference value between every two adjacent elements in the pupil area set to form a pupil area difference set, wherein the difference value is a value of a next element-a value of a previous element;
s103-3, determining an emotion coefficient according to the pupil area difference set.
5. The method according to claim 4, wherein the S103-3 comprises:
s103-3-1, determining the mean value avg of all elements in the pupil area difference setΔ；
S103-3-2, determining the mean value avg of all elements in the pupil area set;
s103-3-3, according to the avgΔDetermining a positive pupil area set and a negative pupil area set by the avg and the pupil area set;
s103-3-4, determining an emotion coefficient according to the positive pupil area set and the negative pupil area set.
6. The method of claim 5, wherein the S103-3-3 comprises:
determining a difference of each element in the set of pupil areas and avg;
the difference is a positive number, and the absolute value of the difference is greater than avgΔThe elements of (a) form a positive pupil area set;
the difference is negative and the absolute value of the difference is greater than min { avg }ΔThe elements of 2.5 form a negative set of pupil areas.
7. The method of claim 5, wherein the S103-3-3 comprises:
determining a difference of each element in the set of pupil areas and avg;
the difference is a positive number, and the absolute value of the difference is greater than avgΔMeanwhile, elements with corresponding element values larger than the element mean value in the signal difference set form a positive pupil area set;
the difference is negative and the absolute value of the difference is greater than min { avg }ΔAnd 2.5, and simultaneously, forming a negative pupil area set by elements of which the corresponding element values in the signal difference set are larger than the element mean values in the signal difference set.
8. The method of claim 7, wherein the step of removing the metal oxide is performed by a chemical vapor deposition processIn that, any difference is a positive number, and the absolute value of the difference is greater than avgΔOr, negative for any difference, and the absolute value of the difference is greater than min { avg [ ]Δ2.5} whose corresponding element value in the set of signal differences is:
if any element is obtained from the last element in the pupil area set, determining the value of the last element in the signal difference set as the corresponding element value in the signal difference set;
if any element is not obtained from the last element in the pupil area set, determining the acquisition time t of any elementiCorresponding to the t-th signal difference in the signal difference seti+1Temporal autonomic nerve signaling and tiThe values of the elements of the difference between the vegetative nerve signals at a time are determined as their corresponding element values in the set of signal differences.
9. The method according to claim 5, wherein the S103-3-4 comprises:
determining a maximum element value in the set of positive pupil areas
Figure FDA0003407925370000031
Element with minimum sum
Figure FDA0003407925370000032
And
Figure FDA0003407925370000033
corresponding time of day
Figure FDA0003407925370000034
And
Figure FDA0003407925370000035
corresponding time of day
Figure FDA0003407925370000036
Determining a maximum element value in the set of negative pupil areas
Figure FDA0003407925370000037
Element with minimum sum
Figure FDA0003407925370000038
And
Figure FDA0003407925370000039
corresponding time of day
Figure FDA00034079253700000310
And
Figure FDA00034079253700000311
corresponding time of day
Figure FDA00034079253700000312
Determining the maximum element value in the pupil area difference set
Figure FDA00034079253700000313
Corresponding element value b in the set of pupil areas1Element with minimum sum
Figure FDA00034079253700000314
Corresponding element value b in the set of pupil areas2(ii) a Wherein, b is1For obtaining from the set of pupil areas
Figure FDA00034079253700000315
Of the two elements of (a) has the largest value, b2For obtaining from the set of pupil areas
Figure FDA00034079253700000316
The element with the largest value of the two elements;
determining the normal systemNumber of
Figure FDA00034079253700000317
Negative coefficient of friction
Figure FDA00034079253700000318
Figure FDA00034079253700000319
Determine emotion coefficient I2 ═ max { I ═ I+，I-}。
10. The method according to claim 1, wherein the S104 comprises:
s104-1, identifying the micro expression of each frame in the facial video;
s104-2, determining the change degree among the micro expressions of each frame;
s104-3, determining the maximum number of continuous frames with the change degree not greater than a change threshold;
s104-4, determining the maximum value I1I 2; wherein I1 is an emotion coefficient, and I2 is an emotion coefficient;
s104-5, if the detection value is larger than a depression threshold value, determining that depression is detected.
CN202111522156.2A 2021-12-13 2021-12-13 Method for performing video analysis-based depression detection by computer Active CN114209322B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111522156.2A CN114209322B (en) 2021-12-13 2021-12-13 Method for performing video analysis-based depression detection by computer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111522156.2A CN114209322B (en) 2021-12-13 2021-12-13 Method for performing video analysis-based depression detection by computer

Publications (2)

Publication Number Publication Date
CN114209322A true CN114209322A (en) 2022-03-22
CN114209322B CN114209322B (en) 2023-08-29

Family

ID=80701634

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111522156.2A Active CN114209322B (en) 2021-12-13 2021-12-13 Method for performing video analysis-based depression detection by computer

Country Status (1)

Country Link
CN (1) CN114209322B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105559802A (en) * 2015-07-29 2016-05-11 北京工业大学 Tristimania diagnosis system and method based on attention and emotion information fusion
CN111967363A (en) * 2020-08-10 2020-11-20 河海大学 Emotion prediction method based on micro-expression recognition and eye movement tracking
KR20210066697A (en) * 2019-11-28 2021-06-07 경희대학교 산학협력단 Apparatus and method for predicting human depression level using multi-layer bi-lstm with spatial and dynamic information of video frames

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105559802A (en) * 2015-07-29 2016-05-11 北京工业大学 Tristimania diagnosis system and method based on attention and emotion information fusion
KR20210066697A (en) * 2019-11-28 2021-06-07 경희대학교 산학협력단 Apparatus and method for predicting human depression level using multi-layer bi-lstm with spatial and dynamic information of video frames
CN111967363A (en) * 2020-08-10 2020-11-20 河海大学 Emotion prediction method based on micro-expression recognition and eye movement tracking

Also Published As

Publication number Publication date
CN114209322B (en) 2023-08-29

Similar Documents

Publication Publication Date Title
Can et al. Personal stress-level clustering and decision-level smoothing to enhance the performance of ambulatory stress detection with smartwatches
US7761144B2 (en) Method, apparatus, and computer program product for stochastic psycho-physiological assessment of attentional impairments
Throckmorton et al. Bayesian approach to dynamically controlling data collection in P300 spellers
Lee et al. Three cases of developmental prosopagnosia from one family: Detailed neuropsychological and psychophysical investigation of face processing
Yuvaraj et al. Detection of emotions in Parkinson's disease using higher order spectral features from brain's electrical activity
JP6124140B2 (en) Assessment of patient cognitive function
AU2019246815B2 (en) Systems and methods for diagnosing sleep
US20160029946A1 (en) Wavelet analysis in neuro diagnostics
WO2015111331A1 (en) Cognitive function evaluation apparatus, method, system, and program
KR20150076167A (en) Systems and methods for sensory and cognitive profiling
Al-Nuaimi et al. Tsallis entropy as a biomarker for detection of Alzheimer's disease
Vick et al. Data-driven subclassification of speech sound disorders in preschool children
EP2884890A1 (en) Method for detection of an abnormal sleep pattern in a person
Gunnarsdottir et al. A novel sleep stage scoring system: combining expert-based rules with a decision tree classifier
Skorucak et al. Automatic detection of microsleep episodes with feature-based machine learning
JP2023538765A (en) Method and system for quantifying attention
CN114190942B (en) Method for computer-implemented depression detection based on audio analysis
CN114209322A (en) Method for detecting depression based on video analysis
Van Ingelghem et al. An auditory temporal processing deficit in children with dyslexia
Gonzalez-Navarro et al. Feedback related potentials for EEG-based typing systems
Verhoef et al. Bio-sensing for emotional characterization without word labels
CN110675953B (en) System for identifying psychotic patients using artificial intelligence and big data screening
CN113712542A (en) Tinnitus electroencephalogram convolutional neural network model construction method based on deep learning
Schönweiler et al. Classification of passive auditory event-related potentials using discriminant analysis and self-organizing feature maps
Zlibut et al. Identifying and describing subtypes of spontaneous empathic facial expression production in autistic adults

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 301, 3rd Floor, No. 8 Sijiqing Road, Haidian District, Beijing, 100195

Applicant after: WOMIN HIGH-NEW SCIENCE & TECHNOLOGY (BEIJING) CO.,LTD.

Address before: 100086 West, 14th floor, block B, Haidian culture and art building, 28a Zhongguancun Street, Haidian District, Beijing

Applicant before: WOMIN HIGH-NEW SCIENCE & TECHNOLOGY (BEIJING) CO.,LTD.

GR01 Patent grant
GR01 Patent grant