RU2777948C1 - Method for objective evaluation of facial expressions in patients with parkinson's disease - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: claimed invention relates to the field of medicine, namely to neurology, neurorehabilitation and can be used to diagnose hypomimia in patients with Parkinson’s disease for diagnostic purposes when diagnosing Parkinson’s disease, as well as analyzing the effectiveness of therapy, rehabilitation measures. A method for diagnosing Parkinson’s disease in patients is proposed, including video analysis of the kinematic parameters of the movements of the eyebrows, eyelids and mouth by facial reference points, characterized in that the patient performs 6 diagnostic tests: test 1, “smile”, in which the patient is asked to smile with maximum amplitude and speed, baring his teeth, for 10 seconds, on the basis of which the movement of the mouth along the X axis, the Y axis in mm, the number of smiles in 10 seconds are estimated, test 2, the “letter O”, in which the patient is asked to draw the letter “O” with closed lips, stretched forward, covering the maximum area for 10 seconds, on the basis of which the area of the drawn letter “O” is estimated in mm², test 3, “winking”, in which the patient is asked to wink at maximum speed in for 10 seconds, based on which the number of blinks for 10 seconds is estimated, test 4 test, “eyebrow lift”, in which the patient is asked to bring the eyebrows together with maximum amplitude and speed, and then raise the eyebrows up, spreading them apart, on the basis of which the movement of the eyebrows along the X axis and Y axis in mm is estimated, as well as the number of eyebrow lifts up in 10 seconds, test 5, “frown eyebrows”, in which the patient is asked to bring the eyebrows together and apart with maximum speed and amplitude, on the basis of which the movement of the eyebrows along the X axis and Y axis in mm is estimated, as well as the number of frowns in 10 seconds, test 6, “reading”, in which the patient is asked to read the text with an expression, on the basis of which the movement of the eyebrows, mouth along the X axis and the Y axis is evaluated during reading. In this case, kinematic parameters are evaluated in a non-contact way. Then the movement of the eyebrows, eyelids and mouth is analyzed in diagnostic tests of the amplitude of the movements of the eyebrows and mouth along the X axis and Y axis, and the frequency of raising eyebrows, frowns, winks, smiles. After that, each parameter is compared with the reference values of the kinematic parameters of the movements of the facial muscles. With values of all 9 indicators less than their reference values, the presence of hypomimia is determined, with values of all kinematic indicators of movements of facial muscles greater than their reference values, its absence is determined.

EFFECT: invention provides an analysis of the kinematic parameters of facial musculature movements, which makes it possible to compare the facial expressions of healthy people and patients with PD, as well as to evaluate the effectiveness of drug therapy and rehabilitation.

1 cl, 2 dwg, 2 ex

Description

The method is designed to analyze the kinematic parameters of movements of the mimic muscles of the face in patients with Parkinson's disease in order to detect hypomimia. Due to the fact that hypomimia is one of the manifestations of hypokinesia, its assessment is an important aspect in the clinical practice of a parkinsonologist. The method of objective assessment of facial expressions can be used for diagnostic purposes in establishing the diagnosis of Parkinson's disease, as well as analyzing the effectiveness of therapy and rehabilitation measures.

The most common method for assessing facial expressions in patients with Parkinson's disease is the International Movement Disorders Society's Unified Parkinson's Disease Rating Scale (MDS UPDRS) part 3, paragraph 3.2. facial expression. When conducting a clinical examination, the doctor observes the facial expressions of a person in a relaxed state for 10 seconds. The expression of the patient's face is estimated from 0 to 4 points, where 0 points means the absence of violations of the movements of mimic muscles, and 4 points is characterized by a pronounced degree of hypomimia [Bucka P.O., Wilsonb R.E., Seebergerc L.C., Connera J.B., Castelli-Haley J. Examination of the UPDRS Bradykinesia Subscale: Equivalence, Reliability and Validity. Journal of Parkinson's Disease. (2011):1;253–258. DOI 10.3233/JPD-2011-11035].

The disadvantages of this method are its subjectivity, the inability to assess the amplitude and speed of facial muscle movements, which does not allow an objective assessment of the manifestations of hypomimia.

One of the possible methods for assessing facial expressions is computer digital visualization with an assessment of quantitative dynamic indicators. The time indicator is also evaluated, during which the maximum expressiveness of the face is achieved in the manifestation of such emotions as happiness, sadness, fear, anger, disgust, surprise. During the study, a tape with 2 light-emitting diodes fixed on it is fixed on the upper part of the frontal area, in this way the face was recognized. Subjects are asked to depict the emotions indicated above on their faces as vividly and expressively as possible. Data capture and digitization is carried out using a Sony video player, a personal computer with an Iscan-PCI video card, and EYEVIEW (Imaging Technology) software. The facial expression is analyzed by automatically calculating the change in pixel intensity with each successive frame. There is a sequential comparison of frames of images during the execution of each of the emotions by the patient. Face recognition involves 20 reference points, which were chosen empirically. [Bowers D., Miller K., Bosch W., Gokcay D., Pedraza O., Springer U., Okun M. Faces of emotion in Parkinsons disease: Micro-expressivity and bradykinesia during voluntary facial expressions. Journal of the International Neuropsychological Society. 2006;12(6):765–773. DOI 10.10170S135561770606111X].

This method is experimental and has a small evidence base. This program does not provide an opportunity to obtain amplitude and speed indicators with more accurate units of measurement.

The closest analogue of an objective assessment of facial expressions is a method using a 3D optoelectronic system (SMART motion, and SMART Analyzer BTS Engineering, Milan, Italy). This equipment has a wide range of applications for video analysis of movements in neurology, neurorehabilitation, traumatology, orthopedics and sports medicine. In particular, to assess facial expressions, three infrared cameras were used, which fix the displacement of reflective markers, which are adhesive reflective material 5 mm in diameter, which are light sources in the visible or invisible (infrared) part of the light spectrum, fixed on the face of the subject. Two markers were placed in the corners of the mouth, two in the middle of the lower eyelid, and two in the medial corners of the eyebrows. As a result, the peak speed and amplitude of movements of the eyebrows, eyelids and mouth were analyzed. During the study, patients were asked to imitate the emotions presented in the images that were displayed on a computer screen for 5 seconds. Movements of the lower face were studied by asking patients to show their teeth "as quickly and as wide as possible". Thus, the peak speed was measured in mm/sec., and the range of motion in mm. [Marsili L., Agostino R., Bologna M., Belvisi D., Palma A., Fabbrini G., Berardelli A. Bradykinesia of posed smiling and voluntary movement of the lower face in Parkinson’s disease. Parkinsonism and Related Disorders.2014;20(4):370-375. DOI 10.1016/j.parkreldis.2014.01.013].

The disadvantage is the high cost of equipment, the complexity of the diagnostic method and the need for special long-term training of personnel performing diagnostics.

The objective of the proposed method is to objectively assess facial expressions in patients with Parkinson's disease in order to identify a decrease in amplitude and speed indicators, which reflects the presence of hypomimia in patients. Since this syndrome is a characteristic manifestation of Parkinson's disease, this will improve the diagnosis of this disease, and will also allow us to evaluate the effectiveness of L-DOPA therapy and rehabilitation measures.

The task is carried out by analyzing the kinematic indicators of the movements of the facial muscles, which show the amplitude and speed of the movements of the eyebrows, mouth, blinking frequency.

This method is based on the assessment of movements of the facial muscles using computer video analysis of facial expressions using a software and hardware complex consisting of software (SW) created in the Visual studio 2015 programming environment using the PostgreSQL database management system and a video camera with a resolution of 1280x720.

Assessment of facial muscle movements is carried out in a non-contact way by recording video materials on a webcam. The software detects the coordinates of 68 key facial points using the open source Dlib computer vision library of the Python programming language. The patient is asked to perform 6 diagnostic tests, during which the movement of the eyebrows, eyelids, and mouth is evaluated.

The software analyzes the movement of the eyebrows, eyelids and mouth in 6 diagnostic tests and issues a report in Excel spreadsheet format. This table shows the amplitudes of movements of the eyebrows and mouth along the X axis and the Y axis in mm, and the frequency (number / 10 seconds) of raising the eyebrows, frowning, blinking, smiling. Then there is a comparison with all reference values, which determine the presence or absence of hypomimia.

If the indicator is less than all the reference values, then there is hypomimia, if more, then it is not.

The method of objective assessment of facial expressions is as follows: the patient is placed sitting on a chair at a distance of 64 cm from the webcam at eye level (figure 1).

During the video analysis of the movements of mimic muscles, the patient is asked to perform 6 diagnostic tests.

1 test - "smile". The patient is asked to smile with maximum amplitude and speed, baring his teeth, for 10 seconds. This test evaluates the movement of the mouth along the X axis, the Y axis in mm, the number of smiles in 10 seconds.

2 test - "letter O". The patient is asked to draw the letter “O” with closed lips stretched forward, covering the maximum area within 10 seconds. This test evaluates the area of the drawn letter "O" in mm ² .

3 test - "blinking". The patient is asked to blink at maximum speed for 10 seconds. This test evaluates the number of blinks within 10 seconds.

4 test - "raising the eyebrows." The patient is invited to bring the eyebrows together with maximum amplitude and speed, and then raise the eyebrows up, spreading them apart. This test evaluates the movement of the eyebrows along the X-axis and the Y-axis in mm, as well as the number of eyebrows raised in 10 seconds.

5 test - "frown your eyebrows." The patient is offered to reduce and part the eyebrows with maximum speed and amplitude. This test evaluates the movement of the eyebrows along the X-axis and the Y-axis in mm, as well as the number of frowns in 10 seconds.

6 test - "reading". The patient is invited to read the text “Lake Baikal” with an expression; the time period for completing the task is not limited. This test assesses the movement of the eyebrows, mouth along the X-axis and Y-axis during reading.

At the time of performing diagnostic tests, the software detects the coordinates of 68 key points of the face, in a two-dimensional coordinate system, using the open computer vision library of the Python programming language (Fig. 2.) The amplitude characteristics of the movements of the eyebrows, eyelids, and mouth are determined by the following facial landmarks:

The movement of the mouth along the X axis is determined by the points: 49.61, 55, 65. Along the Y axis: 51, 53, 59 and 57 points. In the test, the letter "O" is the average values of the Y and X coordinates of points from 61 to 68. It is calculated using the Gauss area formula for irregular closed polygons. Eyebrow movement along the X axis is determined by points: 20 and 25, and along the Y axis: 20, 21, 24, 25 points.

Next, the software converts the obtained values from pixels to mm. These data are displayed after processing in an Excel spreadsheet. The data obtained are compared with all reference values, thus determining the presence of hypomia.

The sensitivity and specificity of the method are presented in Table 1.

Assessment of diagnostic value with the determination of sensitivity and specificity, identification of reference values was carried out using the IBM SPSS Statistics 23 software package, when conducting a statistical ROC analysis method.

Table 1. Diagnostic value of the method.

diagnostic test sign Sensitivity Specificity Reference values AUC CI 95% R Smile x mouth 81.3% 76.7% >30.32 0.890 [0.811;0.968] <0.001 At the mouth 84.4% 53.3% >19.91 0.789 [0.675;0.903] <0.001 Frequency (times/10 sec.) 84.4% 76.7% >9 0.873 [0.781;0.965] <0.001 Letter "O" S 87.1% 87.1% >215 0.940 [0.884;0.996] <0.001 blinking Frequency (count/10 sec) 90.3% 64.5% >42 0.846 [0.752;0.940] <0.001 Eyebrow lift X eyebrows 80.6% 71% >12.73 0.841 [0.741;0.941] <0.001 At the eyebrows 80.6% 71% >21.32 0.823 [0.716;0.929] <0.001 Frequency (count/10 sec) 90.3% 90.3% >7 0.935 [0.863;1] <0.001 To frown X eyebrows 80.6% 61.3% >13.82 0.790 [0.674;0.905] <0.001 At the eyebrows 80.6% 61.3% >13.83 0.817 [0.712;0.923] <0.001 Frequency (count/10 sec) 80.6% 96.8% >6 0.934 [0.865;1] <0.001 Reading x mouth 80.6% 67.7% >8.99 0.760 [0.638;0.882] <0.001 Y mouth 77.4% 61.3% >15.15 0.706 [0.576;0.835] 0.005 X eyebrows 54.8% 48.4 >2.24 0.597 [0.454;0.740] 0.188 Y eyebrows 80.6% 71% >1.33 0.783 [0.663;0.902] <0.001

As a result of the application of this method, in patients with Parkinson's disease, a decrease in the amplitude and speed indicators of the movement of mimic muscles was determined, namely: movements of the eyebrows, eyelids, mouth, which indicates hypomimia.

Approbation of the method

The study was conducted at the Department of Nervous Diseases with a PO course of the Krasnoyarsk State Medical University named after Professor V.F. Voyno-Yasenetsky. The kinematic parameters of movements of mimic muscles in a clinically healthy patient and a patient with Parkinson's disease were analyzed by the method of an objective assessment of facial expressions, and facial expressions were also evaluated at the end and peak of the dose of levodopa.

Clinical example No. 1.

When conducting a video analysis of the movements of the mimic muscles of the face in a clinically healthy woman aged 72 years and a patient with Parkinson's disease, 71 years old, 2.0 st. According to Hen and Yahr, differences were revealed at the outcome of the action of Levodopa, which are presented in Table 2.

Table 2. Kinematic indicators of facial expressions in a patient with Parkinson's disease and a clinically healthy patient.

No. diagnostic test accounting signs PD patient
71 years old Clinically healthy woman
72 years old Reference values one. "Smile" X mouth (mm) 11.93 41.46 >30.32 At the mouth (mm) 6.51 31.35 >19.91 Frequency (times/10 sec.) eight 13 >9 2. Letter "O" S (mm ²⁾ 193 408 >215 3. blinking Frequency (times/10 sec.) 32 43 >42 four. Eyebrow lift X eyebrow (mm) 10.27 20.70 >12.73 Eyebrow (mm) 18.70 28.16 >21.32 Frequency (times/10 sec.) 6 9 >7 5. "To frown" X eyebrow (mm) 13.70 16.40 >13.82 Eyebrow (mm) 8.33 13.90 >13.83 Frequency (times/10 sec.) 6 9 >6 6. Reading text X mouth (mm) 8.13 22.22 >8.99 At the mouth (mm) 15.08 21.93 >15.15 X eyebrow (mm) 2.3 2.45 >2.24 Eyebrow (mm) 1.21 2.43 >1.33

Reference values show that all indicators in a patient with Parkinson's disease are below normal, which confirms the presence of hypomimia as a manifestation of the disease, which is absent in a clinically healthy woman.

Clinical example 2.

Using this diagnostic method, the effect of dopaminergic therapy was evaluated in a patient with PD with Hoehn and Yahr stage 2.0. Table 3 presents the results of the movement of mimic muscles at the end and peak of the dose of action of the drug L-DOPA.

Table 3. Analysis of facial expressions at the end and peak of the action of L-DOPA preparations.

No. diagnostic test accounting signs PD patient
dose outcome PD patient
peak dose Reference
values one. "Smile" X mouth (mm) 11.93 28.77 >30.32 At the mouth (mm) 6.51 9.90 >19.91 Frequency (times/10 sec.) eight ten >9 2. Letter "O" S (mm ²⁾ 193 481 >215 3. blinking Frequency (times/10 sec.) 32 37 >42 four. Eyebrow lift X eyebrow (mm) 10.27 14.70 >12.73 Eyebrow (mm) 18.70 18.77 >21.32 Frequency (times/10 sec.) 6 12 >7 5. "To frown" X eyebrow (mm) 13.70 22.28 >13.82 Eyebrow (mm) 8.33 18.56 >13.83 Frequency (times/10 sec.) 6 ten >6 6. Reading text X mouth (mm) 8.13 11.73 >8.99 At the mouth (mm) 15.08 22.41 >15.15 X eyebrow (mm) 2.30 2.68 >2.24 Eyebrow (mm) 1.21 1.24 >1.33

According to the results of video analysis of the movements of mimic muscles, the following kinematic indicators improved: the amplitude of mouth movements along the X and Y axes, the frequency of smiles, the area of the drawn letter “O”, the blinking frequency, the movement of the eyebrows along the X and Y axes, the frequency of raising the eyebrows. In the reading test, an increase in the amplitude of movements of the mouth, eyebrows along the X and Y axes was observed. The use of this method for an objective assessment of facial expressions in patients with Parkinson's disease makes it possible to analyze the effect of L-DOPA preparations on hypomimia. From this clinical example, it can be seen that the drug L-DOPA has its own effectiveness and reduces the manifestation of hypomimia, such indicators as the frequency of smiles, the area of \u200b\u200bdrawing the letter "O", the amplitude of raising the eyebrows along the X axis, the frequency of raising the eyebrows, the amplitude when frowning the eyebrows along the X axis and Y, the frequency of frowns, as well as the amplitude indicators of mouth movements along the X and Y axes, eyebrows along the X axis became the same as in clinically healthy individuals when compared with the reference values.

The technical result of the method is an objective diagnosis of the presence of hypomia, which may indicate the presence of Parkinson's disease in a patient, as well as in assessing the effectiveness of the treatment in patients with previously diagnosed Parkinson's disease. The method is effective, simple and affordable to use, comfortable for the patient, eliminates the need for special training of personnel.

Claims (1)

A method for diagnosing patients with Parkinson's disease, including conducting a video analysis of the kinematic parameters of the movements of the eyebrows, eyelids and mouth along facial reference points, characterized in that the patient performs 6 diagnostic tests: 1 test - "smile", in which the patient is asked to smile with maximum amplitude and speed , showing teeth, for 10 seconds, on the basis of which the movement of the mouth along the X axis, Y axis in mm, the number of smiles in 10 seconds, 2 test - “letter O”, in which the patient is asked to draw the letter with closed lips, stretched forward, is estimated "O", covering the maximum area during 10 seconds, on the basis of which the area of the drawn letter "O" is estimated in mm ² , 3 test - "blinking", in which the patient is asked to blink at a maximum speed for 10 seconds, based on which evaluates the number of blinks within 10 seconds, 4 test - "raising the eyebrows", in which the patient is asked to reduce the maximum amplitude and speed eyebrows together, and then raise the eyebrows up, spreading them to the sides, on the basis of which the movement of the eyebrows along the X-axis and the Y-axis in mm is evaluated, as well as the number of eyebrow lifts up in 10 seconds, test 5 - “frown the eyebrows”, in which the patient it is proposed to reduce and spread the eyebrows with maximum speed and amplitude, on the basis of which the movement of the eyebrows along the X axis and the Y axis in mm is estimated, as well as the number of frowns in 10 seconds, 6 test - “reading”, in which the patient is asked to read the text with an expression, on the basis of which the movement of the eyebrows, mouth along the X-axis and the Y-axis during reading is evaluated, while the kinematic parameters are evaluated in a non-contact way, then the movement of the eyebrows, eyelids and mouth is analyzed in diagnostic tests of the amplitude of the movements of the eyebrows and mouth along the X-axis and Y-axis, and the frequency of raising the eyebrows, frowning, blinking, smiling; after which each parameter is compared with the reference values of the kinematic parameters of the movements of the mimic muscles; when the values of all 9 indicators are less than their reference values, the presence of hypomia is determined, when the values of all kinematic indicators of the movements of mimic muscles are greater than their reference values, its absence is determined.

Images