KR101634730B1 - Facial Nerve Palsy Grading Apparatus and Method - Google Patents

Abstract

The present invention relates to a facial nerve palsy evaluation apparatus and a flat bag method,
An image acquiring unit including a camera for successively photographing a face image of a subject; An image processor for converting the RGB image obtained from the camera into a YCbCr image and identifying a marker attached to a face of the subject; A facial nerve paralysis evaluating unit for evaluating whether the facial paralysis is occurring by calculating the degree of motion of the marker; And a display unit for displaying a result of calculating the degree of motion of the image and the marker of the subject received through the camera, thereby providing an effect of objectively and accurately evaluating whether or not the subject is facial paralysis.

Description

[0001] Facial Nerve Palsy Grading Apparatus and Method [0002]

The present invention relates to a facial paralysis evaluating apparatus using a video apparatus and a method thereof, and more particularly, to a facial paralysis evaluating apparatus using a video apparatus and a method thereof.

 Facial nerves, which are associated with facial expressions, secretions of tears and saliva, taste and hearing, are among the most easily paralyzed of human body parts. Unilateral facial paralysis is a relatively common disease with an incidence rate of 2025 per 100,000 population. However, bilateral facial nerve paralysis is much less common and secondary to other common diseases.

In Oriental medicine, facial nerve paralysis is called 风口 后, 口 眼 僻, 안 왜 사 口 口 口 口 口 口 口 口 이 이 이 이 이 이 이, The emptiness of the void and the lack of sanctity lead to the cold being caught in the meridian and being in the middle of the meridian, (陽 不 不 당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당당 2008: 128). In the present study, we compared the effects of acupuncture treatment on the facial nerve palsy in patients with acute facial nerve palsy. (4), 179-85), (Lee, Chang-Hwan, Yoon Hyun-Min, et al., Comparison of efficacy of hydration control method and SBV herbal treatment for peripheral facial nerve palsy. ≪ / RTI >2011; 28 (4): 85-92).

In order to objectify the medical treatment effect of facial nerve paralysis patients, it is necessary to develop an objective and versatile facial nerve paralysis device.

Methods for evaluating facial nerve paralysis can be divided into objective methods using diagnostic devices and subjective methods using physician 's senses.

An objective method uses a diagnostic device. Nerve conduction study (ENoG, Electroneuronography) was used to evaluate the degree of degeneration of the facial nerve directly or indirectly to evaluate the function of the facial autonomic nerve by using the infrared thermography system3). (EMG) and nerve conduction or EMG and heart rate variability were used to evaluate the muscle and nerve at the same time. There have also been studies on the use of transfer rocks or arm strength shorts mainly used in oriental medicine.

The subjective method is the Gross scale, which evaluates the entire facial nerve palsy and secondary joints at once, the Regional scale, which evaluates the degree of paralysis of each facial area and converts it to the overall grade, And a specific facial nerve grading system that classifies the secondary comorbidities separately. HB, Nottingham and Yanagihara methods of double generalization are frequently used in clinical practice.

 Seung Won Seok, Kim Pil Gun, Kwon Hyuk Koo, Hee Kyung Yoo, Seok Kyung Hwan, Lee Joo Hyun, Kim Min Jung, Park Yeon Cheol, Usefulness of EMG and early nerve conduction study (ENoG) as a prognostic factor of facial nerve palsy: Based on Hanyang 's cooperative treatment. Journal of Korean Acupuncture Society. 2013; 30 (4): 115-123.  Kim, Jong - In, Lee, Sang - Hoon, Park, Dong - Seok. Usefulness and correlation of EMG (electromyography) and HRV (heart rate variability) as clinical prognostic factors in patients with peripheral facial nerve palsy. Journal of Korean Acupuncture Society. 2008; 25 (2): 189-197.  Yoon Seung - yun, Young - Jin Kwon, Young - Jin Kim, Seung - Youn Yoon, Jung Yoon Jung, Kwak Hyun Young, A study on the transfer index as a diagnostic tool for patients with idiopathic facial nerve palsy. Journal of Korean Acupuncture Society. 2010; 27 (6): 95-100.  Joe Hyun, Lee Jin Bok, Lim Jung Kyun, Park Ge Young, Yoo Tae Han, Song Bum Yong. Correlation Study between GuanWaSa and 7 - Zone Diagnosis System: Focused on the Type of Factor AA. Journal of Korean Acupuncture Society. 2010; 27 (2): 1-10.  Kim, Jong - In. A Study on the Evaluation Method of. Journal of Korean Acupuncture Society. 2001; 18 (2): 1-17.  House JW, Brackmann DE. Facial nerve grading system. Otolaryngol Head Neck Surg. 1985; 93 (2): 146-147.  Murty GE, Diver JP, Kelly PJ, O'Donoghue GM, Bradley PJ. The Nottingham System: an objective assessment of facial nerve function in the clinic. Otolaryngol Head Neck Surg. 1994; 110 (2): 156-161.  Satoh Y, Kanzaki J, Yoshihara S. A comparison and conversion table of 'House-Brackmann facial nerve grading system' and 'Yanagihara grading system'. Auris Nasus Larynx. 2000; 27 (3): 207-212.

However, as an evaluation method according to the prior art, an objective method using a diagnostic device is accurate and objective, but remains at the level of a research stage reflecting a personal taste, and is not universally used as a subjective method. The subjective method has the advantage that the accessibility is easy and the evaluation index is widely used around the world, but there is a possibility that the subject of the doctor is likely to intervene.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for objectively evaluating whether or not facial nerve paralysis is possible.

It is another object of the present invention to provide an apparatus and method for observing the moving distance of a marker in real time according to a change in facial expression.

According to another aspect of the present invention, there is provided an apparatus for evaluating facial nerve paralysis comprising: an image obtaining unit including a camera for continuously photographing a face image of a subject; An image processor for converting the RGB image obtained from the camera into a YCbCr image and identifying a marker attached to a face of the subject; A facial nerve paralysis evaluating unit for evaluating whether the facial paralysis is occurring by calculating the degree of motion of the marker; And a display unit for displaying a result of calculating the degree of motion of the image and the marker of the subject received through the camera.

The camera is mounted on the helmet to maintain a constant distance from the face.

And the camera is mounted so as to maintain an angle of 90 degrees with the face of the subject when the subject wore the helmet.

The helmet is further provided with weight on the opposite side to which the camera is mounted.

And the image processing unit binarizes the region where the marker is attached and the remaining region except the marker.

One or more of the markers are attached, and each marker is given a unique ID.

The marker is characterized in that it is attached to two or more of the orbital point, the inferior and inferior point, the eye tail point and the tail point.

And the marker is attached symmetrically to the left and right sides of the face.

Wherein the facial nerve paralysis evaluating unit evaluates whether or not the facial paralysis is caused by measuring at least one of a distance between an orbital point and an inferior oblique point, and a distance between a trotterial point and a tricuspid point.

The facial nerve paralysis evaluating unit evaluates whether facial paralysis is possible by comparing the distance measured in the first state with the distance measured in the second state.

The first state is a state in which the eye is comfortably opened, and the second state is any one of a state in which the eye is raised upward, a state in which the eye is tightly closed, and a state in which the eye is in a smiling state.

The facial nerve paralysis evaluating unit calculates the difference between the measured distances in the first state and the second state, adds the respective differences, and then calculates the left and right values of the face as a percentage.

Wherein the facial nerve paralysis evaluating unit is evaluated using a Nottingham scale.

According to another aspect of the present invention, there is provided a method for evaluating facial nerve paralysis, comprising the steps of: (a) capturing an image of a subject having a plurality of markers attached thereto to obtain an RGB image; (b) converting the RGB image into a YCbCr color image; (c) identifying the marker by binarizing the color image into a region to which the marker is attached and a region excluding the marker; (d) measuring a distance between the markers in a first state; (e) measuring a distance between the markers in a second state; And (f) evaluating whether or not facial nerve paralysis is occurring by calculating a difference between the distance measured in the first state and the distance measured in the second state.

The method may further include: (c1) assigning a unique number to the identified marker.

Wherein the marker is attached to at least two of the orbital point, the anterior and posterior point, the taurine point, and the tail bit point.

And the marker is attached symmetrically to the left and right sides of the face.

The first state is characterized in that the eye is comfortably opened.

The second state is characterized in that it is at least one of a state in which the eyes are raised, a state in which the eyes are tightly closed, and a state in which the eyes are smiled.

The step (f) may be performed using a Nottingham scale.

As described above, according to the facial nerve paralysis evaluating apparatus and method according to the present invention, it is possible to objectively and accurately evaluate whether or not the facial nerve paralysis of the test subject.

According to the apparatus and method for evaluating facial nerve paralysis according to the present invention, it is possible to observe the movement of a marker in real time according to a change in facial expression of a subject.

1 is a block diagram of a facial nerve paralysis device according to a preferred embodiment of the present invention;
FIG. 2 shows an embodiment of the image acquisition unit shown in FIG. 1,
FIG. 3 is a view illustrating an embodiment of the image conversion unit shown in FIG. 1,
FIG. 4 shows an embodiment of the facial nerve palsy evaluation unit shown in FIG. 1,
FIG. 5 illustrates an embodiment of the display portion shown in FIG. 1,
FIG. 6 is a graph showing the results of an experiment on a normal person according to a preferred embodiment of the present invention,
FIG. 7 is a graph showing the results of experiments conducted on facial nerve palsy patients according to a preferred embodiment of the present invention,
FIG. 8 is an enlarged view of the graph portion of FIG. 6 and FIG. 7,
9 is a flow chart of a method for assessing facial nerve paralysis according to a preferred embodiment of the present invention.

Hereinafter, a facial nerve paralysis evaluating apparatus and an evaluating method according to preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram of an apparatus for evaluating facial nerve paralysis according to a preferred embodiment of the present invention, and FIGS. 2 to 6 show an embodiment of the respective components.

A facial nerve paralysis evaluating apparatus according to a preferred embodiment of the present invention includes an image acquiring unit 10, an image processing unit 20, a facial nerve paralysis evaluating unit 30, and a display unit 40.

The image acquiring unit 10 includes a camera and photographs the face of the subject in real time. In this embodiment, the Logth HD Webcam c525 was used as a camera. The camera can fix the camera 14 using a support 13 on equipment such as the one shown in Fig. 2 and the helmet 12. When wearing a helmet, it is desirable to keep the distance between the camera and the face constant and attach the camera so that the angle between the two is maintained at 90 degrees. When the camera is mounted, the center of gravity of the camera will be moved forward. Therefore, it is desirable to attach a weight 16 to the rear portion of the helmet to prevent the camera from leaning forward.

The image processing unit 20 converts the RGB image input through the camera 12 into a YCbCr color image (S22), and binarizes the converted color image to separate the marker region and the remaining regions except the marker S24), and assigning a unique number (ID) to the marker (S26). Before the subject is photographed by the camera, a marker of a specific color on the face of the subject is attached in advance. In the color image, the marker area is white based on Cr 170 or more, and the remaining part except for the marker is binarized to black. In this way, the marker part can be recognized as one object through the labeling process. When a user designates a marker on the image of the subject displayed on the display unit 40 with a mouse, the user can be given a unique number (ID) to the designated marker. The marker unique number may be automatically assigned through the labeling process. However, in order to prevent errors, the user manually inputs the unique number.

The facial nerve paralysis evaluating unit 30 evaluates whether the facial nerve paralysis is occurring by calculating the movement of the marker. In this embodiment, the motion of the marker is calculated by a nottingham scale method. In the Nottingham scale method, four measurement points on each side are selected and the distance between each point is calculated to evaluate the facial nerve paralysis.

Rating Explanation Characteristic I normal Balanced in all areas, normal motion II Light function error ㆍ Overall, I can observe weak association movement
ㆍ When left alone, symmetrical left and right as a whole
ㆍ Motion - The forehead moves well
ㆍ Eye - May be completely closed by weak force
ㆍ Mouth - weak asymmetry III Above normal function ㆍ Overall, it is clear enough to distinguish the different points of both faces when you look at the face, but there is no severe paralysis, build up, or cramps.
ㆍ When left alone, symmetrical left and right as a whole
ㆍ Motion - Forehead: Normal movement
ㆍ Eyes - can close completely
ㆍ Mouth - weak asymmetry IV Severe malfunction ㆍ Asymmetric discrimination as a whole
· When you are quiet, it is well balanced overall
ㆍ Motion - Forehead: Not moving
ㆍ Eye - not completely wound
ㆍ mouth - asymmetry even with great force V Serious malfunction ㆍ Little movement
ㆍ Asymmetry when staying still
ㆍ Motion - Forehead: Not moving
ㆍ Eye - not completely wound
ㆍ mouth - very slight movement VI Paralysis No movement

The measuring method will be described in detail as follows.

First, select four measurement points for left and right. The four points are the supraorbital point SO, the infraorbital point IO, the lateral canthus LC, and the angle of mouth M, ) Is preferably selected. Fig. 4 shows the measurement position of the face portion, Fig. 4 (a) shows the measurement position, and Fig. 4 (b) shows the marker recognition result.

First of all, it is necessary to take a picture while changing the expression in the state where the eye is opened comfortably (S1), the state where the eye is raised upward (S2), the state where the eye is tightly closed (S3) and the state where it is smiling (S4) And the presence of facial nerve paralysis.

The concrete calculation method is as shown in [Table 2] and the following equation.

: 좌측 안와상 지점(SO)과 안와하 지점(IO)사이의 거리

: The distance between the left orbital point (SO) and the anterior and posterior point (IO)

: 좌측 눈꼬리 지점(LC)과 입꼬리 지점(M) 사이의 거리

: Distance between the left eye-tail point LC and the mouth point M

: 우측 안와상 지점(SO)과 안와하 지점(IO)사이의 거리

: Distance between the right ophthalmologic point (SO) and the anterior and posterior point (IO)

: 우측 눈꼬리 지점(LC)과 입꼬리 지점(M) 사이의 거리

: Distance between the right eye tail LC and the tail point M

Raising eyebrows or closing eyes measures the distance (SO-IO) between the upper and lower parts of the orbit, and the smile motion measures the distance (LC-M) between the tail of the eye and the tail. Record the differences M1, M2, and M3 between the measured values at each step and the comfortably wiggled state (S1). Finally, the values obtained at each step are summed with respect to the left and right sides, and the percentage is calculated by using a small value as a denominator as a large value. If the calculated value is less than the predetermined reference value, it is evaluated as a facial paralysis or compared with the above-mentioned Table 1 to evaluate the degree of facial nerve paralysis.

The display unit 40 displays a state of a subject on a monitor in real time. At the same time, an input window is displayed in which the user can perform various operations. 5 shows an embodiment of the display unit 40 in which a moving picture display unit 42, a first input window 44, a second input window 46, and an evaluation result window 48 can be displayed have.

In the moving image display unit 42, images photographed through the camera are displayed in real time. In the first input window 44, a marker tracking command button is provided. In the second input window 46, And the evaluation result window 48 displays a Nottingham scale calculation result in real time as a graph.

When the "Marker tracking" button is clicked, the markers attached to the subject's face are recognized, and a portion having a marker is displayed in a square box. At this time, when the marker is displayed, the user gives a unique number (ID) to the marker. Then click the "Start" button to measure the distance between the markers. The face is photographed in real time and the distance between markers is measured in real time according to the change of expression.

When the buttons of the second input window 46 are clicked after the "start of storage" button is clicked, the respective data are inputted and displayed in the corresponding boxes. For example, when the user clicks the " when the eyes are opened loudly "button at the request of the" make big eyes open ", the value calculated by the facial nerve paralysis evaluating unit 40 And is displayed. The same procedure can be used to measure "when you close your eyes" or "when you smile." Once all of the desired measurements are obtained, click the "Save" button to end the procedure. When you click the "Graph" button after the end of shooting, you can see the real time change of the distance between measurement points in graph (48).

FIG. 6 is a graph showing experimental data of an experiment performed on a general person by the above-described method, and FIG. 7 is a table of data of a patient experiencing facial nerve paralysis. The value of Nottingham score in facial nerve palsy patients is significantly lower.

Fig. 8 is an enlarged view of the graphs of Figs. 6 and 7. Fig. 8 (a) shows the hue used in the graph, Fig. 8 (b) This is a graph of the experimental results of patients with facial nerve paralysis.

8B, since the distance of 'eyebrow-bottom (SO-IO)' is smaller than the distance of 'eye-tip-LC-M', the left and right Eye-mouth (SO-IO) 'of the left eye and the eye-mouth (SO-IO) of the right eye are displayed on the upper part of the graph. The three vertical lines on the graph represent the point at which the button was pressed to obtain a value each time the expressions S2, S3, and S4 were constructed.

In the above figure, we can see that there is a slight difference in the markings of the eyebrows (SO-IO) at the bottom of the graph, which may be an error in the attachment of the markers or a difference in the facial structure, It can be seen that there is almost no difference from the state of the stable state (S1) when the motion is observed.

In the graph of the facial nerve paralysis patient shown in FIG. 8 (c), when the eye is swollen upward (S2) and the eye is tightly closed (when S3 is seen, when the eye is wider than the stable state (S1) (SO-IO) blue line is hardly moving and the right (normal side) eyebrow-under-eye (SO-IO) blue line is moving Can be confirmed.

In the smile state S4, when the smile is compared with the stable state S1, the left (paralyzed) eye-tip LC-M green-yellow line hardly moves and the right (normal) The tip of the eye-tip (LC-M) pink line can confirm movement.

FIG. 9 illustrates a method for evaluating facial nerve paralysis according to a preferred embodiment of the present invention.

First, a face of a subject with a marker is photographed using a camera (S900), and then the RGB image photographed by the camera is converted into a YCbCr color image (S910). In the color image, the marker is recognized by distinguishing the marker region and the remaining regions except the marker (S920). In the color image, markers can be recognized by binarizing the marker to white and the rest to black based on Cr170 or higher. When the marker is recognized, the user gives a unique number (ID) to the marker (S930).

If a unique number is assigned to the marker, the distance between the markers in the steady state is measured (S940). Normal state means that the eyes are opened comfortably (S1). The distance between the markers is the distance (SO-IO) between the upper and lower points of the orbit and the distance (LC-M) between the tail and tail. Next, the distance between the markers is measured by changing the expression (S950). As you can see in the previous section, the expression can be measured by changing the expression in the order of S2, S3, and S4, S2, S3, S4 may be measured.

When the distance measurement is completed, the facial nerve paralysis is evaluated by the Nottingham scale method (S960).

The foregoing description is merely illustrative of the technical idea of the present invention and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

10: image acquiring unit 20: image processing unit
30: Facial nerve paralysis part 40: Display part
42: moving image display unit 44: first input window
46: second input window 48: evaluation result window

Claims (21)

An image acquiring unit including a camera for successively photographing a face image of a subject;
An image processor for converting the RGB image obtained from the camera into a YCbCr image and identifying a marker attached to a face of the subject;
A facial nerve paralysis evaluating unit for evaluating whether the facial paralysis is occurring by calculating the degree of motion of the marker; And
And a display unit for displaying a result of calculating the degree of motion of the image and the marker of the subject received through the camera,
The marker is attached to two or more of the orbital point, the anterior and posterior point, the eye tail point and the tail point,
The markers are attached symmetrically to the left and right sides of the face,
Wherein the facial nerve paralysis evaluating unit compares the distance measured in the first state with the distance measured in the second state to evaluate the facial paralysis.
The method according to claim 1,
Wherein the camera is mounted on the helmet to maintain a constant distance from the face. 3. The method of claim 2,
Wherein the camera is mounted so as to maintain an angle of 90 degrees with the face of the subject when the subject wore the helmet. 3. The method of claim 2,
Wherein the helmet is weight-mounted on the opposite side to which the camera is mounted. The apparatus of claim 1, wherein the image processing unit
And binarizes the region where the marker is attached and the remaining region excluding the marker. The method according to claim 1,
Wherein at least one of the markers is attached, and a unique ID is assigned to each of the markers. delete delete The method according to claim 1, wherein the facial nerve paralysis evaluating unit
A distance between an orbital point and an inferior limb, and a distance between a tear point and a tear point to measure whether or not the facial paralysis is occurring. delete The method according to claim 1,
Wherein the first state is a state in which the eye is comfortably opened, and the second state is any one of a state in which the eye is raised upward, a state in which the eye is tightly closed, and a state in which the eye is in a smiling state. 12. The method according to claim 11, wherein the facial nerve paralysis evaluating unit
Calculating a distance measured in the first state and the second state, summing the differences, and calculating the values of the left and right sides of the face as a percentage. 2. The facial nerve parallax evaluating apparatus according to claim 1, wherein the facial nerve paralysis evaluating unit is evaluated using a Nottingham scale. (a) capturing an image of a subject to which a plurality of markers are attached to obtain an RGB image;
(b) converting the RGB image into a YCbCr color image;
(c) identifying the marker by binarizing the color image into a region to which the marker is attached and a region excluding the marker;
(d) measuring a distance between the markers in a first state;
(e) measuring a distance between the markers in a second state; And
(f) evaluating whether the facial nerve paralysis is occurring by calculating a difference between a distance measured in the first state and a distance measured in the second state. 15. The method of claim 14,
(c1) assigning a unique number to the identified marker. 15. The method of claim 14,
Wherein the marker is attached to at least two of the orbital point, the anterior and posterior point, the eye tail point, and the tip point. 17. The method of claim 16, wherein the markers are attached symmetrically to the left and right sides of the face. 15. The method of claim 14,
Wherein the first state is a state in which the eye is comfortably opened. 15. The method of claim 14,
Wherein the second state is at least one of a state in which the eye is raised upward, a state in which the eye is tightly closed, and a state in which the eye is smile. The method as claimed in claim 19, wherein in step (f), the difference between the measured distances in the first and second states is calculated, and the difference between the left and right sides of the face is calculated as a percentage Of the facial paralysis. 15. The method of claim 14, wherein step (f)
Wherein the evaluation is performed using Nottingham scale.

Images