KR20200008820A - Method for user identification and pathological symptom prediction - Google Patents

Abstract

Tremors occur pathologically or physiologically in the hands, head, and other parts of the body. Pathological tremor is a condition in which a part or all of the body trembles with a regular rhythm. The pathological tremor is the most common symptom of movement disorders, and there are many types of pathological tremor depending on causes. The pathophysiology of tremor, a medical term for trembling, has not yet been precisely identified, but there are main hypotheses that there is a source of tremor in the central nervous system and that tremor occurs due to abnormalities in the cerebellar function. In addition, depending on the number of tremors and the timing of tremors, tremor may be manifested as a precursor to, or complication of various diseases. A representative disease involving tremor is Parkinson′s disease. The tremor of Parkinson′s disease is a resting tremor which occurs when a patient is at rest and is reduced during voluntary exercise. The resting tremor refers to tremor that is present severely when the hands are placed on the knees or chairs, and that is reduced when a patient holds or grips an object. In addition to tremor caused by pathological symptoms, most people have a certain pattern of tremors. The certain pattern of tremors can be used as a feature to classify users. The present invention relates to a method of predicting pathological symptoms for each user as well as identifying a user using a pattern of tremors.

Description

METHOD FOR USER IDENTIFICATION AND PATHOLOGICAL SYMPTOM PREDICTION}

The present invention relates to a method for recognizing a user and predicting morbidity using head micro shaking.

In the existing image-based user recognition technology, classification is performed by focusing on the facial structural features of the user. However, such facial recognition technology has some weaknesses in impersonation. In addition to the user's facial information, the user and distinguished person can be distinguished by using features induced in the individual physiological state.

In the present invention, the model of the head shaking is one of the physiological characteristics of the individual, and the model is trained and generated, and the classification algorithm is used to perform user recognition of the shaking along with facial information.

In addition to user recognition, it analyzes tremor patterns for individual users to predict pathological conditions, and records and manages pathological symptom prediction results for each user to use as a tool for improving status.

The present invention proposes a method for detecting the microscopic shaking of the head that can be used as an individual feature according to the physiological state in order to solve the problem of user recognition through conventional facial recognition.

In addition, by analyzing the microscopic shaking of the head, it is possible to appropriately recognize the impersonators by photographs or other methods, and input the user's head shaking pattern into the pre-configured morbidity model for the head micro shaking pattern. Morbidity prediction is performed.

Technical problems to be achieved in the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

In accordance with at least one embodiment of the present invention, a user recognition method and a pathological prediction method include: photographing a face image of a user using a camera; Detecting a face area of the user in the image; Detecting a straight line connecting any two points of the face area; Measuring a change in length of the straight line for a predetermined time in the image; The method may include obtaining a change in distance between the camera and the face of the user based on the change in the length of the straight line.

According to another embodiment, the straight line may be the width of the user's face.

According to another embodiment, the straight line may be the length of the user's face.

According to another embodiment, setting a first axis in the plane formed by the image; The method may include obtaining, as the first shaking information, the displacement of the face region in the first axial direction during the predetermined time.

According to another embodiment, setting a second axis in the plane formed by the image; The method may include obtaining, as the second shaking information, displacement of the face region in the second axis direction during the predetermined time.

According to another embodiment, the first axis and the second axis may be perpendicular to each other.

According to another embodiment, setting a third axis that is not included in the plane formed by the image; The method may include obtaining, as the third shaking information, displacement of the face region in the third axis direction.

According to another embodiment, the third axis may be perpendicular to the plane.

According to another embodiment, the method may include passing at least one of the first shaking information, the second shaking information, and the third shaking information to the high pass filter.

According to another embodiment, the cutoff frequency of the high pass filter may be 0.5 Hz.

According to another embodiment, the method may include comparing at least one of the first shaking information, the second shaking information, and the third shaking information with pre-stored shaking information of the disease.

According to an embodiment of the present invention, the microscopic shaking of the user is detected using the image, and the shaking pattern is analyzed to predict the user recognition and the pathological signs. In addition, the present invention can be applied to a system for recording / management of individual symptoms of a user.

By using the head shaking, which is a unique characteristic of the individual, the security can be enhanced when applied to user recognition. An embodiment of the present invention uses a camera image and may be combined with other biometric methods.

According to another embodiment of the present invention, a system for predicting a pathological state of a user may be constructed.

The effect obtained in the present invention is not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. .

1 to 14 show an example of a method of measuring the shaking of the user's face.
15 is a flowchart illustrating a method of recognizing a user.
16 is a flowchart illustrating a method of predicting a morbidity symptom of a user.
17 is a flowchart illustrating a method of measuring user tremor pattern and performing user recognition.

The following description may be made in various ways and have a variety of embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the technology described below.

The terms first, second, A, B, etc. may be used to describe various components, but the components are not limited by the terms, but merely for distinguishing one component from other components. Only used as For example, the first component may be referred to as the second component, and similarly, the second component may be referred to as the first component without departing from the scope of the technology described below. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is to be understood that the present invention means that there is a part or a combination thereof, and does not exclude the presence or addition possibility of one or more other features or numbers, step operation components, parts or combinations thereof.

Prior to the detailed description of the drawings, it is intended to clarify that the division of the components in the present specification is only divided by the main function of each component. That is, two or more components to be described below may be combined into one component, or one component may be provided divided into two or more for each of the more detailed functions. Each of the components to be described below may additionally perform some or all of the functions of other components in addition to the main functions of the components, and some of the main functions of each of the components are different. Of course, it may be carried out exclusively by.

In addition, in performing the method or operation method, each process constituting the method may occur differently from the stated order unless the context clearly indicates a specific order. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

Terms such as height, length, width, width, and the like used in the following description may be used for convenience of description and do not have meanings or roles that are distinct from each other.

Tremor can occur pathologically or physiologically in the hands, head, or other parts of the body. Tremor may mean that some or all of the body trembles in a regular rhythm. Tremor may be one of the dyskinesia.

Tremor can be a unique characteristic of a person. By detecting the shaking, a person can be recognized or specified.

Referring to FIG. 1, the photographing apparatus may photograph a user. The photographing apparatus may acquire an image or an image. For example, the photographing apparatus may photograph the head or face of the user. The photographing apparatus may be called a camera.

Referring to FIG. 2, an image or an image acquired by the photographing apparatus may be two-dimensional. Two coordinate axes can be set in a two-dimensional image or image. The two coordinate axes may not be parallel to each other. For example, the two coordinate axes can be the x axis and the y axis. The x and y axes may be orthogonal to each other.

The z-axis may be set to introduce a three-dimensional coordinate system to an image or an image acquired by the photographing apparatus. The z axis may not be parallel to the x or y axis. For example, the z axis can be orthogonal to the x axis or the y axis.

For the user's face, the direction parallel to the x-axis may be referred to as the width and the direction parallel to the y-axis. For any time t, any coordinates of the user's face may be represented as (Xt, Yt, Zt). The coordinates Xt, Yt, and Zt may be referred to as the center points of the face.

Referring to FIG. 3A, the center point may be represented as (Xt1, Yt1, Zt1) with respect to time t1. Referring to FIG. 3B, the center point may be represented as (Xt2, Yt2, Zt2) with respect to time t2. When time t1 elapses to time t2, the amount of change in the x- or y-axis coordinates can be calculated using the coordinates of the pixel and the like. However, since the change amount of the z-axis coordinate in the image or image is not easily measured, a method of measuring the change amount of the z-axis coordinate may be necessary.

In this case, in order to measure the amount of change in the z-axis coordinate, the width or height of the user's face shown in the image or the image may be used. At time t1, the width of the user's face may be referred to as W1 and the height of the user's face may be referred to as H1. At time t2, the width of the user's face may be W2, and the height of the user's face may be H2.

In addition to the shaking of the user, facial facial muscles may be shaken due to psychogenic factors. In order to measure the shaking, a shaking pattern may be measured by the same function as the above method by detecting the eyes, the lips, and the like in the face region.

Referring to FIG. 4, the horizontal axis of the graph may represent the width of the user's face, and the vertical axis of the graph may represent the distance between the user's face and the camera. The relationship between the distance and the width W of the face may be represented by Equation 1. The distance may be in cm. The width W of the face may be obtained from an image or an image acquired by the camera.

Referring to FIG. 5, the horizontal axis of the graph may represent a distance between a camera and a user face, and the vertical axis of the graph may represent an actual distance p of an image or an image of 1 pixel. The relationship between the distance of 1 pixel p and the distance may be as shown in Equation 2. The distance may be in cm. The distance may be a value calculated from Equation 1.

Referring to FIG. 6, the horizontal axis of the graph may be the face width W of the user measured in the image or the image, and the vertical axis of the graph may represent the face height H measured in the image or the image. The measurements shown on the graph can form a straight line. That is, even if the distance between the camera and the user's face changes, the ratio of the width W of the face to the height H of the face may be constant. Accordingly, in order to obtain the distance between the user's face and the camera, the height H of the user's face may be used instead of the width W of the user's face.

Referring to FIG. 7, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent the x-axis displacement. The unit of time may be second. The reference point of the x-axis displacement may be a center point (Xt, Yt, Zt). The unit of displacement may be cm. The graph may be referred to as x-axis shake information or shake information.

Referring to FIG. 8, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent the y-axis displacement. The unit of time may be second. The reference point of the y-axis displacement may be a center point (Xt, Yt, Zt). The unit of displacement may be cm. The graph may be referred to as y-axis shake information or shake information.

Referring to FIG. 9, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent the z-axis displacement. The unit of time may be second. The reference point of the z-axis displacement may be a center point (Xt, Yt, Zt). The unit of displacement may be cm. The graph may be referred to as z-axis shaking information or shaking information.

7 to 9 may show the shaking information of the user's face and the movement information of the user's head. The motion information of the user's head may be referred to as deflection information of the user's head. The motion information of the user's head may be a motion distinguished from the shaking of the user's head. Therefore, by removing the motion information of the user's head, only the shaking information of the user's face can be extracted. To this end, the graphs shown in FIGS. 7 to 9 may be filtered. For example, high pass filtering that passes frequencies above 0.5 Hz can be performed.

Referring to FIG. 10, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent x-axis displacement. The graph of FIG. 10 may be a graph from which the user's head movement information is removed from the graph of FIG. 7 and the shaking information of the user's face is extracted. The unit of time may be second. The reference point of the x-axis displacement may be a center point (Xt, Yt, Zt). The unit of displacement may be cm. The graph may be referred to as x-axis shake information or shake information.

Referring to FIG. 11, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent the y-axis displacement. The graph of FIG. 11 may be a graph from which the head movement information of the user is removed from the graph of FIG. 8 and the shaking information of the user's face is extracted. The unit of time may be second. The reference point of the y-axis displacement may be a center point (Xt, Yt, Zt). The unit of displacement may be cm. The graph may be referred to as y-axis shake information or shake information.

Referring to FIG. 12, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent the z-axis displacement. The graph of FIG. 12 may be a graph from which the user's head movement information is removed from the graph of FIG. 9 and the shaking information of the user's face is extracted. The unit of time may be second. The reference point of the z-axis displacement may be a center point (Xt, Yt, Zt). The unit of displacement may be cm. The graph may be referred to as z-axis shaking information or shaking information.

Referring to FIG. 13, the horizontal axis of the graph may be time, and the vertical axis of the graph may represent displacement. Tremor intensity may refer to the amount of displacement due to tremor. The number of tremors can be calculated based on the number of crossings of the cross reference line and the graph. The cross reference line may be a line parallel to the x axis. For example, when the cross reference line is 7, the number of crossings may be 14 times. For example, when the cross reference line is -1, the crossing frequency may be 22 times. For example, when the cross reference line is -2, the number of crossings may be 22 times. For example, when the cross reference line is -7, the crossing frequency may be 10 times. The relationship between the number of shaking and the number of crossings may be as shown in Equation 3. Tremor intensity, tremor frequency, etc. can form a tremor pattern.

Referring to FIG. 14, -9, -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7 of the graph A value represented by 8, 9 may mean a displacement. 0, 200, 400, 600, 800, 1000 or 1200 of the graph may refer to the number of shaking. The graph may be representative of the number of vibrations on any one of the x-axis, the y-axis, or the z-axis. The user may have a unique shaking pattern, and the graph may appear in a unique form for each user. Therefore, by analyzing the shaking pattern of the user, it can be utilized as an example of biometrics such as user authentication.

Tremor or tremor symptoms may have a specific pattern. By analyzing the shaking pattern of the user, it is possible to determine whether the user has a certain shaking or tremor symptom.

Table 1 is an example showing the characteristics of tremor or tremor symptoms.

Tremor symptoms are found in all people and pathological progress occurs under various conditions. The amplitude and frequency of tremor can be used as a key feature to characterize tremor. As shown in Table 1, various pathological symptoms such as Parkinson's disease, essential, muscle tone, neuropathy, and post-traumatic psychogenicity can be predicted according to the shaking conditions and the number of tremors per second.

According to an embodiment of the present invention, by acquiring the z-axis coordinates from the image of the user, the accuracy of the recognition of the user's vibration pattern may be increased. Recognition of the tremor pattern of the user may be implemented by machine learning.

By using the machine learning method, we can create a confusion matrix by experimenting with the vibration pattern recognition rate of 12 people. Table 2 shows the results of measuring the recognition rate with the shaking patterns of the x- and y-axes.

Table 3 shows the result of measuring the recognition rate with the shaking pattern of the x-axis, y-axis and z-axis.

Experimental results show that the recognition rate is 58% when using the x-axis and the y-axis, and 67% when the x-axis, the y-axis, and the z-axis are used.

Or, a tremor pattern recognition confusion matrix of 12 people can be created using a support vector machine (SVM). Table 4 shows the results of measuring the recognition rate with the shaking patterns of the x- and y-axes.

Table 5 shows the result of measuring the recognition rate with the shaking pattern of the x-axis, y-axis and z-axis.

As a result of the experiment, 80% recognition rate can be obtained using the x-axis and y-axis, and 100% recognition rate can be obtained using the x-axis, y-axis, and z-axis.

That is, the recognition rate can be increased when using the z-axis, which is the front and back motion information, rather than using only the x-axis and the y-axis information, which are up, down, left, and right.

Referring to FIG. 15, a flowchart may illustrate a method of measuring user tremor pattern to perform user recognition. In operation S1510, the method may include photographing a head or a face of the user with a camera. In operation S1530, the method may include detecting a face of the user from the captured image. In operation S1550, the method may include obtaining 3D coordinates of the movement of the user face. In operation S1570, the method may include obtaining a shaking pattern by measuring the small shaking of the user's face. In operation S1590, the method may include recognizing a user by contrasting the obtained vibration pattern with previously stored information.

Referring to FIG. 16, a flowchart may illustrate a method of predicting or determining a symptom of morbidity by measuring a user's tremor pattern. In operation S1610, the method may include photographing a head or a face of the user with a camera. The method may include detecting a face of the user from the captured image (S1620). In operation S1630, the method may include obtaining 3D coordinates of the movement of the user face. In operation S1640, the method may include obtaining a shaking pattern by measuring the small shaking of the user's face. It may include the step of contrasting the obtained vibration pattern and the previously stored morbidity information (S1650). It may include the step of predicting the morbid symptoms of the user (S1660).

17 shows a flow chart of the present invention. Referring to FIG. 17, a flowchart may illustrate a method of measuring user tremor pattern to perform user recognition. In operation S1720, the method may include photographing a head or a face of the user with a camera. In operation S1730, the method may include detecting a face of the user from the captured image. In operation S1740, the method may include obtaining 3D coordinates of the movement of the user face. In operation S1750, the method may include obtaining a shaking pattern by measuring a small shaking of the user's face. It may include the step of recognizing the user (S1760) or predicting the pathological symptoms of the user (S1770) in contrast to the obtained vibration pattern and the previously stored information.

The embodiments and the drawings attached to this specification are merely to clearly show a part of the technical idea included in the above-described technology, and those skilled in the art can easily make it within the scope of the technical idea included in the specification and drawings of the above-described technology. Modifications and specific embodiments that can be inferred will be obviously included in the scope of the above-described technology.

Claims (11)

Photographing a face image of a user with a camera;
Detecting a face area of the user in the image;
Detecting a straight line connecting two arbitrary points within the face area;
Measuring a change in length of the straight line for a predetermined time in the image; And,
And acquiring a change in distance between the camera and the face of the user based on the change in the length of the straight line.
According to claim 1,
And wherein the straight line is the width of the user's face.
According to claim 1,
The straight line is the length of the user's face, the user recognition and pathological signs prediction method.
According to claim 1,
Setting a first axis on a plane formed by the image;
And acquiring a displacement of the face region as first shaking information during the predetermined time, in the first axial direction.
The method of claim 4, wherein
Setting a second axis on a plane formed by the image;
And acquiring a displacement of the face area as second shaking information during the predetermined time, in the second axis direction.
The method of claim 5,
And the first axis and the second axis are orthogonal to each other.
The method of claim 5,
Setting a third axis not included in a plane formed by the image;
And acquiring a displacement of the face region as third shaking information during the predetermined time, in the third axis direction.
The method of claim 7, wherein
And the third axis is orthogonal to the plane.
The method of claim 8,
And passing at least one of the first shaking information, the second shaking information, and the third shaking information to a high pass filter.
The method of claim 9,
And a cutoff frequency of the high pass filter is 0.5 Hz.
The method of claim 9,
And comparing at least one of the first shaking information, the second shaking information, and the third shaking information with pre-stored shaking information of a disease.

Images