CN112364694B - Human body sitting posture identification method based on key point detection - Google Patents

Abstract

The invention discloses a human body _sitting posture recognition method based on key point detection. A hardware _environment is built through a PC, an _infrared ranging sensor and a camera _. The point coordinates are determined. When detecting the sitting posture of the human body, the key point coordinates under the correct sitting posture of the human body are first obtained as a reference, and then the human sitting posture is judged in real time in combination with the real-time key point coordinates of the human body and four judgment thresholds, and when three consecutive judgments are the same If there is an incorrect sitting posture, the voice will broadcast to remind the user. When two or more incorrect sitting postures appear at the same time for 3 times in a row, the sitting posture with the highest priority will be broadcast during the voice broadcast. The advantage is that the implementation process is simple and the computing power of the hardware is relatively high. Low, and can be transplanted to embedded devices later, with high practicability, low cost, high real-time performance, and good interactivity.

Description

A method for human sitting posture recognition based on key point detection

Technical Field

The invention relates to a method for recognizing a sitting posture of a human body, and in particular to a method for recognizing a sitting posture of a human body based on key point detection.

Background Art

In work and life, people sit most of the time, and if they are not careful, they will adopt an incorrect sitting posture. Long-term incorrect sitting posture may cause scoliosis, cervical spondylosis, myopia and a series of complications. Good sitting posture has an important impact on improving people's life and work efficiency and maintaining physical and mental health. Correctly identifying people's sitting posture can help people develop good sitting habits. For this reason, human sitting posture recognition technology has been widely studied.

Most of the existing human sitting posture recognition technologies are based on machine learning. For example, a Chinese patent with application publication number CN111414780A discloses a human sitting posture recognition method. The method collects user sitting posture images in real time, identifies key points of human body features, and calculates current sitting posture data based on key point data of human body features. The key point data includes eye coordinates, mouth coordinates, neck coordinates, and shoulder coordinates. The current sitting posture data includes the current head tilt angle, the current shoulder tilt angle, the height difference between the current neck and face, and the height difference between the current shoulder and face. Finally, the current sitting posture data is compared with the standard sitting posture data to determine whether the current sitting posture is abnormal. The standard sitting posture data includes a standard head tilt angle threshold, a standard shoulder tilt angle threshold, a standard eye near difference ratio threshold, and a standard lying on the table difference ratio threshold. These four thresholds are obtained by big data training through supervised learning classification algorithms of machine learning. The supervised learning classification algorithms of machine learning have high requirements on the computing power of hardware. A large amount of data is required during training to ensure the accuracy of the algorithm, and it takes a certain amount of time to calculate the corresponding results. Therefore, when implementing the above-mentioned human sitting posture recognition method, it has high requirements on the computing power of the hardware and high cost. In order to ensure its accuracy, it takes a lot of time to produce a large amount of training data. The implementation process is complicated, and it takes a long time to calculate the results during recognition, and the real-time performance is not high.

Summary of the invention

The technical problem to be solved by the present invention is to provide a human sitting posture recognition method based on key point detection, which has a simple implementation process, low requirements on hardware computing power, high practicality, low cost, high real-time performance, and good interactivity.

The technical solution adopted by the present invention to solve the above technical problems is: a method for human sitting posture recognition based on key point detection, comprising the following steps:

(1) A PC with a pre-stored image processing program, an infrared distance sensor and a camera are equipped. The infrared distance sensor and the camera are assembled and connected to the PC. The infrared distance sensor and the camera are on the same vertical plane and the distance between them is no more than 5 cm. The upper left corner of the image captured by the camera in real time is the origin of the coordinate system, the horizontal right direction is the positive direction of the x-axis, and the vertical downward direction is the positive direction of the y-axis. A coordinate system is established. Four judgment thresholds T ₁ , T ₂ , T ₃ and T ₄ are also pre-stored in the PC. These four judgment thresholds are pre-determined by the following method:

Step 1-1, divide the sitting posture into 9 categories: too close distance, too far distance, head tilted to the left, head tilted to the right, body leaning to the left, body leaning to the right, shoulders not parallel, spine curvature and correct sitting posture;

Step 1-2, select 120 females with a height between 120 cm and 180 cm and 120 males with a height between 130 cm and 190 cm as pre-inspection personnel, wherein 120 cm to 180 cm is divided into 6 levels every 10 cm, each level has 20 females, and 130 cm to 190 cm is divided into 6 levels every 10 cm, each level has 20 males; 240 pre-inspection personnel are randomly numbered from 1 to 240, and the pre-inspection personnel numbered i is called the i-th pre-inspection personnel, i = 1, 2, ..., 240;

Step 1-3: Conduct pre-tests on 240 pre-test personnel. The specific process is as follows:

S1. The camera is facing the face of the pre-inspection personnel, with a distance of 30 to 50 cm. The face and shoulders of the pre-inspection personnel cannot be blocked;

S2. Each pre-inspection person takes seven sitting postures in front of the camera, namely, correct sitting posture, head tilted to the left, head tilted to the right, body tilted to the left, body tilted to the right, spine bent, and shoulders not parallel. The camera takes images of the seven sitting postures of the pre-inspection person and sends them to the PC. The seven sitting postures are numbered 1-7 in sequence. The sitting posture numbered j is called the jth sitting posture, j=1, 2, ..., 7. The correct sitting posture is to keep the back naturally straight, the chest open, the shoulders flat, and the neck, chest and waist all kept straight. The other six sitting postures except the correct sitting posture are implemented according to personal usual habits.

右眼瞳孔的坐标记为

鼻尖的坐标记为

颈部的坐标记为

左肩的坐标记为

右肩的坐标记为

S3. Use an image processing program on a PC to obtain and record the coordinates of the six key points of the left pupil, right pupil, nose tip, neck (the concave point at the connection of the two clavicles), left shoulder and right shoulder of each pre-inspection person in the seven sitting postures, and obtain 240 sets of coordinate data. Each set of coordinate data includes the left pupil coordinates, right pupil coordinates, nose tip coordinates, neck coordinates, left shoulder coordinates and right shoulder coordinates of a pre-inspection person in the seven sitting postures. The coordinates of the left pupil of the i-th pre-inspection person in the j-th sitting posture are marked as

The coordinates of the right pupil are

The coordinates of the nose tip are

The coordinates of the neck are

The coordinates of the left shoulder are

The coordinates of the right shoulder are

S4. The left deviation of the left eye on the x-axis when the body of the i-th pre-inspection person leans to the left is taken as the left deviation, recorded as ΔL _i ; the right deviation of the right eye on the x-axis when the body leans to the right is taken as the right deviation, recorded as ΔR _i ; the displacement of the neck on the y-axis when the spine is bent is taken as the neck deviation, recorded as ΔC _i ; the difference between the two shoulder key points on the y-axis when the shoulders are not parallel is taken as the shoulder deviation, recorded as ΔH _i ; ΔL _i , ΔR _i , ΔC _i and ΔH _i are calculated using formulas (1), (2), (3) and (4) respectively:

In formula (4), || is the absolute value symbol;

S5. After integrating the 240 sets of coordinate data according to the sitting posture categories, the data are divided into 7 sets according to the 7 sitting posture categories to obtain 7 sets of sitting posture data. Each set of sitting posture data includes the left eye pupil coordinates, right eye pupil coordinates, nose tip coordinates, neck coordinates, left shoulder coordinates and right shoulder coordinates of the 240 test persons in the sitting posture.

S6. Determine the values of the determination thresholds T ₁ , T ₂ , T ₃ and T ₄ respectively, wherein the specific process of determining the determination threshold T ₁ is as follows:

A. Using 240 left-leaning deviations from ΔL ₁ to ΔL ₂₄₀ to form an original data set, and taking the original data set as the 0th generation data set;

B. Set the iteration variable t and initialize t to 1;

C. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

C1. Calculate the kurtosis of the t-1 generation data set

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则令判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的左倾偏量的值作为判定阈值T₁，如果

大于3，则计算第t-1代数据组中每个左倾偏量与

之差的平方值，将最大平方值对应的左倾偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤C，进行下一次迭代，直至

不大于3；C2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the maximum left-leaning deviation value in the t-1 generation data set is used as the decision threshold T ₁ .

If it is greater than 3, then calculate the left deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the left deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step C for the next iteration until

Not more than 3;

The specific process of determining the judgment threshold _T2 is:

D. Using 240 right-leaning deviations from ΔR ₁ to ΔR ₂₄₀ to form an original data set, and taking the original data set as the 0th generation data set;

E. Set the iteration variable t and initialize t to 1;

F. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

F1, calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则令判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的右倾偏量的值作为判定阈值T₂，如果

大于3，则计算第t-1代数据组中每个右倾偏量与

之差的平方值，将最大平方值对应的左倾偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤F，进行下一次迭代，直至

不大于3；F2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the maximum right deviation value in the t-1 generation data set is used as the decision threshold T ₂ .

If it is greater than 3, then calculate the right deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the left deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step F for the next iteration until

Not more than 3;

The specific process of determining the judgment threshold _T3 is:

G. Using 240 neck deflections from ΔC ₁ to ΔC ₂₄₀ to form an original data set, and using the original data set as the 0th generation data set;

H. Set the iteration variable t and initialize t to 1;

I. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

I1. Calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则令判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的颈部偏量的值作为判定阈值T₃，如果

大于3，则计算第t-1代数据组中每个左倾偏量与

之差的平方值，将最大平方值对应的颈部偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤I，进行下一次迭代，直至

不大于3；I2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the value of the maximum neck deviation in the t-1 generation data set is used as the judgment threshold T ₃ .

If it is greater than 3, then calculate the left deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the neck deviation corresponding to the maximum square value from the t-1 generation data set to obtain the t generation data set. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step I for the next iteration until

Not more than 3;

The specific process of determining the decision threshold _T4 is:

J. Using 240 shoulder deviations from ΔH ₁ to ΔH ₂₄₀ to form an original data set, and using the original data set as the 0th generation data set;

K. Set the iteration variable t and initialize t to 1;

L. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

L1, calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的肩膀偏量的值作为判定阈值T₄，如果

则计算第t-1代数据组中每个左倾偏量与

之差的平方值，将最大平方值对应的肩膀偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤L，进行下一次迭代，直至

不大于3；L2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, then the threshold is determined

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the value of the maximum shoulder deviation in the t-1 generation data set is used as the decision threshold T ₄ .

Then calculate each left-leaning deviation in the t-1 generation data set and

The square value of the difference between the two values is used to delete the shoulder deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the step L is returned to perform the next iteration until

Not more than 3;

(2) When the user's sitting posture needs to be recognized, the camera is installed at the corresponding position where the sitting posture recognition needs to be performed, and the user's reference data is first collected. The specific process is as follows: the user first sits in front of the camera with a correct sitting posture, with the camera facing the user's face, the distance between the two is 30 to 50 cm, and the user's face and shoulders cannot be blocked. The camera takes an image of the user's correct sitting posture and sends it to the PC. The PC uses the image processing program pre-stored therein to process the image of the user's correct sitting posture, determine and record the coordinates of the user's left pupil, right pupil, nose tip, neck (the concave point at the connection of the two clavicles), left shoulder and right shoulder in the correct sitting posture, and mark the coordinates of the left pupil of the user in the correct sitting posture as (lx, ly), the coordinates of the right pupil as (rx, ry), the coordinates of the nose tip as (nx, ny), the coordinates of the neck as (bx, by), the coordinates of the left shoulder as (lsx, lsy), and the coordinates of the right shoulder as (rsx, rsy);

(3) After determining the user's reference data, the user's sitting posture is recognized in real time. The specific process is as follows:

Step 3-1, the PC collects an image of the user's sitting posture from the camera every 2 seconds, and uses an image processing program to process the real-time image of the user's sitting posture, determine and record the coordinates of the user's left pupil, right pupil, nose tip, neck (the concave point where the two clavicles are connected), left shoulder and right shoulder in the current sitting posture, and at the same time receive the distance between the user and the camera measured by the infrared ranging sensor, and mark the coordinates of the left pupil of the user in the current sitting posture as (lx _N , ly _N ), the coordinates of the right pupil as (rx _N , ry _N ), the coordinates of the nose tip as (nx _N , ny _N ), the coordinates of the neck as (bx _N , by _N ), the coordinates of the left shoulder as (lsx _N , lsy _N ), and the coordinates of the right shoulder as (rsx _N , rsy _N ), the distance between the user and the camera is recorded as D, the left eye pupil key point is connected to the nose tip key point and the line connecting the two is recorded as line segment a, the right eye pupil key point is connected to the nose tip key point respectively and the line connecting the two is recorded as line segment b, the nose tip key point is connected to the neck key point and the line connecting the two is recorded as line segment c, the left shoulder key point is connected to the neck key point and the line connecting the two is recorded as line segment d, the right shoulder key point is connected to the neck key point and the line connecting the two is recorded as line segment e, the angle between the line segment c and the line segment d is recorded as angle α, and the angle between the line segment c and the line segment e is recorded as angle β;

Step 3-2: Based on the real-time data from step 3-1, the user's sitting posture is judged in real time. The specific judgment criteria are:

If D is less than 30 cm, it is judged as too close;

If D is greater than 50 cm, it is judged to be too far away;

If α is greater than 0° and less than or equal to 70°, the current sitting posture is judged as head tilted to the left;

If β is greater than 0° and less than or equal to 70°, the current sitting posture is judged as head right tilt;

If lx-lx _N >T ₁ , the current sitting posture is determined to be a left leaning posture;

If rx _N -rx>T ₂ , the current sitting posture is determined to be right-leaning;

If |lsy _N -rsy _N |>T ₄ , the current sitting posture is judged as shoulders not being parallel;

If by _N -by>T ₃ , the current sitting posture is determined to be spinal curvature;

If it is any other situation than the above situation, the current sitting posture is determined to be the correct sitting posture;

Step 3-3, if the same incorrect sitting posture is determined for three consecutive times, a voice broadcast will be used to remind the user. When two or more incorrect sitting postures appear for three consecutive times at the same time, the sitting posture with the highest priority will be broadcasted. The priorities of the eight incorrect sitting postures from high to low are: distance too close, distance too far, head tilted to the left, head tilted to the right, body leaning to the left, body leaning to the right, shoulders not parallel and spine curvature.

Compared with the prior art, the advantage of the present invention is that a hardware environment is built by a PC pre-stored with an image processing program, an infrared ranging sensor and a camera, a coordinate system is established with the upper left corner of the picture collected by the camera in real time as the coordinate origin, the horizontal right direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis, and four determination thresholds T ₁ , T ₂ , T ₃ and T ₄ are also pre-stored in the PC, and the four determination thresholds T ₁ , T ₂ , T ₃ and T ₄ are determined in advance by the key point coordinates. When detecting the sitting posture of a human body, the key point coordinates of the human body in the correct sitting posture are first obtained as a reference, and then the real-time key point coordinates of the human body and the four determination thresholds T ₁ , T ₂ , T ₃ and T 4 are combined to determine the sitting posture of the human body. _4. Real-time judgment of human sitting posture. In addition, an infrared distance sensor is used to obtain the distance between the user and the camera. The infrared distance sensor is low in price and has a fast response speed. When the same incorrect sitting posture is judged for three consecutive times, a voice broadcast is used to remind the user. When two or more incorrect sitting postures appear for three consecutive times at the same time, the sitting posture with the highest priority is broadcast during the voice broadcast. Compared with the existing machine learning, the four judgment threshold determination methods in the present invention do not need to produce a large amount of training data. While ensuring high accuracy, the calculation process is simplified and the time required for calculation is shortened. Therefore, the implementation process of the present invention is simple, the requirement for hardware computing power is low, and it can be transplanted to embedded devices later. It has high practicality, low cost, high real-time performance, and good interactivity.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the key points, key point connecting lines and angles between connecting lines of the human sitting posture recognition method based on key point detection of the present invention.

DETAILED DESCRIPTION

The present invention is further described in detail below with reference to the accompanying drawings.

Embodiment: A method for human sitting posture recognition based on key point detection comprises the following steps:

(1) A PC with a pre-stored image processing program, an infrared distance sensor and a camera are equipped. The infrared distance sensor and the camera are assembled and connected to the PC. The infrared distance sensor and the camera are on the same vertical plane and the distance between them is no more than 5 cm. The upper left corner of the image captured by the camera in real time is the origin of the coordinate system. The horizontal right direction is the positive direction of the x-axis and the vertical downward direction is the positive direction of the y-axis. A coordinate system is established. Four judgment thresholds T ₁ , T ₂ , T ₃ and T ₄ are also pre-stored in the PC. These four judgment thresholds are pre-determined by the following method:

Step 1-1, divide the sitting posture into 9 categories: too close distance, too far distance, head tilted to the left, head tilted to the right, body leaning to the left, body leaning to the right, shoulders not parallel, spine curvature and correct sitting posture;

Step 1-2, select 120 females with a height between 120 cm and 180 cm and 120 males with a height between 130 cm and 190 cm as pre-inspection personnel, wherein 120 cm to 180 cm is divided into 6 levels every 10 cm, each level has 20 females, and 130 cm to 190 cm is divided into 6 levels every 10 cm, each level has 20 males; randomly number the 240 pre-inspection personnel from 1 to 240, and the pre-inspection personnel numbered i is called the i-th pre-inspection personnel, i = 1, 2, ..., 240;

Step 1-3: Conduct pre-tests on 240 pre-test personnel. The specific process is as follows:

S1. The camera is facing the face of the pre-inspection personnel, with a distance of 30 to 50 cm. The face and shoulders of the pre-inspection personnel cannot be blocked;

S2. Each pre-inspection person takes seven sitting postures in front of the camera, namely, correct sitting posture, head tilted to the left, head tilted to the right, body tilted to the left, body tilted to the right, spine bent, and shoulders not parallel. The camera takes images of the seven sitting postures of the pre-inspection person and sends them to the PC. The seven sitting postures are numbered 1-7 in sequence. The sitting posture numbered j is called the jth sitting posture, j=1, 2, ..., 7. The correct sitting posture is to keep the back naturally straight, the chest open, the shoulders flat, and the neck, chest and waist all kept straight. The other six sitting postures except the correct sitting posture are implemented according to personal usual habits.

右眼瞳孔的坐标记为

鼻尖的坐标记为

颈部的坐标记为

左肩的坐标记为

右肩的坐标记为

S3. Use an image processing program on a PC to obtain and record the coordinates of the six key points of the left pupil, right pupil, nose tip, neck (the concave point at the connection of the two clavicles), left shoulder and right shoulder of each pre-inspection person in the seven sitting postures, and obtain 240 sets of coordinate data. Each set of coordinate data includes the left pupil coordinates, right pupil coordinates, nose tip coordinates, neck coordinates, left shoulder coordinates and right shoulder coordinates of a pre-inspection person in the seven sitting postures. The coordinates of the left pupil of the i-th pre-inspection person in the j-th sitting posture are marked as

The coordinates of the right pupil are

The coordinates of the nose tip are

The coordinates of the neck are

The coordinates of the left shoulder are

The coordinates of the right shoulder are

S4. The left deviation of the left eye on the x-axis when the body of the i-th pre-inspection person leans to the left is taken as the left deviation, recorded as ΔL _i ; the right deviation of the right eye on the x-axis when the body leans to the right is taken as the right deviation, recorded as ΔR _i ; the displacement of the neck on the y-axis when the spine is bent is taken as the neck deviation, recorded as ΔC _i ; the difference between the two shoulder key points on the y-axis when the shoulders are not parallel is taken as the shoulder deviation, recorded as ΔH _i ; ΔL _i , ΔR _i , ΔC _i and ΔH _i are calculated using formulas (1), (2), (3) and (4) respectively:

In formula (4), || is the absolute value symbol;

S5. After integrating the 240 sets of coordinate data according to the sitting posture categories, the data are divided into 7 sets according to the 7 sitting posture categories to obtain 7 sets of sitting posture data. Each set of sitting posture data includes the left eye pupil coordinates, right eye pupil coordinates, nose tip coordinates, neck coordinates, left shoulder coordinates and right shoulder coordinates of the 240 test persons in the sitting posture.

S6. Determine the values of the determination thresholds T ₁ , T ₂ , T ₃ and T ₄ respectively, wherein the specific process of determining the determination threshold T ₁ is as follows:

A. Using 240 left-leaning deviations from ΔL ₁ to ΔL ₂₄₀ to form an original data set, and taking the original data set as the 0th generation data set;

B. Set the iteration variable t and initialize t to 1;

C. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

C1. Calculate the kurtosis of the t-1 generation data set

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则令判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的左倾偏量的值作为判定阈值T₁，如果

大于3，则计算第t-1代数据组中每个左倾偏量与

之差的平方值，将最大平方值对应的左倾偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤C，进行下一次迭代，直至

不大于3；C2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the maximum left-leaning deviation value in the t-1 generation data set is used as the decision threshold T ₁ .

If it is greater than 3, then calculate the left deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the left deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step C for the next iteration until

Not more than 3;

The specific process of determining the judgment threshold _T2 is:

D. Using 240 right-leaning deviations from ΔR ₁ to ΔR ₂₄₀ to form an original data set, and taking the original data set as the 0th generation data set;

E. Set the iteration variable t and initialize t to 1;

F. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

F1, calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则令判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的右倾偏量的值作为判定阈值T₂，如果

大于3，则计算第t-1代数据组中每个右倾偏量与

之差的平方值，将最大平方值对应的左倾偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤F，进行下一次迭代，直至

不大于3；F2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the maximum right deviation value in the t-1 generation data set is used as the decision threshold T ₂ .

If it is greater than 3, then calculate the right deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the left deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step F for the next iteration until

Not more than 3;

The specific process of determining the judgment threshold _T3 is:

G. Using 240 neck deflections from ΔC ₁ to ΔC ₂₄₀ to form an original data set, and using the original data set as the 0th generation data set;

H. Set the iteration variable t and initialize t to 1;

I. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

I1. Calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则令判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的颈部偏量的值作为判定阈值T₃，如果

大于3，则计算第t-1代数据组中每个左倾偏量与

之差的平方值，将最大平方值对应的颈部偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤I，进行下一次迭代，直至

不大于3；I2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the value of the maximum neck deviation in the t-1 generation data set is used as the judgment threshold T ₃ .

If it is greater than 3, then calculate the left deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the neck deviation corresponding to the maximum square value from the t-1 generation data set to obtain the t generation data set. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step I for the next iteration until

Not more than 3;

The specific process of determining the decision threshold _T4 is:

J. Using 240 shoulder deviations from ΔH ₁ to ΔH ₂₄₀ to form an original data set, and using the original data set as the 0th generation data set;

K. Set the iteration variable t and initialize t to 1;

L. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is:

均值

和标准差

L1, calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

是否大于3，如果

不大于3，且

与3的差值不大于1，则判定阈值

如果

不大于3，且

与3的差值大于1，则将第t-1代数据组中最大的肩膀偏量的值作为判定阈值T₄，如果

则计算第t-1代数据组中每个左倾偏量与

之差的平方值，将最大平方值对应的肩膀偏量从第t-1代数据组中删除，得到第t代数据组，然后采用t的当前值加1的和更新t的取值，返回步骤L，进行下一次迭代，直至

不大于3；L2. Judgment

Is it greater than 3? If

Not more than 3, and

If the difference between the value of the threshold value and 3 is not greater than 1, the threshold value is determined.

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the value of the maximum shoulder deviation in the t-1 generation data set is used as the decision threshold T ₄ .

Then calculate each left-leaning deviation in the t-1 generation data set and

The square value of the difference between the two values is used to delete the shoulder deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the step L is returned to perform the next iteration until

Not more than 3;

(2) When the user's sitting posture needs to be recognized, the camera is installed at the corresponding position where the sitting posture recognition needs to be performed, and the user's reference data is first collected. The specific process is as follows: the user first sits in front of the camera with a correct sitting posture, with the camera facing the user's face, the distance between the two is 30 to 50 cm, and the user's face and shoulders cannot be blocked. The camera takes an image of the user's correct sitting posture and sends it to the PC. The PC uses the image processing program pre-stored therein to process the image of the user's correct sitting posture, determine and record the coordinates of the user's left pupil, right pupil, nose tip, neck (the concave point at the connection of the two clavicles), left shoulder and right shoulder in the correct sitting posture, and mark the coordinates of the left pupil of the user in the correct sitting posture as (lx, ly), the coordinates of the right pupil as (rx, ry), the coordinates of the nose tip as (nx, ny), the coordinates of the neck as (bx, by), the coordinates of the left shoulder as (lsx, lsy), and the coordinates of the right shoulder as (rsx, rsy);

(3) After determining the user's reference data, the user's sitting posture is recognized in real time. The specific process is as follows:

Step 3-1, the PC collects an image of the user's sitting posture from the camera every 2 seconds, and uses an image processing program to process the real-time image of the user's sitting posture, determine and record the coordinates of the user's left pupil, right pupil, nose tip, neck (the concave point at the connection of the two clavicles), left shoulder and right shoulder in the current sitting posture, and at the same time receive the distance between the user and the camera measured by the infrared ranging sensor, and mark the coordinates of the left pupil of the user in the current sitting posture as (lx _N , ly _N ), the coordinates of the right pupil as (rx _N , ry _N ), the coordinates of the nose tip as (nx _N , ny _N ), the coordinates of the neck as (bx _N , by _N ), the coordinates of the left shoulder as (lsx _N , lsy _N ), and the coordinates of the right shoulder as (rsx _N , rsy _N ), the distance between the user and the camera is recorded as D, the left eye pupil key point is connected to the nose tip key point and the line connecting the two is recorded as line segment a, the right eye pupil key point is connected to the nose tip key point respectively and the line connecting the two is recorded as line segment b, the nose tip key point is connected to the neck key point and the line connecting the two is recorded as line segment c, the left shoulder key point is connected to the neck key point and the line connecting the two is recorded as line segment d, the right shoulder key point is connected to the neck key point and the line connecting the two is recorded as line segment e, the angle between the line segment c and the line segment d is recorded as angle α, and the angle between the line segment c and the line segment e is recorded as angle β;

Step 3-2: Based on the real-time data from step 3-1, the user's sitting posture is judged in real time. The specific judgment criteria are:

If D is less than 30 cm, it is judged as too close;

If D is greater than 50 cm, it is judged to be too far away;

If α is greater than 0° and less than or equal to 70°, the current sitting posture is judged as head tilted to the left;

If β is greater than 0° and less than or equal to 70°, the current sitting posture is judged as head right tilt;

If lx-lx _N >T ₁ , the current sitting posture is determined to be a left leaning posture;

If rx _N -rx>T ₂ , the current sitting posture is determined to be right-leaning;

If |lsy _N -rsy _N |>T ₄ , the current sitting posture is judged as shoulders not being parallel;

If by _N -by>T ₃ , the current sitting posture is determined to be spinal curvature;

If it is any other situation than the above situation, the current sitting posture is determined to be the correct sitting posture;

Step 3-3, if the same incorrect sitting posture is determined for three consecutive times, a voice broadcast will be used to remind the user. When two or more incorrect sitting postures appear for three consecutive times at the same time, the sitting posture with the highest priority will be broadcasted. The priorities of the eight incorrect sitting postures from high to low are: distance too close, distance too far, head tilted to the left, head tilted to the right, body leaning to the left, body leaning to the right, shoulders not parallel and spine curvature.

Claims (1)

1. A method for human sitting posture recognition based on key point detection, characterized by comprising the following steps: (1) A PC with a pre-stored image processing program, an infrared distance sensor and a camera are equipped. The infrared distance sensor and the camera are assembled and connected to the PC. The infrared distance sensor and the camera are on the same vertical plane and the distance between them is no more than 5 cm. The upper left corner of the image captured by the camera in real time is the origin of the coordinate system. The horizontal right direction is the positive direction of the x-axis and the vertical downward direction is the positive direction of the y-axis. A coordinate system is established. Four judgment thresholds T ₁ , T ₂ , T ₃ and _Ti are also pre-stored in the PC. These four judgment thresholds are pre-determined by the following method: Step 1-1, divide the sitting posture into 9 categories: too close distance, too far distance, head tilted to the left, head tilted to the right, body leaning to the left, body leaning to the right, shoulders not parallel, spine curvature and correct sitting posture; Step 1-2, select 120 females with a height between 120 cm and 180 cm and 120 males with a height between 130 cm and 190 cm as pre-inspection personnel, wherein 120 cm to 180 cm is divided into 6 levels every 10 cm, each level has 20 females, and 130 cm to 190 cm is divided into 6 levels every 10 cm, each level has 20 males; 240 pre-inspection personnel are randomly numbered from 1 to 240, and the pre-inspection personnel numbered i is called the i-th pre-inspection personnel, i = 1, 2, ..., 240; Step 1-3: Conduct pre-tests on 240 pre-test personnel. The specific process is as follows: S1. The camera is facing the face of the pre-inspection personnel, with a distance of 30 to 50 cm. The face and shoulders of the pre-inspection personnel cannot be blocked; S2. Each pre-inspection person takes seven sitting postures in front of the camera, namely, correct sitting posture, head tilted to the left, head tilted to the right, body tilted to the left, body tilted to the right, spine bent, and shoulders not parallel. The camera takes images of the seven sitting postures of the pre-inspection person and sends them to the PC. The seven sitting postures are numbered 1-7 in sequence. The sitting posture numbered j is called the jth sitting posture, j=1, 2, ..., 7. The correct sitting posture is to keep the back naturally straight, the chest open, the shoulders flat, and the neck, chest and waist all kept straight. The other six sitting postures except the correct sitting posture are implemented according to personal usual habits. S3. Use an image processing program at the PC to obtain and record the coordinates of the six key points of the left pupil, right pupil, nose tip, neck, left shoulder and right shoulder of each pre-inspection person in the seven sitting postures, and obtain 240 sets of coordinate data. Each set of coordinate data includes the left pupil coordinates, right pupil coordinates, nose tip coordinates, neck coordinates, left shoulder coordinates and right shoulder coordinates of a pre-inspection person in the seven sitting postures. The coordinates of the left pupil of the i-th pre-inspection person in the j-th sitting posture are marked as

The coordinates of the right pupil are

The coordinates of the nose tip are

The coordinates of the neck are

The coordinates of the left shoulder are

The coordinates of the right shoulder are

S4. The left deviation of the left eye on the x-axis when the body of the i-th pre-inspection person leans to the left is taken as the left deviation, recorded as ΔL _i ; the right deviation of the right eye on the x-axis when the body leans to the right is taken as the right deviation, recorded as ΔR _i ; the displacement of the neck on the y-axis when the spine is bent is taken as the neck deviation, recorded as ΔC _i ; the difference between the two shoulder key points on the y-axis when the shoulders are not parallel is taken as the shoulder deviation, recorded as ΔH _i ; ΔL _i , ΔR _i , ΔC _i and ΔH _i are calculated using formulas (1), (2), (3) and (4) respectively:

In formula (4), || is the absolute value symbol; S5. After integrating the 240 sets of coordinate data according to the sitting posture categories, the data are divided into 7 sets according to the 7 sitting posture categories to obtain 7 sets of sitting posture data. Each set of sitting posture data includes the left eye pupil coordinates, right eye pupil coordinates, nose tip coordinates, neck coordinates, left shoulder coordinates and right shoulder coordinates of the 240 test persons in the sitting posture. S6. Determine the values of the determination thresholds T ₁ , T ₂ , T ₃ and T ₄ respectively, wherein the specific process of determining the determination threshold T ₁ is as follows: A. Using 240 left-leaning deviations from ΔL ₁ to ΔL ₂₄₀ to form an original data set, and taking the original data set as the 0th generation data set; B. Set the iteration variable t and initialize t to 1; C. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is: C1. Calculate the kurtosis of the t-1 generation data set

Mean

and standard deviation

C2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the maximum left-leaning deviation value in the t-1 generation data set is used as the decision threshold T ₁ .

If it is greater than 3, then calculate the left deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the left deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step C for the next iteration until

Not more than 3; The specific process of determining the decision threshold _T2 is: D. Using 240 right-leaning deviations from ΔR ₁ to ΔR ₂₄₀ to form an original data set, and taking the original data set as the 0th generation data set; E. Set the iteration variable t and initialize t to 1; F. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is: F1, calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

F2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the maximum right deviation value in the t-1 generation data set is used as the decision threshold T ₂ .

If it is greater than 3, then calculate the right deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the left deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step F for the next iteration until

Not more than 3; The specific process of determining the judgment threshold _T3 is: G. Using 240 neck deflections from ΔC ₁ to ΔC ₂₄₀ to form an original data set, and using the original data set as the 0th generation data set; H. Set the iteration variable t and initialize t to 1; I. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is: I1. Calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

I2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, so let the judgment threshold

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the value of the maximum neck deviation in the t-1 generation data set is used as the judgment threshold T ₃ .

If it is greater than 3, then calculate the left deviation of each data set in the t-1th generation and

The square value of the difference between the two values is used to delete the neck deviation corresponding to the maximum square value from the t-1 generation data set to obtain the t generation data set. Then, the value of t is updated by adding 1 to the current value of t, and the process returns to step I for the next iteration until

Not more than 3; The specific process of determining the decision threshold _T4 is: J. Using 240 shoulder deviations from ΔH ₁ to ΔH ₂₄₀ to form an original data set, and using the original data set as the 0th generation data set; K. Set the iteration variable t and initialize t to 1; L. Perform the t-th iteration update to obtain the t-th generation data set. The specific process is: L1, calculate the kurtosis of the t-1 generation data group

Mean

and standard deviation

L2. Judgment

Is it greater than 3? If

Not more than 3, and

The difference between 3 and 1 is not greater than 1, then the threshold is determined

if

Not more than 3, and

If the difference between t and 3 is greater than 1, the value of the maximum shoulder deviation in the t-1 generation data set is used as the decision threshold T ₄ .

Then calculate each left-leaning deviation in the t-1 generation data set and

The square value of the difference between the two values is used to delete the shoulder deviation corresponding to the maximum square value from the t-1 generation data group to obtain the t generation data group. Then, the value of t is updated by adding 1 to the current value of t, and the step L is returned to perform the next iteration until

Not more than 3; (2) When the user's sitting posture needs to be recognized, the camera is installed at the corresponding position where the sitting posture recognition needs to be performed, and the user's reference data is first collected. The specific process is as follows: the user first sits in front of the camera with a correct sitting posture, with the camera facing the user's face, the distance between the two is 30 to 50 cm, and the user's face and shoulders cannot be blocked. The camera takes an image of the user's correct sitting posture and sends it to the PC. The PC uses the image processing program pre-stored therein to process the image of the user's correct sitting posture, determine and record the coordinates of the six key points of the user's left pupil, right pupil, nose tip, neck, left shoulder and right shoulder in the correct sitting posture, and mark the coordinates of the left pupil of the user in the correct sitting posture as (lx, ly), the coordinates of the right pupil as (rx, ry), the coordinates of the nose tip as (nx, ny), the coordinates of the neck as (bx, by), the coordinates of the left shoulder as (lsx, lsy), and the coordinates of the right shoulder as (rsx, rsy); (3) After determining the user's reference data, the user's sitting posture is recognized in real time. The specific process is as follows: Step 3-1, the PC collects an image of the user's sitting posture from the camera every 2 seconds, and uses an image processing program to process the real-time image of the user's sitting posture, determine and record the coordinates of the six key points of the user's left pupil, right pupil, nose tip, neck, left shoulder and right shoulder in the current sitting posture, and at the same time receive the distance between the user and the camera measured by the infrared ranging sensor, and mark the coordinates of the left pupil of the user in the current sitting posture as (lx _N , ly _N ), the coordinates of the right pupil as (rx _N , ry _N ), the coordinates of the nose tip as (nx _N , ny _N ), the coordinates of the neck as (bx _N , by _N ), the coordinates of the left shoulder as (lsx _N , lsy _N ), and the coordinates of the right shoulder as (rsx _N , rsy _N ), the distance between the user and the camera is recorded as D, the left eye pupil key point is connected to the nose tip key point and the line connecting the two is recorded as line segment a, the right eye pupil key point is connected to the nose tip key point respectively and the line connecting the two is recorded as line segment b, the nose tip key point is connected to the neck key point and the line connecting the two is recorded as line segment c, the left shoulder key point is connected to the neck key point and the line connecting the two is recorded as line segment d, the right shoulder key point is connected to the neck key point and the line connecting the two is recorded as line segment e, the angle between the line segment c and the line segment d is recorded as angle α, and the angle between the line segment c and the line segment e is recorded as angle β; Step 3-2: Based on the real-time data from step 3-1, the user's sitting posture is judged in real time. The specific judgment criteria are: If D is less than 30 cm, it is judged as too close; If D is greater than 50 cm, it is judged to be too far away; If α is greater than 0° and less than or equal to 70°, the current sitting posture is judged as head tilted to the left; If β is greater than 0° and less than or equal to 70°, the current sitting posture is judged as head right tilt; If lx-lx _N >T ₁ , the current sitting posture is determined to be a left leaning posture; If rx _N -rx>T ₂ , the current sitting posture is determined to be right-leaning; If |lsy _N -rsy _N |>T ₄ , the current sitting posture is judged as shoulders not being parallel; If by _N -by>T ₃ , the current sitting posture is determined to be spinal curvature; If it is any other situation than the above situation, the current sitting posture is determined to be the correct sitting posture; Step 3-3, if the same incorrect sitting posture is determined for three consecutive times, a voice broadcast will be used to remind the user. When two or more incorrect sitting postures appear for three consecutive times at the same time, the sitting posture with the highest priority will be broadcasted. The priorities of the eight incorrect sitting postures from high to low are: distance too close, distance too far, head tilted to the left, head tilted to the right, body leaning to the left, body leaning to the right, shoulders not parallel and spine curvature.

Images