CN117593763A - Bad sitting posture detection method and related equipment - Google Patents

Abstract

The invention discloses a bad sitting posture detection method and related equipment, wherein the method comprises the following steps: acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area; detecting face feature points, shoulder feature points and pixel feature points in a face window area; screening a preset number of face feature points, and calculating head Euler angles according to the preset number of face feature points; judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture. According to the invention, whether the current sitting posture of the target object is a poor sitting posture is judged by detecting the face characteristic points, the shoulder characteristic points and the pixel characteristic points, so that the influence of complex environments such as external illumination and the like is not easy to happen; the bad sitting posture can be judged without additionally manufacturing a sample set, and the manpower is saved.

Description

Bad sitting posture detection method and related equipment

Technical Field

The invention relates to the technical field of automatic sitting posture detection, in particular to a poor sitting posture detection method, a poor sitting posture detection system and a poor sitting posture detection terminal.

Background

With the change of the life style of modern people, more and more people sit in front of an office or a table at home for a long time to work, resulting in a plurality of health problems such as cervical spondylosis, lumbar vertebra disease, eye fatigue and the like; therefore, it has become very important to develop a technique capable of automatically monitoring and correcting bad sitting postures.

However, at present, no complete detection method for judging the bad sitting posture of the human body by combining the characteristic point positions of the head and the shoulder is available.

The existing poor sitting posture detection based on machine vision needs to be performed in a complex background environment, and can also cause interference problems of background light and the like; and requires a long calculation time, resulting in poor real-time; in addition, the bad sitting posture detection method generally needs a large amount of labeling data for training and evaluation, and some methods may occupy a large amount of computing resources, have high requirements on hardware configuration of a detection system, and limit the application of the detection system on low-end equipment; furthermore, there is also a bias in the model that may be due to data set bias or imbalance.

Accordingly, the prior art is still in need of improvement and development.

Disclosure of Invention

The invention mainly aims to provide a bad sitting posture detection method, a system and a terminal, and aims to solve the problems of time consumption, low stability and low accuracy caused by the fact that a large amount of data are required for training and evaluation in the bad sitting posture detection method in the prior art.

In order to achieve the above object, the present invention provides a bad sitting posture detection method, comprising the steps of:

acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area;

detecting face feature points, shoulder feature points and pixel feature points in the face window area;

screening a preset number of face feature points, and calculating head Euler angles according to the preset number of face feature points;

judging whether the current sitting posture of the target object is a poor sitting posture or not according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture.

Optionally, in the method for detecting a bad sitting posture, the acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area specifically includes:

acquiring a human body image of a target object in a detection area;

scanning the human body image in a sliding window mode;

and marking a face window area in the human body image by using the trained cascade classifier.

Optionally, the method for detecting a bad sitting posture specifically includes:

Detecting face feature points of the target object in the face window area based on a depth camera and a PFLD face feature point detection algorithm;

detecting shoulder feature points of the target object in the face window area based on a lightweight OpenPose model subjected to deep learning;

and detecting pixel characteristic points of the target object in the face window area based on the depth camera.

Optionally, in the poor sitting posture detection method, the calculating according to the preset number of face feature points to obtain a head euler angle specifically includes:

acquiring face feature point coordinates of the preset number of face feature points in a world coordinate system;

calculating according to the face feature point coordinates to obtain a corresponding rotation matrix;

and obtaining a rotation vector of the rotation matrix by using a solvePnP function of OpenCV, and converting the rotation vector to obtain the head Euler angle.

Optionally, the poor sitting posture detecting method, wherein the head euler angle comprises a pitch angle, a yaw angle and a roll angle; the shoulder characteristic points comprise left shoulder characteristic points and right shoulder characteristic points; the pixel characteristic points comprise neck pixel characteristic points and nose pixel characteristic points;

Judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, wherein the method specifically comprises the following steps of:

judging whether the pitch angle falls in a first angle range or not to obtain a pitch angle judging result;

or judging whether the yaw angle falls in a second angle range or not to obtain a yaw angle judging result;

or judging whether the rolling angle falls within a third angle range or not to obtain a rolling angle result judgment result;

judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result;

acquiring shoulder characteristic point coordinates of the left shoulder characteristic point and the right shoulder characteristic point in a world coordinate system, and judging whether the current sitting posture of the target object is a poor sitting posture or not according to the shoulder characteristic point coordinates;

and acquiring a first included angle formed by the connecting line of the neck pixel characteristic points and the nose pixel characteristic points and a z coordinate axis in the world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, and judging whether the current sitting posture of the target object is poor sitting posture according to the first included angle judging result.

Optionally, the method for detecting a bad sitting posture, wherein the bad sitting posture comprises a head left turn transition, a head right turn transition, a head overlook transition, a head bottom view transition, a head left bias transition and a head right bias transition;

judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result, and then further comprising:

if the pitch angle is smaller than the minimum value of the second angle range, judging that the head is overlooked excessively; if the pitch angle is larger than the maximum value of the second angle range, judging that the head is excessively looked up;

if the yaw angle is smaller than the minimum value of the first angle range, judging that the head turns left excessively; if the yaw angle is larger than the maximum value of the first angle range, judging that the head turns right excessively;

if the rolling angle is smaller than the minimum value of the third angle range, judging that the head is excessively left offset; and if the rolling angle is larger than the maximum value of the third angle range, judging that the head is excessively right-deviated.

Optionally, the bad sitting posture detection method, wherein the bad sitting posture comprises a left shoulder height, a right shoulder height, a left shoulder forward inclination, a right shoulder forward inclination and a shoulder too close to a screen;

The step of obtaining shoulder feature point coordinates of the left shoulder feature point and the right shoulder feature point in a world coordinate system, and judging whether the current sitting posture of the target object is a bad sitting posture according to the shoulder feature point coordinates, and then further comprising:

acquiring a first left included angle and a first right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a y coordinate axis in the world coordinate system respectively, comparing the first left included angle and the first right included angle with a first preset angle respectively, and judging that the single side of the shoulder is too high if the first left included angle or the first right included angle is larger than the first preset angle;

acquiring a second left included angle and a second right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a z coordinate axis in the world coordinate system respectively, comparing the magnitudes of the second left included angle and the second right included angle with a second preset angle respectively, and judging that the shoulder leans forward if the second left included angle or the second right included angle is larger than the first preset angle;

comparing the positions of the left shoulder characteristic points and the right shoulder characteristic points on a y coordinate axis and a z coordinate axis in the world coordinate system;

if the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder is high;

If the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is smaller than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder is high;

if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder tilts forward;

if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is smaller than that of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder leans forward;

and if the difference value between the value of the z coordinate axis of the left shoulder feature point in the world coordinate system and the value of the z coordinate axis of the right shoulder feature point in the world coordinate system is smaller than a first preset distance, judging that the shoulder is too close to the screen.

Optionally, the method for detecting a bad sitting posture, wherein the bad sitting posture comprises head forward tilting and head backward tilting;

the method comprises the steps of obtaining a first included angle formed by a connecting line of the neck pixel feature points and the nose pixel feature points and a z coordinate axis in a world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, judging whether the current sitting posture of a target object is bad according to the first included angle judging result, and then further comprising:

If the first included angle is larger than the maximum value of the fourth angle range, judging that the head tilts forwards;

and if the first included angle is smaller than the minimum value of the fourth angle range, judging that the head tilts backwards.

In addition, in order to achieve the above object, the present invention also provides a bad sitting posture detection system, wherein the bad sitting posture detection system comprises:

the face window screening module is used for acquiring a human body image of a target object in the detection area, scanning the human body image and screening to obtain an area;

the feature point acquisition module is used for detecting face feature points, shoulder feature points and pixel feature points in the face window area;

the head Euler angle acquisition module is used for screening a preset number of face feature points and calculating head Euler angles according to the preset number of face feature points;

the result acquisition module is used for judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judgment result of the poor sitting posture.

In addition, to achieve the above object, the present invention also provides a terminal, wherein the terminal includes: the device comprises a memory, a processor and a bad sitting posture detection program stored on the memory and capable of running on the processor, wherein the bad sitting posture detection program realizes the steps of the bad sitting posture detection method when being executed by the processor.

In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium storing a bad sitting posture detection program which, when executed by a processor, implements the steps of the bad sitting posture detection method as described above.

In the invention, a human body image of a target object in a detection area is acquired, scanned, and a human face window area is obtained through screening; detecting face feature points, shoulder feature points and pixel feature points in a face window area; screening a preset number of face feature points, and calculating head Euler angles according to the preset number of face feature points; judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture. According to the invention, the characteristic points of the face and the shoulder are identified by adopting the lightweight PFLD and the OpenPose deep learning model, the characteristic points are not easily influenced by complex environments such as external illumination, and the operating speed of 12 frames can be achieved on GTX1080Ti in a multithreading mode, so that the real-time requirement is met, the bad sitting posture can be judged without additionally manufacturing a sample set, and the manpower is saved.

Drawings

FIG. 1 is a flow chart of a preferred embodiment of the bad sitting posture detection method of the present invention;

FIG. 2 is a schematic diagram of a bad sitting posture detection platform according to a preferred embodiment of the bad sitting posture detection method of the present invention;

FIG. 3 is a diagram of a window area for detecting a face in a preferred embodiment of the method for detecting a bad sitting posture according to the present invention;

FIG. 4 is a schematic diagram of detecting facial feature points in a preferred embodiment of the bad sitting posture detection method of the present invention;

FIG. 5 is a schematic diagram of the detection of shoulder feature points in a preferred embodiment of the bad sitting posture detection method of the present invention;

FIG. 6 is a schematic diagram of a face feature point screening method according to a preferred embodiment of the present invention;

FIG. 7 is a schematic diagram showing the definition of Euler angle direction in a preferred embodiment of the method for detecting a bad sitting posture according to the present invention;

FIG. 8 is a schematic view showing the head leaning forward in a preferred embodiment of the method for detecting a bad sitting posture according to the present invention;

FIG. 9 is a schematic view of a single shoulder height in a preferred embodiment of the method for detecting a bad sitting posture according to the present invention;

FIG. 10 is a schematic view of a single-sided shoulder forward leaning in a preferred embodiment of the method for detecting a bad sitting posture of the invention;

FIG. 11 is a schematic diagram of a method for detecting a bad sitting posture according to the preferred embodiment of the invention;

FIG. 12 is a schematic diagram of detecting a bad sitting posture according to the preferred embodiment of the method for detecting a bad sitting posture of the present invention;

FIG. 13 is a schematic diagram of a person not detected in a preferred embodiment of the method for detecting a bad sitting posture of the invention;

FIG. 14 is a flow chart of the bad sitting posture detection in the preferred embodiment of the bad sitting posture detection method of the present invention;

FIG. 15 is a schematic diagram of a preferred embodiment of a bad sitting posture detection system of the present invention;

FIG. 16 is a schematic view of the operating environment of a preferred embodiment of the terminal of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

The method for detecting a bad sitting posture according to the preferred embodiment of the present invention, as shown in fig. 1, comprises the following steps:

step S10, acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area.

Specifically, acquiring a human body image of a target object in a detection area; scanning the human body image in a sliding window mode; and marking a face window area in the human body image by using the trained cascade classifier.

As shown in fig. 2 and fig. 3, the bad sitting posture detection platform acquires images in real time, uses Haar features to distinguish human body images and non-human body images of different areas in the images, and scans the detected human body images in a sliding window mode; and training a cascade classifier by using an AdaBoost algorithm, screening out a possible face window area from a human body image by using the trained cascade classifier (a deep learning algorithm and the like can be used for identifying the face area), and marking out the face window area.

Most of the existing bad sitting posture detection technologies can be realized by wearing equipment outside, so that physiological and psychological burden of a user can be increased, discomfort is caused, and the use comfort of the user is reduced.

And step S20, detecting face feature points, shoulder feature points and pixel feature points in the face window area.

Specifically, face feature points of the target object in the face window area are detected based on a depth camera and a PFLD face feature point detection algorithm; detecting shoulder feature points of the target object in the face window area based on a lightweight OpenPose model subjected to deep learning; and detecting pixel characteristic points of the target object in the face window area based on the depth camera.

In the range of the face window area, as shown in fig. 4, the face feature points are detected by using a D435i depth camera of RealSense and a PFLD face feature point detection algorithm through a dlib library of OpenCV or a method of MTCNN and other deep learning methods, and head pose, namely euler angles of the head in a space coordinate system, is calculated through the positioned face feature points; compared with the OpenCV dlib library, the stability of the face feature point identification is better. The basic network of the PFLD is improved based on the MobileNet V2, so that the network performance can be maintained, and the parameters and the operand of the network, even the reasoning time, can be greatly reduced; can run at high speed on mobile equipment and has good detection precision. By default, no auxiliary network is used during testing. The aim of the method is to adjust the loss parameters according to the acquired head attitude angles in the training process, so that the model is more concerned with rare samples and samples with overlarge attitude angles, the predicted coordinates of the key points are more stable, and the robustness is better.

As shown in FIG. 5, the detection of the shoulder feature points uses a lightweight OpenPose, namely, a feature backup part uses a MobileNet structure with cavity convolution to replace the original VGG structure, the model has higher precision compared with the traditional OpenPose, is not easily influenced by complex environments such as external illumination, can reach the running speed of 26FPS on a CPU, can reach the running speed of 12 frames on GTX1080Ti in a multithreading mode, and greatly improves the detection efficiency.

The traditional OpenPose is a model for detecting human skeleton points, and can detect 18 skeleton points of a human body. The traditional OpenPose adopts a 'bottom-up' method, which is to detect all key points of human bodies in pictures, then correspond the key points to different individual characters, and the detection speed is superior to that of a method for detecting the key points after detecting the human bodies. Because the traditional OpenPose model has very low running speed, the traditional OpenPose model can hardly run on a CPU, and feature points of the shoulders can be extracted through an image processing method.

Through a lightweight OpenPose model, a D435i depth camera of RealSense is used for identifying a plurality of pixel feature points of a human body, and three-dimensional space pixel coordinates (x, y, z) of the pixel feature points in a world coordinate system can be obtained according to depth information.

And S30, screening a preset number of face feature points, and calculating to obtain head Euler angles according to the preset number of face feature points.

Specifically, acquiring face feature point coordinates of the preset number of face feature points in a world coordinate system; calculating according to the face feature point coordinates to obtain a corresponding rotation matrix; and obtaining a rotation vector of the rotation matrix by using a solvePnP function of OpenCV, and converting the rotation vector to obtain the head Euler angle.

Wherein, as shown in fig. 6, based on 98 face feature points detected in the PFLD, 14 face feature points are determined further according to face anatomical criteria, i.e., key positions of the face and shapes and structures capable of accurately describing the face, including left eyebrow left, left eyebrow right, right eyebrow left, right eyebrow right, left eye left, left eye right, right eye left, right eye right, nose left, nose right, left lip, right lip, lower jaw (these are feature points that are easily recognized by the face), and a 3D universal head model is used, i.e., in world coordinate system, a head model is established, the coordinates of each point in the world coordinate system of the model are fixed; finding the corresponding world coordinate system in the head rigid body model can be regarded as mapping from 3D points to 2D points, when enough point pairs (for example, the 14 determined face feature points) exist, we can solve the corresponding rotation matrix, and according to the 14 face feature points, solve the rotation vector of the head through the SolvePnP function, so as to perform gesture estimation.

Further, as shown in fig. 7, the inside and outside parameter matrix and distortion coefficient of the depth camera are known, the 14 face feature points obtained by recognition are provided, the 3D points and the corresponding 2D points, and the parameters of the depth camera are provided, the rotation vector of the rotation matrix is calculated by using the OpenCV sovepnp function, and then the rotation vector is converted into euler angles to obtain the pose information of the head.

And S40, judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture.

Wherein the head Euler angles comprise pitch angle, yaw angle and roll angle; the shoulder feature points comprise left shoulder feature points and right shoulder feature points; the pixel characteristic points comprise neck pixel characteristic points and nose pixel characteristic points; the bad sitting postures comprise excessive left-turning of the head, excessive right-turning of the head, excessive overlooking of the head, excessive upward viewing of the head, excessive left-turning of the head, excessive right-turning of the head, high left shoulder, high right shoulder, forward tilting of the left shoulder, forward tilting of the right shoulder, backward tilting of the head and too close of the eyes to the screen.

Specifically, judging whether the pitch angle falls in a first angle range or not to obtain a pitch angle judging result; or judging whether the yaw angle falls in a second angle range or not to obtain a yaw angle judging result; or judging whether the rolling angle falls within a third angle range or not to obtain a rolling angle result judgment result; judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result; acquiring shoulder characteristic point coordinates of the left shoulder characteristic point and the right shoulder characteristic point in a world coordinate system, and judging whether the current sitting posture of the target object is a poor sitting posture or not according to the shoulder characteristic point coordinates; the method comprises the steps of obtaining a first included angle formed by a connecting line of the neck pixel characteristic points and the nose pixel characteristic points and a z coordinate axis in a world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, judging whether the current sitting posture of a target object is a bad sitting posture according to the first included angle judging result, further judging the type of the bad sitting posture, pertinently reminding a current user how to correct the bad sitting posture, avoiding additionally manufacturing a sample set, avoiding training a model, judging the bad sitting posture according to a joint angle threshold value, and saving space resources and manpower to a great extent.

Further, if the pitch angle is smaller than the minimum value of the second angle range, judging that the head is overlooked excessively; if the pitch angle is larger than the maximum value of the second angle range, judging that the head is excessively looked up; if the yaw angle is smaller than the minimum value of the first angle range, judging that the head turns left excessively; if the yaw angle is larger than the maximum value of the first angle range, judging that the head turns right excessively; if the rolling angle is smaller than the minimum value of the third angle range, judging that the head is excessively left offset; and if the rolling angle is larger than the maximum value of the third angle range, judging that the head is excessively right-deviated.

Further, a first left included angle and a first right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a y coordinate axis in the world coordinate system are obtained, the first left included angle and the first right included angle are compared with a first preset angle respectively, and if the first left included angle or the first right included angle is larger than the first preset angle, the single side of the shoulder is judged to be too high; acquiring a second left included angle and a second right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a z coordinate axis in the world coordinate system respectively, comparing the magnitudes of the second left included angle and the second right included angle with a second preset angle respectively, and judging that the shoulder leans forward if the second left included angle or the second right included angle is larger than the first preset angle; comparing the positions of the left shoulder characteristic points and the right shoulder characteristic points on a y coordinate axis and a z coordinate axis in the world coordinate system; if the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder is high; if the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is smaller than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder is high; if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder tilts forward; if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is smaller than that of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder leans forward; and if the difference value between the value of the z coordinate axis of the left shoulder feature point in the world coordinate system and the value of the z coordinate axis of the right shoulder feature point in the world coordinate system is smaller than a first preset distance, judging that the shoulder is too close to the screen.

Further, if the first included angle is larger than the maximum value of the fourth angle range, judging that the head tilts forward; and if the first included angle is smaller than the minimum value of the fourth angle range, judging that the head tilts backwards.

And comparing the calculated pitch angle, yaw angle and roll angle with corresponding set thresholds respectively, and judging that the sitting posture of the current user is poor if the pitch angle does not fall within (-1 degrees, 1 degrees), or the yaw angle does not fall within (-10 degrees, 10 degrees), or the roll angle does not fall within (-15 degrees, 15 degrees).

Further, if the pitch angle is larger than 1 degree, the head is judged to be excessively upward, and if the pitch angle is smaller than-1 degree, the head is judged to be excessively downward; if the yaw angle is larger than 10 degrees, judging that the head is excessively left, and if the yaw angle is smaller than-10 degrees, judging that the head is excessively right; if the rolling angle is larger than 15 degrees, the head is judged to be excessively turned right, and if the rolling angle is smaller than-15 degrees, the head is judged to be excessively turned left.

As shown in fig. 8, 9 and 10, coordinates of the neck and nose pixel feature points on a world coordinate system are obtained, head forward inclination is judged through included angles θ1 and θ2 of connecting lines of the neck and the nose in the z-axis direction, θ1 is in a normal angle threshold range, namely, atan (Δz/Δy) is in a normal angle threshold range (30 degrees and 40 degrees), θ2 exceeds the normal angle threshold, namely, atan (Δz/Δy) is larger than 40 degrees, head forward inclination is judged, and head backward inclination is judged when atan (Δz/Δy) is smaller than 30 degrees; wherein Deltaz is the difference value of the z coordinate axis of the neck and nose pixel characteristic points on the world coordinate system, and Deltay is the difference value of the y coordinate axis of the neck and nose pixel characteristic points on the world coordinate system; acquiring coordinates of two shoulder feature points on a world coordinate system, for example, the coordinates of a left shoulder feature point are (x 3, y3, z 3), the coordinates of a right shoulder feature point are (x 4, y4, z 4), judging the height of a shoulder by comparing the included angles theta 3 and theta 4 on the y coordinate axes of 2 shoulder feature points, and judging that the single side of the shoulder is too high if atan (deltay/deltax) is larger than 5 degrees; further, judging which side shoulder is too high according to the sizes of y3 and y 4; if y3> y4, the left shoulder is judged to be too high, and if y3< y4, the right shoulder is judged to be too high; judging the forward inclination of the shoulder by comparing the included angles theta 5 and theta 6 of the left and right points of the shoulder on the z axis, and judging the forward inclination of one side of the shoulder to be excessive if atan (deltaz/deltax) is larger than 5 degrees; further according to the size of z3 and z4, judging which side shoulder is excessively inclined forward; if z3> z4, judging that the left shoulder is excessively inclined forward, and if z3< z4, judging that the right shoulder is excessively inclined forward; if the difference between z3 and z4 is less than 50 cm, the current user shoulder is judged to be too close to the screen.

Further, a Pyqt5 is utilized to design a human-computer interaction interface, and clear icons, labels and indications are used to enable data to be visualized, so that a user can quickly understand and operate the interface; explicit feedback and guidance is provided to assist the user in properly using the system. As shown in fig. 11, the bad sitting posture detection platform does not detect the bad sitting posture; as shown in fig. 12, the bad sitting posture detection platform detects a bad sitting posture; as shown in fig. 13, the bad sitting posture detection platform does not detect the user.

The head and shoulder characteristic points are combined to analyze the bad sitting postures, a set of upper computer is developed and designed, a man-machine interaction interface is displayed in real time, the bad sitting postures are detected, and 13 bad sitting postures can be detected altogether; the depth camera is also used for detecting bad sitting postures, a user does not need to wear any equipment, the operation is simple, the comfort level of the user is improved, the feature points of the face and the shoulders are detected by using the PFLD and the OpenPose deep learning model, the influence of complex environments such as external illumination is not easy to be affected, and compared with the dlib library of OpenCV, the stability of the feature points of the face is better. The characteristic points of the face and the shoulder are identified by adopting a lightweight PFLD and OpenPose deep learning model, and the operating speed of 12 frames can be achieved on GTX1080Ti by adopting a multithreading mode (shown in fig. 14), so that the requirement of real-time performance is met; the model is not required to be additionally manufactured and trained, the bad sitting posture is judged by means of the joint angle threshold value, and the labor is saved.

Further, as shown in fig. 15, based on the above-mentioned bad sitting posture detection method, the present invention further provides a bad sitting posture detection system, wherein the bad sitting posture detection system includes:

the face window screening module 51 is configured to obtain a human body image of a target object in a detection area, scan the human body image, and screen the detection area to obtain an area;

a feature point obtaining module 52, configured to detect a face feature point, a shoulder feature point, and a pixel feature point in the face window area;

the head euler angle obtaining module 53 is configured to screen a preset number of face feature points, and calculate a head euler angle according to the preset number of face feature points;

the result obtaining module 54 is configured to determine whether the current sitting posture of the target object is a poor sitting posture according to the head euler angle, the shoulder feature point or the pixel feature point, and output a determination result of the poor sitting posture.

Further, as shown in fig. 16, based on the above-mentioned bad sitting posture detection method and system, the present invention further provides a terminal correspondingly, which includes a processor 10, a memory 20 and a display 30. Fig. 16 shows only some of the components of the terminal, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead.

The memory 20 may in some embodiments be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 20 may in other embodiments also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal. Further, the memory 20 may also include both an internal storage unit and an external storage device of the terminal. The memory 20 is used for storing application software installed in the terminal and various data, such as program codes of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores a bad sitting posture detection program 40, and the bad sitting posture detection program 40 can be executed by the processor 10, so as to implement the bad sitting posture detection method in the present application.

The processor 10 may in some embodiments be a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chip for executing program code or processing data stored in the memory 20, for example for performing the bad sitting posture detection method or the like.

The display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like in some embodiments. The display 30 is used for displaying information at the terminal and for displaying a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

In one embodiment, the following steps are implemented when the processor 10 executes the bad sitting posture detection program 40 in the memory 20:

acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area;

detecting face feature points, shoulder feature points and pixel feature points in the face window area;

screening a preset number of face feature points, and calculating head Euler angles according to the preset number of face feature points;

judging whether the current sitting posture of the target object is a poor sitting posture or not according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture.

The method comprises the steps of acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area, wherein the method specifically comprises the following steps:

Acquiring a human body image of a target object in a detection area;

scanning the human body image in a sliding window mode;

and marking a face window area in the human body image by using the trained cascade classifier.

The detecting face feature points, shoulder feature points and pixel feature points in the face window area specifically includes:

detecting face feature points of the target object in the face window area based on a depth camera and a PFLD face feature point detection algorithm;

detecting shoulder feature points of the target object in the face window area based on a lightweight OpenPose model subjected to deep learning;

and detecting pixel characteristic points of the target object in the face window area based on the depth camera.

The head euler angle is calculated according to the preset number of the face feature points, and specifically includes:

acquiring face feature point coordinates of the preset number of face feature points in a world coordinate system;

calculating according to the face feature point coordinates to obtain a corresponding rotation matrix;

and obtaining a rotation vector of the rotation matrix by using a solvePnP function of OpenCV, and converting the rotation vector to obtain the head Euler angle.

Wherein the head euler angles include pitch angle, yaw angle and roll angle; the shoulder characteristic points comprise left shoulder characteristic points and right shoulder characteristic points; the pixel characteristic points comprise neck pixel characteristic points and nose pixel characteristic points;

judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, wherein the method specifically comprises the following steps of:

judging whether the pitch angle falls in a first angle range or not to obtain a pitch angle judging result;

or judging whether the yaw angle falls in a second angle range or not to obtain a yaw angle judging result;

or judging whether the rolling angle falls within a third angle range or not to obtain a rolling angle result judgment result;

judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result;

acquiring shoulder characteristic point coordinates of the left shoulder characteristic point and the right shoulder characteristic point in a world coordinate system, and judging whether the current sitting posture of the target object is a poor sitting posture or not according to the shoulder characteristic point coordinates;

and acquiring a first included angle formed by the connecting line of the neck pixel characteristic points and the nose pixel characteristic points and a z coordinate axis in the world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, and judging whether the current sitting posture of the target object is poor sitting posture according to the first included angle judging result.

Wherein the poor sitting posture comprises excessive left-turn of the head, excessive right-turn of the head, excessive overlook of the head, excessive look-up of the head, excessive left-bias of the head and excessive right-bias of the head;

judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result, and then further comprising:

if the pitch angle is smaller than the minimum value of the second angle range, judging that the head is overlooked excessively; if the pitch angle is larger than the maximum value of the second angle range, judging that the head is excessively looked up;

if the yaw angle is smaller than the minimum value of the first angle range, judging that the head turns left excessively; if the yaw angle is larger than the maximum value of the first angle range, judging that the head turns right excessively;

if the rolling angle is smaller than the minimum value of the third angle range, judging that the head is excessively left offset; and if the rolling angle is larger than the maximum value of the third angle range, judging that the head is excessively right-deviated.

Wherein the bad sitting posture comprises left shoulder height, right shoulder height, left shoulder forward tilting, right shoulder forward tilting and too close of the shoulders to the screen;

The step of obtaining shoulder feature point coordinates of the left shoulder feature point and the right shoulder feature point in a world coordinate system, and judging whether the current sitting posture of the target object is a bad sitting posture according to the shoulder feature point coordinates, and then further comprising:

acquiring a first left included angle and a first right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a y coordinate axis in the world coordinate system respectively, comparing the first left included angle and the first right included angle with a first preset angle respectively, and judging that the single side of the shoulder is too high if the first left included angle or the first right included angle is larger than the first preset angle;

acquiring a second left included angle and a second right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a z coordinate axis in the world coordinate system respectively, comparing the magnitudes of the second left included angle and the second right included angle with a second preset angle respectively, and judging that the shoulder leans forward if the second left included angle or the second right included angle is larger than the first preset angle;

comparing the positions of the left shoulder characteristic points and the right shoulder characteristic points on a y coordinate axis and a z coordinate axis in the world coordinate system;

if the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder is high;

If the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is smaller than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder is high;

if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder tilts forward;

if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is smaller than that of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder leans forward;

and if the difference value between the value of the z coordinate axis of the left shoulder feature point in the world coordinate system and the value of the z coordinate axis of the right shoulder feature point in the world coordinate system is smaller than a first preset distance, judging that the shoulder is too close to the screen.

Wherein the poor sitting posture includes head forward tilting and head backward tilting;

the method comprises the steps of obtaining a first included angle formed by a connecting line of the neck pixel feature points and the nose pixel feature points and a z coordinate axis in a world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, judging whether the current sitting posture of a target object is bad according to the first included angle judging result, and then further comprising:

If the first included angle is larger than the maximum value of the fourth angle range, judging that the head tilts forwards;

and if the first included angle is smaller than the minimum value of the fourth angle range, judging that the head tilts backwards.

The present invention also provides a computer-readable storage medium storing a bad sitting posture detection program which, when executed by a processor, implements the steps of the bad sitting posture detection method as described above.

In summary, the present invention provides a method for detecting bad sitting postures and related devices, the method comprising: acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area; detecting face feature points, shoulder feature points and pixel feature points in a face window area; screening a preset number of face feature points, and calculating head Euler angles according to the preset number of face feature points; judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture. According to the invention, the PFLD and the OpenPose deep learning model are used for detecting the characteristic points of the face and the shoulder, so that the method is not easily influenced by complex environments such as external illumination, and compared with the dlib library of the OpenCV, the method has the advantage that the stability of identifying the characteristic points of the face is better; the characteristic points of the face and the shoulder are identified by adopting a lightweight PFLD and OpenPose deep learning model, and the operating speed of 12 frames can be achieved on GTX1080Ti in a multithreading mode, so that the real-time requirement is met; the bad sitting posture is judged by means of the joint angle threshold without additionally manufacturing a sample set, so that the labor is saved.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal comprising the element.

Of course, those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by a computer program for instructing relevant hardware (e.g., processor, controller, etc.), the program may be stored on a computer readable storage medium, and the program may include the above described methods when executed. The computer readable storage medium may be a memory, a magnetic disk, an optical disk, etc.

It is to be understood that the invention is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.

Claims (10)

1. A method of detecting a bad sitting posture, the method comprising:

acquiring a human body image of a target object in a detection area, scanning the human body image, and screening to obtain a human face window area;

detecting face feature points, shoulder feature points and pixel feature points in the face window area;

screening a preset number of face feature points, and calculating head Euler angles according to the preset number of face feature points;

judging whether the current sitting posture of the target object is a poor sitting posture or not according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judging result of the poor sitting posture.

2. The bad sitting posture detection method according to claim 1, wherein the steps of obtaining a human body image of a target object in the detection area, scanning the human body image, and screening to obtain a human face window area comprise:

acquiring a human body image of a target object in a detection area;

scanning the human body image in a sliding window mode;

and marking a face window area in the human body image by using the trained cascade classifier.

3. The bad sitting posture detection method according to claim 1, wherein the detecting the face feature point, the shoulder feature point and the pixel feature point in the face window area specifically comprises:

Detecting face feature points of the target object in the face window area based on a depth camera and a PFLD face feature point detection algorithm;

detecting shoulder feature points of the target object in the face window area based on a lightweight OpenPose model subjected to deep learning;

and detecting pixel characteristic points of the target object in the face window area based on the depth camera.

4. The bad sitting posture detection method according to claim 1, wherein the head euler angle is calculated according to the preset number of face feature points, and specifically comprises:

acquiring face feature point coordinates of the preset number of face feature points in a world coordinate system;

calculating according to the face feature point coordinates to obtain a corresponding rotation matrix;

and obtaining a rotation vector of the rotation matrix by using a solvePnP function of OpenCV, and converting the rotation vector to obtain the head Euler angle.

5. A method of detecting a poor sitting posture according to claim 1, wherein the head euler angles include pitch angle, yaw angle and roll angle; the shoulder characteristic points comprise left shoulder characteristic points and right shoulder characteristic points; the pixel characteristic points comprise neck pixel characteristic points and nose pixel characteristic points;

Judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, wherein the method specifically comprises the following steps of:

judging whether the pitch angle falls in a first angle range or not to obtain a pitch angle judging result;

or judging whether the yaw angle falls in a second angle range or not to obtain a yaw angle judging result;

or judging whether the rolling angle falls within a third angle range or not to obtain a rolling angle result judgment result;

judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result;

acquiring shoulder characteristic point coordinates of the left shoulder characteristic point and the right shoulder characteristic point in a world coordinate system, and judging whether the current sitting posture of the target object is a poor sitting posture or not according to the shoulder characteristic point coordinates;

and acquiring a first included angle formed by the connecting line of the neck pixel characteristic points and the nose pixel characteristic points and a z coordinate axis in the world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, and judging whether the current sitting posture of the target object is poor sitting posture according to the first included angle judging result.

6. The method of claim 5, wherein the bad sitting posture comprises head left turn, head right turn, head top down turn, head bottom view turn, head left turn and head right turn;

judging whether the current sitting posture of the target object is a poor sitting posture according to the pitch angle judging result, the yaw angle judging result or the rolling angle judging result, and then further comprising:

if the pitch angle is smaller than the minimum value of the second angle range, judging that the head is overlooked excessively; if the pitch angle is larger than the maximum value of the second angle range, judging that the head is excessively looked up;

if the yaw angle is smaller than the minimum value of the first angle range, judging that the head turns left excessively; if the yaw angle is larger than the maximum value of the first angle range, judging that the head turns right excessively;

if the rolling angle is smaller than the minimum value of the third angle range, judging that the head is excessively left offset; and if the rolling angle is larger than the maximum value of the third angle range, judging that the head is excessively right-deviated.

7. The method of claim 5, wherein the bad sitting posture comprises left shoulder height, right shoulder height, left shoulder forward leaning, right shoulder forward leaning, and off-screen shoulder too close;

The step of obtaining shoulder feature point coordinates of the left shoulder feature point and the right shoulder feature point in a world coordinate system, and judging whether the current sitting posture of the target object is a bad sitting posture according to the shoulder feature point coordinates, and then further comprising:

acquiring a first left included angle and a first right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a y coordinate axis in the world coordinate system respectively, comparing the first left included angle and the first right included angle with a first preset angle respectively, and judging that the single side of the shoulder is too high if the first left included angle or the first right included angle is larger than the first preset angle;

acquiring a second left included angle and a second right included angle of the left shoulder characteristic point and the right shoulder characteristic point on a z coordinate axis in the world coordinate system respectively, comparing the magnitudes of the second left included angle and the second right included angle with a second preset angle respectively, and judging that the shoulder leans forward if the second left included angle or the second right included angle is larger than the first preset angle;

comparing the positions of the left shoulder characteristic points and the right shoulder characteristic points on a y coordinate axis and a z coordinate axis in the world coordinate system;

if the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder is high;

If the value of the y coordinate axis of the left shoulder feature point in the world coordinate system is smaller than the value of the y coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder is high;

if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is larger than the value of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the left shoulder tilts forward;

if the value of the z coordinate axis of the left shoulder feature point in the world coordinate system is smaller than that of the z coordinate axis of the right shoulder feature point in the world coordinate system, judging that the right shoulder leans forward;

and if the difference value between the value of the z coordinate axis of the left shoulder feature point in the world coordinate system and the value of the z coordinate axis of the right shoulder feature point in the world coordinate system is smaller than a first preset distance, judging that the shoulder is too close to the screen.

8. The method of claim 5, wherein the bad sitting posture comprises head forward tilting and head backward tilting;

the method comprises the steps of obtaining a first included angle formed by a connecting line of the neck pixel feature points and the nose pixel feature points and a z coordinate axis in a world coordinate system, judging whether the first included angle falls in a fourth angle range, obtaining a first included angle judging result, judging whether the current sitting posture of a target object is bad according to the first included angle judging result, and then further comprising:

If the first included angle is larger than the maximum value of the fourth angle range, judging that the head tilts forwards;

and if the first included angle is smaller than the minimum value of the fourth angle range, judging that the head tilts backwards.

9. A bad sitting posture detection system, characterized in that the bad sitting posture detection system comprises:

the face window screening module is used for acquiring a human body image of a target object in the detection area, scanning the human body image and screening to obtain an area;

the feature point acquisition module is used for detecting face feature points, shoulder feature points and pixel feature points in the face window area;

the head Euler angle acquisition module is used for screening a preset number of face feature points and calculating head Euler angles according to the preset number of face feature points;

the result acquisition module is used for judging whether the current sitting posture of the target object is a poor sitting posture according to the head Euler angle, the shoulder characteristic points or the pixel characteristic points, and outputting a judgment result of the poor sitting posture.

10. A terminal, the terminal comprising: a memory, a processor, and a bad sitting posture detection program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the bad sitting posture detection method according to any one of claims 1 to 7.