WO2021068781A1 - 一种疲劳状态识别方法、装置和设备 - Google Patents

一种疲劳状态识别方法、装置和设备 Download PDF

Info

Publication number
WO2021068781A1
WO2021068781A1 PCT/CN2020/118353 CN2020118353W WO2021068781A1 WO 2021068781 A1 WO2021068781 A1 WO 2021068781A1 CN 2020118353 W CN2020118353 W CN 2020118353W WO 2021068781 A1 WO2021068781 A1 WO 2021068781A1
Authority
WO
WIPO (PCT)
Prior art keywords
fatigue state
motion data
point motion
joint point
fatigue
Prior art date
Application number
PCT/CN2020/118353
Other languages
English (en)
French (fr)
Inventor
李华亮
杨志欣
张凯
刘羽中
沈雅利
熊超琳
Original Assignee
广东电网有限责任公司电力科学研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广东电网有限责任公司电力科学研究院 filed Critical 广东电网有限责任公司电力科学研究院
Publication of WO2021068781A1 publication Critical patent/WO2021068781A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Electricity, gas or water supply

Definitions

  • This application relates to the technical field of fatigue state recognition, and in particular to a fatigue state recognition method, device and equipment.
  • the scale of the power grid is getting larger and larger, and the safety management of the power grid is not only the safety management of all kinds of power equipment, but also the work safety of the grid staff.
  • the fatigue state of power grid workers will affect the safety of power grid workers during power operations. Therefore, it is necessary to know the fatigue state of power grid workers in time.
  • the acquisition of the fatigue status of grid staff mainly relies on the self-report of the grid staff, allowing the grid staff to report their current fatigue status by themselves.
  • the manager takes corresponding measures , To avoid safety accidents.
  • power grid workers interrupt the current power job to report fatigue status or fill in fatigue measurement questionnaires when performing power operations, it will not only have poor timeliness, but will also affect the work efficiency of power grid workers and affect the smooth execution of power operations. It brings about the safety of power grid movement and the safety of power grid workers, and the reliability is low.
  • This application provides a fatigue state identification method, device and equipment, which are used to solve the technical problems of poor timeliness and low reliability of the existing fatigue state identification methods of power grid workers.
  • the first aspect of the present application provides a method for identifying a fatigue state, including:
  • the independent variable of the preset fatigue state prediction model is joint point motion data, and the dependent variable is the fatigue state;
  • the prediction result of the preset fatigue state prediction model is output to obtain the fatigue state of the person in the fatigue state to be identified.
  • the step of inputting the real-time joint point motion data into a preset fatigue state prediction model for fatigue state prediction also includes:
  • the established behavior data fatigue state recognition model is trained, and the trained behavior data fatigue state recognition model is used as the preset fatigue state prediction model.
  • the behavior data fatigue state recognition model is an SVM classification model.
  • the acquisition of the joint point motion data sample set and the corresponding relationship between each joint point motion data sample and the fatigue state further includes:
  • Obtaining the joint point motion data sample provides the joint point motion data of the person in a normal walking state in the preset walking area;
  • the psychological characteristic scale includes: a Pittsburgh sleep quality index scale, a multidimensional fatigue scale, and/or a simulated visual scale.
  • the preset fatigue induction method includes performing a CPT task and/or a filtering task.
  • the second aspect of the present application provides a fatigue state recognition device, including:
  • the picture acquisition module is used to acquire the real-time behavior picture of the fatigued person to be identified, which is taken by the camera device;
  • a preprocessing module for preprocessing the real-time behavior picture to obtain real-time joint point motion data
  • the output module is used to output the prediction result of the preset fatigue state prediction model to obtain the fatigue state of the fatigue state to be identified.
  • it also includes:
  • the sample module is used to obtain the joint point motion data sample set and the corresponding relationship between each joint point motion data sample and the fatigue state;
  • the training module is used to train the established behavior data fatigue state recognition model based on the exercise data sample set and the corresponding relationship, and use the trained behavior data fatigue state recognition model as the preset fatigue state Forecast model.
  • it also includes:
  • Induction module which is used to induce different degrees of fatigue state of the personnel based on the preset fatigue induction method to induce joint point motion data samples;
  • a data acquisition module for acquiring the joint point motion data sample to provide the joint point motion data of a person in a normal walking state in a preset walking area
  • the psychological feature acquisition module is used to obtain the psychological feature scale completed by the joint point motion data sample provider within a specified time
  • the relationship establishment module is used to establish the joint movement data sample set and the corresponding relationship between each joint movement data sample and the fatigue state based on the joint movement data of the normal walking state and the psychological feature scale.
  • a third aspect of the present application provides a fatigue state recognition device, the device includes a processor and a memory:
  • the memory is used to store program code and transmit the program code to the processor
  • the processor is configured to execute any fatigue state identification method described in the first aspect according to instructions in the program code.
  • the fatigue state recognition method provided in this application includes: obtaining real-time behavior pictures of persons in fatigue state to be recognized by a camera device; preprocessing the real-time behavior pictures to obtain real-time joint point motion data; and moving the real-time joint points
  • the data is input to the preset fatigue state prediction model to predict the fatigue state.
  • the independent variable of the preset fatigue state prediction model is joint point motion data, and the dependent variable is the fatigue state; the prediction result of the preset fatigue state prediction model is output to obtain the fatigue to be identified The fatigue state of the state personnel.
  • the fatigue state recognition method provided in this application uses a camera device to take real-time pictures of the movement of the fatigued personnel to be identified during power grid operations, collect real-time behavior pictures of the fatigued personnel to be identified, and preprocess the real-time behavior pictures to obtain real-time behavior.
  • For joint motion data input real-time joint point motion data into the preset fatigue state prediction model.
  • the preset fatigue state prediction model uses joint point motion data as independent variables and fatigue state as dependent variable to output and input joint point motions
  • the fatigue state prediction result corresponding to the data, the fatigue state of the fatigue state to be identified is obtained.
  • the fatigue state recognition of the fatigue state to be identified is real-time, and will not affect the work efficiency of the fatigue state to be identified, and will not affect the fatigue state of the person to be identified.
  • the electric power operation work of identifying the fatigue state personnel is carried out normally, which solves the technical problems of poor timeliness and low reliability of the existing power grid workers' fatigue state identification methods.
  • FIG. 1 is a schematic flowchart of a method for identifying a fatigue state provided in an embodiment of this application;
  • FIG. 2 is a schematic diagram of another flow chart of a method for identifying a fatigue state provided in an embodiment of the application;
  • FIG. 3 is a schematic structural diagram of a fatigue state recognition device provided in an embodiment of the application.
  • FIG. 4 is a schematic diagram of the external structure of the camera device provided in the embodiment of the application.
  • FIG. 5 is a schematic diagram of the key points of the body provided in an embodiment of the application.
  • Step 101 Acquire a real-time behavior picture of a person to be identified in a fatigue state captured by the camera device.
  • Step 102 Preprocess the real-time behavior picture to obtain real-time joint point motion data.
  • Step 103 Input the real-time joint point motion data into the preset fatigue state prediction model to predict the fatigue state.
  • the independent variable of the preset fatigue state prediction model is joint point motion data, and the dependent variable is the fatigue state.
  • Step 104 Output the prediction result of the preset fatigue state prediction model to obtain the fatigue state of the person to be identified in the fatigue state.
  • the person in the fatigue state to be identified may be a power grid worker who is working on the power grid, and a camera device is used to take real-time behavior pictures of the person in the fatigue state to be identified.
  • the camera device may be a high-definition camera or a camera device.
  • the shape of can be a cube shape, as shown in Figure 4. The installation position of the camera device should meet the requirements of being able to continuously shoot unobstructed gait videos of the fatigued person to be identified.
  • the gait behavior data of real-time behavior pictures collected by a camera presents a data stream in the time domain, which is generally relatively long and is not suitable for feature extraction and selection of the original data. Since walking is a repetitive exercise, the middle section can be intercepted for data analysis. Perform data segmentation and data denoising preprocessing on the behavioral action pictures of the fatigued personnel to be identified in the work process recorded by the high-definition camera to obtain joint point motion data. Describe the movement pattern of the individual by capturing and tracking the key points of the body of the fatigued person, using the body key point detection and tracking algorithm developed based on the Openpose toolkit to capture the two-dimensional recording of the 18 key points of the torso activity in the video Coordinates, as shown in Figure 5.
  • the sampling frequency of the video is 25 Hz.
  • the key points of the body used in video analysis include 18 points, which are distributed on the central axis of the body and on both sides of the body, including left and right eyes, left and right ears, nose, neck, left and right shoulders, left and right elbows, left and right wrists, left and right thighs, Left and right knees and left and right ankles.
  • the sampling frequency of the video is 25 Hz
  • the general time of a complete walking action is about 1 second
  • 200 frames of data can be randomly intercepted as the subsequent analysis data.
  • the data of different lengths are standardized to the same length, which is convenient for processing and improves the efficiency of calculation.
  • Signal denoising generally uses filtering, which generally includes spatial filtering and frequency domain filtering.
  • Frequency filtering needs to be processed by Fourier transform to frequency domain first and then inversely transformed back to spatial domain to restore the signal.
  • Spatial domain filtering is to directly transform the signal data to achieve the purpose of filtering. It is a kind of neighborhood operation, that is, any value of the output signal is obtained by using a certain algorithm, according to the value in a certain neighborhood around the input signal data. If the output signal is a linear combination of the input signal neighborhood, it is called linear filtering (such as the most common mean filtering and Gaussian filtering), otherwise it is non-linear filtering (such as median filtering, edge-preserving filtering, etc.).
  • linear filtering such as the most common mean filtering and Gaussian filtering
  • non-linear filtering such as median filtering, edge-preserving filtering, etc.
  • a low-pass filtering method can be used to denoise the original low-frequency signal.
  • Mean filtering is a commonly used signal filtering and denoising method, which is essentially a low-pass filtering method. This method is simple in operation and has good denoising ability for Gaussian noise.
  • the camera device collects real-time behavior pictures of the emotional personnel to be recognized in real time. After the real-time behavior pictures are data preprocessed, real-time joint point motion data is obtained, and the real-time joint point motion data is input into the predictive system.
  • the emotion prediction model is set to predict the fatigue state.
  • the independent variable of the preset fatigue state prediction model is the joint point motion data, and the dependent variable is the fatigue state.
  • the preset fatigue state prediction model is a machine learning model, and there is a mapping between the independent variable and the dependent variable. Therefore, by inputting joint point motion data to the preset fatigue state prediction model, the output fatigue state prediction results can be obtained.
  • the fatigue state recognition method uses a camera device to take real-time photography of the movement of the fatigued personnel to be identified during power grid operations, collect real-time behavior pictures of the fatigued personnel to be identified, and preprocess the real-time behavior pictures After obtaining the real-time joint point motion data, the real-time joint point motion data is input into the preset fatigue state prediction model.
  • the preset fatigue state prediction model takes the joint point motion data as the independent variable and the fatigue state as the dependent variable.
  • the fatigue state prediction results corresponding to the motion data of the key points are obtained, and the fatigue state of the fatigue state to be identified is obtained.
  • the fatigue state recognition of the fatigue state to be identified is real-time, and will not affect the work efficiency of the fatigue state to be identified.
  • the normal operation of electric power operations that affect the fatigue status of the personnel to be identified is solved, and the technical problems of poor timeliness and low reliability of the fatigue status identification methods of the existing power grid workers are solved.
  • Step 201 Induce joint point motion data samples based on a preset fatigue induction method to provide personnel with different degrees of fatigue.
  • the preset fatigue induction methods in the embodiment of the present application include performing CPT (Continous Performance Test, continuous operation function test) tasks and/or filtering tasks.
  • CPT Continuous Performance Test, continuous operation function test
  • Persistent attention is defined as the attention to a specific single source of information in a continuous period of time.
  • the CPT task monitors the participant’s sustained attention and reaction time by measuring the participant’s response speed to visual stimuli.
  • Sustained attention is closely related to alertness awakening, and can more sensitively reflect the ability to maintain attention and impulsivity.
  • the main indicators are response time, false alarm rate and false alarm rate.
  • Filtering task In the Filtering task, the participant needs to judge whether the target stimulus changes direction between the two screens. The experiment manipulates the number of distracting stimuli. Distracted stimuli will interfere with subjects’ memory of the target stimulus position and direction, which is reflected in the response time and correct rate of judging whether the position and direction of the front and back pictures are consistent. Participants' response time will increase with the increase of distracting stimuli.
  • Step 202 Obtain joint point motion data samples to provide joint point motion data of a person in a normal walking state in a preset walking area.
  • the motion data samples of the joint points can be taken by the camera to provide the behavior pictures of the personnel, and the motion data of the joint points can be obtained after preprocessing the behavior pictures.
  • Step 203 Obtain a joint point motion data sample to provide a psychological feature scale completed by the personnel within a specified time.
  • the psychological characteristic scales in the embodiments of this application include: Pittsburgh sleep quality index (PSQI, Pittsburgh sleep quality index), multidimensional fatigue inventory (MFI-20, multidimensional fatigue inventory-20) and/or simulation Visual Scale (VAS-F, Visual Analogue Scale/Score).
  • PSQI Pittsburgh sleep quality index
  • MFI-20 multidimensional fatigue inventory
  • VAS-F simulation Visual Scale
  • Step 204 Based on the joint point motion data and the psychological feature scale in the normal walking state, establish the joint point motion data sample set and the corresponding relationship between the joint point motion data samples and the fatigue state.
  • the joint point motion data sample provider is first induced to have different degrees of fatigue.
  • the fatigue degree can be divided into grades, and a fatigue state of one degree of fatigue is induced at a time.
  • the person to be identified in the fatigue state is allowed to walk normally in the specified area according to the established rules, and obtain the joint point motion data within the preset time.
  • the preset time is over, the person to be identified in the fatigue state completes the filling of the psychological characteristic scale truthfully and accurately within the specified time.
  • Step 205 Obtain the joint point motion data sample set and the corresponding relationship between each joint point motion data sample and the fatigue state.
  • Step 206 Train the established behavior data fatigue state recognition model based on the exercise data sample set and the corresponding relationship, and use the trained behavior data fatigue state recognition model as a preset fatigue state prediction model.
  • Step 207 Acquire a real-time behavior picture of the person to be identified in the fatigue state captured by the camera device.
  • Step 208 Preprocess the real-time behavior picture to obtain real-time joint point motion data.
  • Step 209 Input the real-time joint point motion data into the preset fatigue state prediction model to predict the fatigue state.
  • the independent variable of the preset fatigue state prediction model is joint point motion data, and the dependent variable is the fatigue state.
  • Step 210 Output the prediction result of the preset fatigue state prediction model to obtain the fatigue state of the person to be identified in the fatigue state.
  • step 207 to step 210 in the embodiment of the present application are consistent with step 101 to step 104 in the previous embodiment.
  • the process of establishing a preset emotion prediction model can be expressed as: data segmentation of gait behavior data of behavior pictures to obtain key point data; data denoising processing of key point data; data translation of key point data; Perform feature extraction and dimensionality reduction processing on points; use the key point data after dimensionality reduction to train the behavioral data fatigue state recognition model to obtain a preset emotion prediction model.
  • the preprocessing method of performing data segmentation on the gait behavior data of the behavior picture to obtain key point data and performing data denoising processing on the key point data may be consistent with the preprocessing method in the previous embodiment.
  • time domain features show the characteristics of the signal data in the time dimension.
  • the frequency domain characteristics represent the characteristics of the signal in the frequency domain.
  • Time domain features are also called statistical features of signals, which represent the characteristics of data in the time dimension. This type of feature is directly calculated from time domain data, with a small amount of calculation and a simple process.
  • the main extracted features include arithmetic sum, mean, extreme value, variance, standard deviation, skewness, kurtosis, correlation coefficient between two axes, etc.
  • the frequency domain feature reflects the characteristics of the signal from the perspective of the frequency domain, and represents the frequency domain characteristics of the signal.
  • the signal Before extracting features in the frequency domain, the signal must first be converted from the time domain to the frequency domain.
  • the commonly used method is Fast Fourier Transform (FFT), and then the relevant features are calculated.
  • FFT Fast Fourier Transform
  • the frequency domain features usually extracted are: FFT coefficients, energy density, discrete cosine transform (Discrete Cosine Transform, DOC) coefficients, spectral energy (Spectral Energy), frequency domain entropy (Frequency Domain Entropy), power spectral density (Power Spectral Density, PSD) and so on.
  • Discrete Fourier Transform (DFT) is mainly used.
  • the Fast Fourier Transform (FFT) is a fast algorithm for calculating the Discrete Fourier Transform and its inverse transform. A total of 19 features were extracted for each body key point, a total of 17 key points, and a total of 323 features were extracted. PCA was used to reduce the dimensionality of the features to avoid overfitting of the prediction model.
  • the embodiment of this application uses a regression algorithm in machine learning to build a model.
  • the regression learning algorithm used is SVR (Support Vector Regression, Support Vector Regression), and SVM (Support Vector Machine, Support vector machine) is an important application branch.
  • SVM Simple Vector Machine, Support vector machine
  • SVM Simple Vector Machine, Support vector machine
  • training set is used to build the model
  • test set is used for evaluation.
  • the accuracy of the model when predicting unknown samples that is, generalization ability.
  • Cross Validation is one of the most useful methods for evaluating the performance of models built by different combinations of feature selection, dimensionality reduction, and learning algorithms. The basic idea is to group the original data (dataset), and part of it as a training set. The other part is used as a test set. First, use the training set for model training, and use the test set to test the trained model, which is used as an index to evaluate the predictive performance of the model. There are many types of cross-validation.
  • K-fold Cross Validation K-CV
  • K-CV K-fold Cross Validation
  • Commonly used K-fold cross-validation methods include 10-fold cross-validation (10-CV) and 5-fold cross-validation (5-CV). K-CV can effectively avoid the occurrence of over-fitting and under-learning, and the final results are more convincing.
  • the fatigue state identification method provided by the embodiments of the present application has the following advantages:
  • this application provides a fatigue state recognition device, including:
  • the picture acquisition module 301 is used to acquire real-time behavior pictures of the fatigued person to be identified, which is captured by the camera.
  • the preprocessing module 302 is used for preprocessing the real-time behavior picture to obtain real-time joint point motion data.
  • the input module 303 is used to input real-time joint point motion data into a preset fatigue state prediction model for fatigue state prediction.
  • the independent variable of the preset fatigue state prediction model is joint point motion data, and the dependent variable is the fatigue state.
  • the output module 304 is used to output the prediction result of the preset fatigue state prediction model to obtain the fatigue state of the fatigue state to be identified.
  • it can also include:
  • the sample module 305 is used to obtain the joint point motion data sample set and the corresponding relationship between each joint point motion data sample and the fatigue state.
  • the training module 306 is configured to train the established behavior data fatigue state recognition model based on the exercise data sample set and the corresponding relationship, and use the trained behavior data fatigue state recognition model as a preset fatigue state prediction model.
  • the inducing module 307 is used for inducing the joint point motion data sample based on the preset fatigue inducing method to provide the personnel with different degrees of fatigue state.
  • the data acquisition module 308 is used to acquire joint motion data samples to provide joint motion data of the person in a normal walking state in the preset walking area.
  • the mental feature acquisition module 309 is used to acquire the joint point motion data sample and provide the psychological feature scale completed by the personnel within the specified time.
  • the relationship establishment module 310 is used to establish the joint movement data sample set and the corresponding relationship between each joint movement data sample and the fatigue state based on the joint movement data and the psychological characteristic scale in the normal walking state.
  • This application provides an embodiment of a fatigue state recognition device, the device includes a processor and a memory:
  • the memory is used to store the program code and transmit the program code to the processor
  • the processor is configured to execute the fatigue state-based identification method in the foregoing fatigue state identification method embodiment according to the instructions in the program code.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), magnetic Various media that can store program codes, such as discs or optical discs.

Abstract

一种疲劳状态识别方法、装置和设备，对待识别疲劳状态人员在进行电网作业时的运动情况进行实时拍摄，采集待识别疲劳状态人员的实时行为图片，对实时行为图片进行预处理得到实时关节点运动数据，将实时关节点运动数据输入预置疲劳状态预测模型，预置疲劳状态预测模型以关节点运动数据为自变量，以疲劳状态为因变量，输出与输入的关节点运动数据对应的疲劳状态预测结果，得到待识别疲劳状态人员的疲劳状态，对待识别疲劳状态人员的疲劳状态识别具有实时性，且不会影响待识别疲劳状态人员的工作效率，不会影响到待识别疲劳状态人员的电力操作作业正常进行，解决了现有的电网工作人员的疲劳状态识别方式时效性差，且可靠性低的技术问题。

Description

一种疲劳状态识别方法、装置和设备
本申请要求于2019年10月12日提交中国专利局、申请号为201910968637.2、发明名称为“一种疲劳状态识别方法、装置和设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及疲劳状态识别技术领域，尤其涉及一种疲劳状态识别方法、装置和设备。
背景技术
随着电力系统的发展，电网的规模越来越大，对电网的安全管理不仅仅是对各类电力设备的安全管理，还要对电网工作人员的工作安全进行管理。
电网工作人员的疲劳状态会影响到电网工作人员在进行电力操作时的安全性，因此，及时得知电网工作人员的疲劳状态有着必要性。目前对电网工作人员疲劳状态的获取主要是依靠电网工作人员的自我报告，让电网工作人员自行报告自身当前的疲劳状态，当电网工作人员的疲劳状态存在工作安全隐患时，管理者采取相应的措施，避免发生安全事故。但是，如果电网工作人员在执行的电力作业时，中断当前的电力作业来报告疲劳状态或者填写疲劳测量问卷，不但时效性差，还会影响电网工作人员的工作效率，影响电力作业的顺利执行，容易带来电网运动安全和电网工作人员的安全问题，可靠性低。
发明内容
本申请提供了一种疲劳状态识别方法、装置和设备，用于解决现有的电网工作人员的疲劳状态识别方式时效性差，且可靠性低的技术问题。
本申请第一方面提供了一种疲劳状态识别方法，包括：
获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片；
对所述实时行为图片进行预处理，得到实时关节点运动数据；
将所述实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，所述预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态；
输出所述预置疲劳状态预测模型的预测结果，得到所述待识别疲劳状态人员的疲劳状态。
可选的，所述将所述实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，之前还包括：
获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系；
基于所述运动数据样本集和所述对应关系，对建立好的行为数据疲劳状态识别模型进行训练，将训练好的所述行为数据疲劳状态识别模型作为所述预置疲劳状态预测模型。
可选的，所述行为数据疲劳状态识别模型为SVM分类模型。
可选的，所述获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系，之前还包括：
基于预置疲劳诱发方式诱发关节点运动数据样本提供人员产生不同程度的疲劳状态；
获取所述关节点运动数据样本提供人员在预置行走区域内正常行走状态的关节点运动数据；
获取所述关节点运动数据样本提供人员在规定时间内完成的心理特征量表；
基于所述正常行走状态的关节点运动数据和所述心理特征量表，建立关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
可选的，所述心理特征量表包括：匹兹堡睡眠质量指数量表、多维疲劳量表和/或模拟视觉量表。
可选的，所述预置疲劳诱发方式包括执行CPT任务和/或Filtering任务。
本申请第二方面提供了一种疲劳状态识别装置，包括：
图片获取模块，用于获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片；
预处理模块，用于对所述实时行为图片进行预处理，得到实时关节点运动数据；
输入模块，用于将所述实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，所述预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态；
输出模块，用于输出所述预置疲劳状态预测模型的预测结果，得到所述待识别疲劳状态人员的疲劳状态。
可选的，还包括：
样本模块，用于获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系；
训练模块，用于基于所述运动数据样本集和所述对应关系，对建立好的行为数据疲劳状态识别模型进行训练，将训练好的所述行为数据疲劳状态识别模型作为所述预置疲劳状态预测模型。
可选的，还包括：
诱发模块，用于基于预置疲劳诱发方式诱发关节点运动数据样本提供人员产生不同程度的疲劳状态；
数据获取模块，用于获取所述关节点运动数据样本提供人员在预置行走区域内正常行走状态的关节点运动数据；
心理特征获取模块，用于获取所述关节点运动数据样本提供人员在规定时间内完成的心理特征量表；
关系建立模块，用于基于所述正常行走状态的关节点运动数据和所述心理特征量表，建立关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
本申请第三方面提供了一种疲劳状态识别设备，所述设备包括处理器以及存储器：
所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
所述处理器用于根据所述程序代码中的指令执行第一方面所述的任意一种疲劳状态识别方法。
从以上技术方案可以看出，本申请具有以下优点：
本申请中提供的一种疲劳状态识别方法，包括：获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片；对实时行为图片进行预处理，得到实时关节点运动数据；将实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态；输出预置疲劳状态预测模型的预测结果，得到待识别疲劳状态人员的疲劳状态。本申请提供的疲劳状态识别方法，使用摄像装置对待识别疲劳状态人员在进行电网作业时的运动情况进行实时拍摄，采集待识别疲劳状态人员的实时行为图片，对实时行为图片进行预处理后得到实时关节点运动数据，将实时关节点运动数据输入到预置疲劳状态预测模型中，预置疲劳状态预测模型以关节点运动数据为自变量，以疲劳状态为因变量，输出与输入的关节点运动数据对应的疲劳状态预测结果，得到待识别疲劳状态人员的疲劳状态，对待识别疲劳状态人员的疲劳状态识别具有实时性，且不会影响到待识别疲劳状态人员的工作效率，不会影响到待识别疲劳状态人员的电力操作作业正常进行，解决了现有的电网工作人员的疲劳状态识别方式时效性差，且可靠性低的技术问题。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其它的附图。
图1为本申请实施例中提供的一种疲劳状态识别方法的一个流程示意图；
图2为本申请实施例中提供的一种疲劳状态识别方法的另一个流程示意图；
图3为本申请实施例中提供的一种疲劳状态识别装置的结构示意图；
图4为本申请实施例中提供的摄像装置的外形结构示意图；
图5为本申请实施例中提供的身体关键点示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了便于理解，请参阅图1，本申请中提供了一种疲劳状态识别方法的实施例，包括：
步骤101、获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片。
步骤102、对实时行为图片进行预处理，得到实时关节点运动数据。
步骤103、将实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态。
步骤104、输出预置疲劳状态预测模型的预测结果，得到待识别疲劳状态人员的疲劳状态。
需要说明的是，本申请实施例中，待识别疲劳状态人员可以是正在进行电网工作的电网工作人员，采用摄像装置拍摄待识别疲劳状态人员的实时行为图片，摄像装置可以是高清摄像头，摄像装置的外形可以是立方体外形，如图4所示。摄像装置的安装位置应满足能够连续拍摄待识别疲劳状态人员的全身无遮挡的步态视频的要求。
利用摄像机采集的实时行为图片的步态行为数据在时域中呈现出数据流的形式，一般都比较长，不适合对原始数据进行特征提取和选择。由于走路是一个不断重复的运动，因此可截取中间段进行数据分析。对高清摄像头记录下的待识别疲劳状态人员在工作过程中的行为动作图片依次进行数据切分和数据去噪预处理，得到关节点运动数据。通过对待识别疲劳状态人员的身体关键点的捕捉和跟踪来描述个体的运动模式，使用基于Openpose工具包开发的躯体关键点检测及跟踪算法，捕捉记录视频中躯干活动的18个关键点的两维坐标，如图5所示。视频的采样频率为25Hz。 视频分析中所使用的躯体关键点包括18个点，分别分布在躯体的中轴线和躯体两侧上，包括左右眼睛、左右耳朵、鼻子、脖颈、左右肩膀、左右手肘、左右手腕、左右大腿、左右膝盖以及左右脚踝。
考虑到视频的采样频率是25Hz，而一个完整的走路动作一般时间为1秒左右，本申请实施例中可以随机截取数据的200帧(共8秒的视频)作为后续的分析数据，这样可以将长度不一的数据规范到相同的长度，方便处理，且提高运算的效率。
在实际场景中，由于自然环境的影响，主要关节点的运动数据中会带有噪声。为了提取更精确的运动特征，需要对原始信号进行信号去噪处理。
信号去噪一般采用滤波的方式，总体上讲包括空域滤波和频域滤波。频率滤波需要先进行傅立叶变换至频域处理然后再反变换回空间域还原信号，空域滤波是直接对信号的数据做空间变换达到滤波的目的。它是一种邻域运算，即输出信号任何的值都是通过采用一定的算法，根据输入信号数据周围一定邻域内的值得来的。如果输出信号是输入信号邻域的线性组合则称为线性滤波(例如最常见的均值滤波和高斯滤波)，否则为非线性滤波(如中值滤波、边缘保持滤波等)。
本申请实施例中，由于采集的人体的行为数据都是属于低频数据，因此，可以采用低通滤波的方法对原始低频信号去噪。均值滤波是一种常用的信号滤波去噪方法，其在本质上是一种低通滤波的方法。该方法运算简单，对高斯噪声具有良好的去噪能力。算法中用局部窗口内各信号的算术平均值替代中心信号的值，即g(x，y)＝1/m∑f(x，y)，m为该模板中包含当前像素在内的像素总个数。
待识别疲劳状态人员进入电力作业环境工作，摄像装置实时采集待识别情绪人员的实时行为图片，对实时行为图片进行数据预处理之后，得到实时关节点运动数据，将实时关节点运动数据输入到预置情绪预测模型进行疲劳状态预测，预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态，预置疲劳状态预测模型属于机器学习模型，自变量与因变量之间存在映射关系，因此，向预置疲劳状态预测模型输入关节点运动数据，可获得输出的疲劳状态预测结果。
本申请实施例中提供的疲劳状态识别方法，使用摄像装置对待识别疲劳状态人员在进行电网作业时的运动情况进行实时拍摄，采集待识别疲劳状态人员的实时行为图片，对实时行为图片进行预处理后得到实时关节点运动数据，将实时关节点运动数据输入到预置疲劳状态预测模型中，预置疲劳状态预测模型以关节点运动数据为自变量，以疲劳状态为因变量，输出与输入的关节点运动数据对应的疲劳状态预测结果，得到待识别疲劳状态人员的疲劳状态，对待识别疲劳状态人员的疲劳状态识别具有实时性，且不会影响到待识别疲劳状态人员的工作效率，不会影响到待识别疲劳状态人员的电力操作作业正常进行，解决了现有的电网工作人员的疲劳状态识别方式时效性差，且可靠性低的技术问题。
为了便于理解，请参阅图2，本申请提供了一种疲劳状态识别的另一个实施例，包括：
步骤201、基于预置疲劳诱发方式诱发关节点运动数据样本提供人员产生不同程度的疲劳状态。
需要说明的是，本申请实施例中预置疲劳诱发方式包括执行CPT(Continous Performance Test，持续操作功能测试)任务和/或Filtering任务。
CPT任务：持续性注意被定义为对特定单一的信息源在连续时间段内的注意保持，CPT任务通过测量被试者对视觉刺激的反应速度以监测其持续性注意力及反应时间。持续性注意与警觉唤醒紧密相连，可以比较灵敏地反映注意的维持能力和冲动性。主要指标有反应时、漏报率和误报率。
Filtering任务：在Filtering任务中，被试者需要对目标刺激在两屏之间是否改变了方向进行判断。实验操纵的是分心刺激的数目。分心刺激会干扰被试对目标刺激位置和方向的记忆，体现在对前后图片位置方向是否一致的判断反应时和正确率中。被试者的反应时会随着分心刺激的增多而增加。
步骤202、获取关节点运动数据样本提供人员在预置行走区域内正常行走状态的关节点运动数据。
可以通过摄像装置拍摄关节点运动数据样本提供人员的行为图片，对 行为图片进行预处理后得到关节点运动数据。
步骤203、获取关节点运动数据样本提供人员在规定时间内完成的心理特征量表。
需要说明的是，本申请实施例中心理特征量表包括：匹兹堡睡眠质量指数量表(PSQI，Pittsburgh sleep quality index)、多维疲劳量表(MFI-20，multidimensional fatigue inventory-20)和/或模拟视觉量表(VAS-F，Visual Analogue Scale/Score)。
步骤204、基于正常行走状态的关节点运动数据和心理特征量表，建立关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
需要说明的是，通过待识别疲劳状态人员的关节点运动数据来预测疲劳状态，需要建立关节点运动数据与疲劳状态的对应关系，关节点运动数据可以是运动加速度数据。本申请实施例中，首先诱发关节点运动数据样本提供人员的产生不同程度的疲劳状态，疲劳程度可以按等级划分，单次诱发一种疲劳程度的疲劳状态。在诱发疲劳状态的工作完成后，让待识别疲劳状态人员按照制定的规则要求在规定的区域内进行正常行走，获取预置时间内关节点运动数据。并在预置时间结束后，待识别疲劳状态人员在规定时间内真实准确的完成心理特征量表的填写。
步骤205、获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
步骤206、基于运动数据样本集和对应关系，对建立好的行为数据疲劳状态识别模型进行训练，将训练好的行为数据疲劳状态识别模型作为预置疲劳状态预测模型。
步骤207、获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片。
步骤208、对实时行为图片进行预处理，得到实时关节点运动数据。
步骤209、将实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态。
步骤210、输出预置疲劳状态预测模型的预测结果，得到待识别疲劳状态人员的疲劳状态。
需要说明的是，本申请实施例中的步骤207-步骤210与上一实施例中的步骤101-步骤104一致。
预置情绪预测模型的建立过程可以表述为：对行为图片的步态行为数据进行数据切分，得到关键点数据；对关键点数据进行数据去噪处理；对关键点数据进行数据平移；对关键点进行特征提取和降维处理；利用降维后的关键点数据去训练行为数据疲劳状态识别模型，得到预置情绪预测模型。
对行为图片的步态行为数据进行数据切分，得到关键点数据和对关键点数据进行数据去噪处理的预处理方法可以与上一实施例中的预处理方法一致。
由于用户在矩形区域内行走时，在矩形区域内的位置相对于Kinect来说不是固定的，这就会导致坐标系统不是唯一的，为了消除由于坐标原点的不一致带来的误差，坐标平移是必要的。以图5中的鼻为坐标原点进行坐标平移，用剩余的17个躯干关键点进行特征提取和建模。
在基于视频行为数据的疲劳状态识别装置的建立过程中，特征提取是整个过程的重要环节。一般的模式识别是无法直接处理原始数据的，关节点运动数据(加速度数据、空间坐标数据)并不能直接用于预测模型的建立。因此，在建立预测模型之前，需要对行为数据进行特征提取，提取出有用的信息作为特征值。对信号的特征提取，主要考虑两个方面的特征：时域特征和频域特征。时域特征展示了信号数据在时间维度上的特性。频域特征代表了信号在频域的特点。时域特征也称作信号的统计特征，代表了数据在时间维度上的特性，这类特征直接由时域数据计算而来，计算量较小，过程简单。在时域特征中，主要提取的特征有算术和、均值、极值、方差、标准差、偏度、峰度、两轴之间的相关系数等。
频域特征是从频域的角度反映信号的特性，代表信号的频域特点。在对信号进行频域特征提取之前首先要先将信号从时域转换到频域，常用的方法为快速傅里叶变换(Fast Fourier Transform，FFT)，然后在进行相关特 征的计算。通常提取的频域特征有：FFT系数、能量密度、离散余弦变换(Discrete Cosine Transform，DOC)系数、频谱能量(Spectral Energy)、频域熵(Frequency Domain Entropy)、功率谱密度(Power Spectral Density，PSD)等。由于本申请实施例中采集到的信号在时域上都是离散形式的，在做频域特征提取时，主要使用离散傅里叶变换(Discrete Fourier Transform，DFT)。快速傅里叶变换(FFT)是计算离散傅里叶变换及其逆变换的快速算法。针对每个躯体关键点共提取19个特征，共17个关键点，共提取323个特征，并采用PCA对特征进行降维，避免预测模型发生过拟合。
为了建立基于行为视频数据的疲劳状态识别装置，本申请实施例采用机器学习中的回归算法建立模型，使用的回归学习算法是SVR(Support Vector Regression，支持向量回归)，是SVM(Support Vector Machine，支持向量机)的重要应用分支。使用SVM建立分类模型，就是找到一个平面，让两个分类集合的支持向量或者所有的数据离分类平面最远；使用SVR建立回归模型，就是找到一个回归平面，让一个集合的所有数据到该平面的距离最近。
在机器学习和模式识别的相关研究中，经常会将数据集(dataset)分为训练集(training set)和测试集(testing set)两个子集，训练集用以建立模型，测试集用来评估该模型对未知样本进行预测时的精确度，即泛化能力(generalization ability)。交叉验证(Cross Validation)是评估特征选择、降维、以及学习算法的不同组合所建立的模型性能最有用的方法之一，基本思想是将原始数据(dataset)进行分组，一部分做为训练集，另一部分做为测试集。首先用训练集进行模型训练，在利用测试集来测试训练得到的模型，以此来做为评价模型预测性能的指标。交叉验证有许多种，常见的交叉验证方法是k折交叉验证(K-fold Cross Validation，K-CV)，即将原始数据均分成K组，将每个子集数据分别做一次测试集，其余的K-1组子集数据作为训练集，这样会得到K个模型，用这K个模型最终的验证集的结果作为K-CV下模型的性能指标。常用的K折交叉验证方法有10折交叉验证(10-CV)和5折交叉验证(5-CV)。K-CV可以有效的避免过拟合 以及欠学习状态的发生，最后得到的结果也比较具有说服性。
对于本申请实施例中的不同疲劳程度状态下的动作行为数据具体的采集方案可以是：
Figure PCTCN2020118353-appb-000001
与现有技术相比，本申请实施例提供的疲劳状态识别方法具有以下优点：
1)对疲劳状态的自动识别，无需电网工作人员自我报告疲劳状态，时效性高；
2)通过视频采集数据，对电网工作人员没有任何干扰，能够更自然地实现对行为数据的记录和预测，生态效度高；
3)将疲劳状态的自动识别模型与视频行为数据的采集相结合，能够做到实时的疲劳状态识别和及时跟踪；
4)由于目前视频摄像头应用广泛，能够方便测量在不同场景下电网工作人员的疲劳状态；
5)识别效率提高；
6)满足各种工作场景下对人员疲劳状态的监测要求。
为了便于理解，请参阅图3，本申请中提供一种疲劳状态识别装置，包括：
图片获取模块301，用于获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片。
预处理模块302，用于对实时行为图片进行预处理，得到实时关节点运动数据。
输入模块303，用于将实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态。
输出模块304，用于输出预置疲劳状态预测模型的预测结果，得到待识别疲劳状态人员的疲劳状态。
作为改进，还可以包括：
样本模块305，用于获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
训练模块306，用于基于运动数据样本集和对应关系，对建立好的行为数据疲劳状态识别模型进行训练，将训练好的行为数据疲劳状态识别模型作为预置疲劳状态预测模型。
还可以包括：
诱发模块307，用于基于预置疲劳诱发方式诱发关节点运动数据样本提供人员产生不同程度的疲劳状态。
数据获取模块308，用于获取关节点运动数据样本提供人员在预置行走区域内正常行走状态的关节点运动数据。
心理特征获取模块309，用于获取关节点运动数据样本提供人员在规定时间内完成的心理特征量表。
关系建立模块310，用于基于正常行走状态的关节点运动数据和心理特征量表，建立关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
本申请中提供了一种疲劳状态识别设备的实施例，所述设备包括处理器以及存储器：
存储器用于存储程序代码，并将程序代码传输给处理器；
处理器用于根据程序代码中的指令执行前述的疲劳状态识别方法实施例中的基于疲劳状态识别方法。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(英文全称：Read-Only Memory，英文缩写：ROM)、随机存取存储器(英文全称：Random Access Memory，英文缩写：RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照 前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (10)

  1. 一种疲劳状态识别方法，其特征在于，包括：
    获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片；
    对所述实时行为图片进行预处理，得到实时关节点运动数据；
    将所述实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，所述预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态；
    输出所述预置疲劳状态预测模型的预测结果，得到所述待识别疲劳状态人员的疲劳状态。
  2. 根据权利要求1所述的疲劳状态识别方法，其特征在于，所述将所述实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，之前还包括：
    获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系；
    基于所述运动数据样本集和所述对应关系，对建立好的行为数据疲劳状态识别模型进行训练，将训练好的所述行为数据疲劳状态识别模型作为所述预置疲劳状态预测模型。
  3. 根据权利要求2所述的疲劳状态识别方法，其特征在于，所述行为数据疲劳状态识别模型为SVM分类模型。
  4. 根据权利要求2所述的疲劳状态识别方法，其特征在于，所述获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系，之前还包括：
    基于预置疲劳诱发方式诱发关节点运动数据样本提供人员产生不同程度的疲劳状态；
    获取所述关节点运动数据样本提供人员在预置行走区域内正常行走状态的关节点运动数据；
    获取所述关节点运动数据样本提供人员在规定时间内完成的心理特征量表；
    基于所述正常行走状态的关节点运动数据和所述心理特征量表，建立 关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
  5. 根据权利要求4所述的疲劳状态识别方法，其特征在于，所述心理特征量表包括：匹兹堡睡眠质量指数量表、多维疲劳量表和/或模拟视觉量表。
  6. 根据权利要求4所述的疲劳状态识别方法，其特征在于，所述预置疲劳诱发方式包括执行CPT任务和/或Filtering任务。
  7. 一种疲劳状态识别装置，其特征在于，包括：
    图片获取模块，用于获取摄像装置拍摄到的待识别疲劳状态人员的实时行为图片；
    预处理模块，用于对所述实时行为图片进行预处理，得到实时关节点运动数据；
    输入模块，用于将所述实时关节点运动数据输入到预置疲劳状态预测模型进行疲劳状态预测，所述预置疲劳状态预测模型的自变量为关节点运动数据，因变量为疲劳状态；
    输出模块，用于输出所述预置疲劳状态预测模型的预测结果，得到所述待识别疲劳状态人员的疲劳状态。
  8. 根据权利要求7所述的疲劳状态识别装置，其特征在于，还包括：
    样本模块，用于获取关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系；
    训练模块，用于基于所述运动数据样本集和所述对应关系，对建立好的行为数据疲劳状态识别模型进行训练，将训练好的所述行为数据疲劳状态识别模型作为所述预置疲劳状态预测模型。
  9. 根据权利要求8所述的疲劳状态识别装置，其特征在于，还包括：
    诱发模块，用于基于预置疲劳诱发方式诱发关节点运动数据样本提供人员产生不同程度的疲劳状态；
    数据获取模块，用于获取所述关节点运动数据样本提供人员在预置行走区域内正常行走状态的关节点运动数据；
    心理特征获取模块，用于获取所述关节点运动数据样本提供人员在规定时间内完成的心理特征量表；
    关系建立模块，用于基于所述正常行走状态的关节点运动数据和所述心理特征量表，建立关节点运动数据样本集和各关节点运动数据样本与疲劳状态的对应关系。
  10. 一种疲劳状态识别设备，其特征在于，所述设备包括处理器以及存储器：
    所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
    所述处理器用于根据所述程序代码中的指令执行权利要求1-6任一项所述的疲劳状态识别方法。
PCT/CN2020/118353 2019-10-12 2020-09-28 一种疲劳状态识别方法、装置和设备 WO2021068781A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910968637.2A CN110717461A (zh) 2019-10-12 2019-10-12 一种疲劳状态识别方法、装置和设备
CN201910968637.2 2019-10-12

Publications (1)

Publication Number Publication Date
WO2021068781A1 true WO2021068781A1 (zh) 2021-04-15

Family

ID=69212587

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/118353 WO2021068781A1 (zh) 2019-10-12 2020-09-28 一种疲劳状态识别方法、装置和设备

Country Status (2)

Country Link
CN (1) CN110717461A (zh)
WO (1) WO2021068781A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344095A (zh) * 2021-06-21 2021-09-03 北京惠朗时代科技有限公司 一种基于多特征算子的K-means工人疲劳简易判别方法
CN113591937A (zh) * 2021-07-09 2021-11-02 国家电网有限公司 一种基于局部距离编码的电力系统关键节点识别方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110717461A (zh) * 2019-10-12 2020-01-21 广东电网有限责任公司 一种疲劳状态识别方法、装置和设备
CN111476114A (zh) * 2020-03-20 2020-07-31 深圳追一科技有限公司 疲劳检测方法、装置、终端设备及存储介质
CN112001325A (zh) * 2020-08-25 2020-11-27 广东电网有限责任公司电力科学研究院 一种配电不安全行为的预测方法、装置和服务器

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107212890A (zh) * 2017-05-27 2017-09-29 中南大学 一种基于步态信息的运动识别和疲劳检测方法及系统
US20190019343A1 (en) * 2013-03-04 2019-01-17 Alex C. Chen Method and Apparatus for Recognizing Behavior and Providing Information
CN109477951A (zh) * 2016-08-02 2019-03-15 阿特拉斯5D公司 在保护隐私的同时识别人及/或识别并量化疼痛、疲劳、情绪及意图的系统及方法
CN110147738A (zh) * 2019-04-29 2019-08-20 中国人民解放军海军特色医学中心 一种驾驶员疲劳监测预警方法及系统
CN110717461A (zh) * 2019-10-12 2020-01-21 广东电网有限责任公司 一种疲劳状态识别方法、装置和设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104978495A (zh) * 2015-07-20 2015-10-14 西安科技大学 一种基于脑电检测的矿工脑力负荷评价方法
CN107788970B (zh) * 2017-11-15 2020-08-07 中国航天员科研训练中心 一种基于多生理参数方法融合的脑力负荷评估方法
CN108830215B (zh) * 2018-06-14 2021-07-13 南京理工大学 基于人员骨架信息的危险行为识别方法
CN109919089A (zh) * 2019-03-06 2019-06-21 百度在线网络技术（北京）有限公司 用于推送信息的方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190019343A1 (en) * 2013-03-04 2019-01-17 Alex C. Chen Method and Apparatus for Recognizing Behavior and Providing Information
CN109477951A (zh) * 2016-08-02 2019-03-15 阿特拉斯5D公司 在保护隐私的同时识别人及/或识别并量化疼痛、疲劳、情绪及意图的系统及方法
CN107212890A (zh) * 2017-05-27 2017-09-29 中南大学 一种基于步态信息的运动识别和疲劳检测方法及系统
CN110147738A (zh) * 2019-04-29 2019-08-20 中国人民解放军海军特色医学中心 一种驾驶员疲劳监测预警方法及系统
CN110717461A (zh) * 2019-10-12 2020-01-21 广东电网有限责任公司 一种疲劳状态识别方法、装置和设备

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344095A (zh) * 2021-06-21 2021-09-03 北京惠朗时代科技有限公司 一种基于多特征算子的K-means工人疲劳简易判别方法
CN113344095B (zh) * 2021-06-21 2024-03-29 北京惠朗时代科技有限公司 一种基于多特征算子的K-means工人疲劳简易判别方法
CN113591937A (zh) * 2021-07-09 2021-11-02 国家电网有限公司 一种基于局部距离编码的电力系统关键节点识别方法
CN113591937B (zh) * 2021-07-09 2023-09-26 国家电网有限公司 一种基于局部距离编码的电力系统关键节点识别方法

Also Published As

Publication number Publication date
CN110717461A (zh) 2020-01-21

Similar Documents

Publication Publication Date Title
WO2021068781A1 (zh) 一种疲劳状态识别方法、装置和设备
Li et al. A novel vision-based real-time method for evaluating postural risk factors associated with musculoskeletal disorders
Vankipuram et al. Toward automated workflow analysis and visualization in clinical environments
US9239956B2 (en) Method and apparatus for coding of eye and eye movement data
US9443144B2 (en) Methods and systems for measuring group behavior
WO2021068783A1 (zh) 一种情绪识别方法、装置和设备
Pang et al. Automatic detection and quantification of hand movements toward development of an objective assessment of tremor and bradykinesia in Parkinson's disease
KR20200005987A (ko) 터치입력을 이용한 인지장애 진단 시스템 및 방법
Chaaraoui et al. Abnormal gait detection with RGB-D devices using joint motion history features
CN108197592B (zh) 信息获取方法和装置
JP5225870B2 (ja) 情動分析装置
Liu Objects detection toward complicated high remote basketball sports by leveraging deep CNN architecture
JP2022547595A (ja) 顔属性認識方法、装置、電子機器及び記憶媒体
Ahn et al. A digital twin city model for age-friendly communities: Capturing environmental distress from multimodal sensory data
Sengan et al. Cost-effective and efficient 3D human model creation and re-identification application for human digital twins
Creagh et al. Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones
Gündüz et al. A new YOLO-based method for real-time crowd detection from video and performance analysis of YOLO models
Thaman et al. Face mask detection using mediapipe facemesh
CN114758787A (zh) 区域疫情信息处理方法、装置和系统
JP7306152B2 (ja) 感情推定装置、感情推定方法、プログラム、情報提示装置、情報提示方法及び感情推定システム
Abdullah et al. Multiple linear regression and deep learning in body temperature detection and mask detection
CN116543455A (zh) 建立帕金森症步态受损评估模型、使用方法、设备及介质
Yashaswini et al. Stress detection using deep learning and IoT
Zhang et al. An approach of region of interest detection based on visual attention and gaze tracking
Ji et al. Influence of embedded microprocessor wireless communication and computer vision in Wushu competition referees’ decision support

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20873579

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20873579

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 18.10.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20873579

Country of ref document: EP

Kind code of ref document: A1