CN118062021A - Intelligent control method and system for incapacitation of driving operator of civil aviation vehicle equipment - Google Patents
Intelligent control method and system for incapacitation of driving operator of civil aviation vehicle equipment Download PDFInfo
- Publication number
- CN118062021A CN118062021A CN202410188178.7A CN202410188178A CN118062021A CN 118062021 A CN118062021 A CN 118062021A CN 202410188178 A CN202410188178 A CN 202410188178A CN 118062021 A CN118062021 A CN 118062021A
- Authority
- CN
- China
- Prior art keywords
- driver
- state
- driving
- safety
- expression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000009295 sperm incapacitation Effects 0.000 title claims abstract description 10
- 230000014509 gene expression Effects 0.000 claims abstract description 36
- 230000008921 facial expression Effects 0.000 claims abstract description 30
- 230000002159 abnormal effect Effects 0.000 claims abstract description 17
- 230000001815 facial effect Effects 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims abstract description 8
- 230000006399 behavior Effects 0.000 claims description 18
- 238000012544 monitoring process Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 9
- 230000003044 adaptive effect Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000005259 measurement Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012706 support-vector machine Methods 0.000 claims description 4
- 230000036541 health Effects 0.000 claims description 3
- 230000007774 longterm Effects 0.000 claims description 3
- 230000003449 preventive effect Effects 0.000 claims description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000011835 investigation Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008560 physiological behavior Effects 0.000 description 1
- 230000004962 physiological condition Effects 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
Landscapes
- Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
Abstract
The invention relates to the technical field of incapacitation safety intelligent control, and discloses a method and a system for controlling incapacitation safety intelligent of a driver of civil aviation vehicle equipment, wherein the method comprises the steps of respectively capturing facial expression and heart rate parameters of the driver according to a camera and wearable equipment; comparing the heart rate parameter with a preset threshold value, and judging whether the driver is in a normal working state or not by combining the facial expression; if the state of the driving operator is abnormal, the driving operator is judged and evaluated for a plurality of times, and if the abnormal state exists, the main system automatically takes over the operation. The facial expression and heart rate parameters of the driver are monitored, the face position is accurately detected, facial feature points are extracted, the expression state of the driver is effectively judged, and the accuracy is improved. And comparing the heart rate parameter with a preset threshold value, and reducing the accident risk by combining the facial expression state interval. According to different driver states, the automatic intervention is performed, emergency stop and help seeking functions are provided, and the safety of the driver and passengers is further ensured.
Description
Technical Field
The invention relates to the technical field of incapacitation safety intelligent control, in particular to a method and a system for incapacitation safety intelligent control of a driving operator of civil aviation vehicle equipment.
Background
With the popularization of vehicles at present, the mobility of civil aviation driving operators is increased, the working intensity is high, the frequency of telephone or interphone receiving is high, and the like, and the harm of driving is increased due to the phenomena of distraction, attention reduction and the like in the driving operation process.
Disclosure of Invention
The present invention has been made in view of the above-mentioned problems occurring in the prior art.
Therefore, the invention provides the intelligent control method for incapacitation safety of the driving operator of the civil aviation vehicle equipment, which integrates different data sources including images, heart rate, holding power and the like, and can more comprehensively evaluate the states of the driver including emotion states, physiological states and behavior states by comprehensively analyzing the data.
In order to solve the technical problems, the invention provides the following technical scheme, namely a method for controlling incapacitation safety and intelligence of a driving operator of civil aviation vehicle equipment, which comprises the following steps:
capturing facial expression and heart rate parameters of a driver according to a camera and wearable equipment respectively;
comparing the heart rate parameter with a preset threshold value, and judging whether the driver is in a normal working state or not by combining the facial expression;
If the state of the driving operator is abnormal, the driving operator is judged and evaluated for a plurality of times, and if the abnormal state exists, the main system automatically takes over the operation, and the state and behavior data of the driving operator are recorded.
As a preferable scheme of the intelligent control method for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the facial expression includes the expression of the face,
Face detection is performed using deep learning, stepwise accurate detection of face positions is expressed as,
Y＝f(X；W)
Wherein X is an input image, W is a model parameter, Y is a detected face position, and f is a face position extracted from the input image X;
Facial feature point extraction was performed using the 68-point model in the Dlib library,
P＝h(Y；V)
Wherein P is a facial feature point, V is a model parameter, and h is a facial feature point extracted from a face position Y;
facial expression features are extracted by calculating euclidean distances and angles between feature points,
F＝d(P)
Where F is the calculated feature value and d is a feature related to the facial expression calculated from the extracted facial feature points P;
the expression classification is performed using a support vector machine,
E＝s(F；U)
Where U is a model parameter, E is an expression state interval, and s is a map for mapping the feature F to the expression state interval.
As a preferable scheme of the intelligent control method for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the comparison of the heart rate parameter with the preset threshold value comprises the steps of judging according to the expression state interval when H is less than or equal to H low and the duration exceeds T d, judging the state of the driver to be A1 if the expression state interval is a first interval and the duration exceeds T d, and judging the state of the driver to be A2 if the expression state interval is a second interval and the duration of H is less than or equal to H low does not exceed T d;
When H is more than or equal to H high and the duration exceeds T d, judging according to the expression state interval, if the expression state interval is a first interval and the duration exceeds T d, judging the driver state as A1, and if the expression state interval is a second interval and the duration of H is more than or equal to H high does not exceed T d, judging the driver state as A2;
Wherein H low is a low heart rate threshold, H high is a high heart rate threshold, H is a driver heart rate, and T d is safe time.
As a preferable scheme of the intelligent control method for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the step of carrying out the driver state judgment and evaluation on the driving operator comprises the steps of carrying out third evaluation on the driving operator by detecting contact of the steering wheel if WHR eye>HHReyef and the duration exceeds T d when the physical state of the driving operator is A1, and carrying out third evaluation on the driving operator by detecting contact of the steering wheel if WHR mouth>HHRmouthf and the duration exceeds T d;
The third evaluation of the driver by detecting the contact of the steering wheel comprises that when F (t) < F a (t), the system starts an emergency alarm, sends out an audio or visual warning signal, the main system requires the intervention of a decision module, decelerates and guides the vehicle to a parking state safely, the emergency communication module is automatically connected to a preset emergency contact network, sends out a help signal and provides vehicle position information, and the warning module is activated;
When the physical state of the driving operator is A2, the contact surface of the steering wheel is detected to evaluate the driver, if the driving operator F (t) > F a (t) is used, the system performs conventional vehicle monitoring and driver state monitoring, and if the driving operator F (t) < F a (t) is used, the holding power is recovered within a specified safety time and is larger than the self-adaptive holding power threshold value, the state of the driver is judged to be B1;
If the driver F (t) < F a (t), when the grip strength is recovered and is greater than the adaptive grip strength threshold value in the safety time and less than the dangerous time, judging that the driver state is B2;
wherein F (t) is the holding strength of the driver at the time of t, and F is the holding strength.
As a preferable scheme of the intelligent control method for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the automatic takeover operation of the main system comprises the steps that when the system detects that the state of a driver is A1 or the driving state score is Q, the system immediately intervenes in taking over the vehicle, an emergency mode system is started, the driving assistance level is adaptively adjusted to be primary driving assistance according to the performance of the driver, a safety signal lamp is automatically decelerated and started, the vehicle is automatically guided to a safe parking area according to the actual road condition, meanwhile, internal and external safety protocols are activated, emergency medical services are notified, and the system records events and driver state data;
When the system detects that the state of the driver is A2 or the score of the driving state is W, the system enhances the driving assistance function, the system adaptively adjusts the driving assistance level to be secondary driving assistance according to the performance of the driver, automatically keeps a lane, adjusts the speed of the vehicle and keeps a safe distance, and the warning system gives visual and audible warnings to the driver to remind the driver of abnormal driving and strengthen the detection of the state of the driver so as to take corresponding measures when the state of the driver changes, and the system records the state and behavior data of the driver in real time so as to perform long-term analysis and future preventive measures;
When the system detects that the state of the driver is B1 or the driving state score is E, the system enhances the monitoring of the driver, the system adaptively adjusts the driving assistance level to three-level driving assistance according to the performance of the driver, captures the behavior change which causes safety risk in real time, provides health and safety prompts through a user interface, automatically adjusts the vehicle setting according to the driving environment, adjusts the vehicle speed and the vehicle distance, ensures that the safety can be still maintained when the response capability of the driver is reduced, records the behavior data in the current state, and feeds back to the driver through the user interface;
When the system detects that the driver state is B2 or the driving state score R, the system provides light driving assistance, the system adaptively adjusts the driving assistance level to be four according to the performance of the driver, and state reminding is provided for the driver regularly.
As a preferable scheme of the intelligent control method for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the evaluating the driver by detecting contact of the steering wheel includes,
Fa(t)＝Fb-k·σn(t)
Wherein F b is the baseline force, k is the adjustment coefficient, F a (t) is the adaptive threshold, σ n (t) is the standard deviation calculated based on the last n measurements at time t, used to update the adaptive threshold, n is the number of the last measurements to calculate the current standard deviation.
As a preferable scheme of the intelligent control method for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the driving state score comprises Q is more than or equal to 80; w is more than or equal to 60 and less than 79; e is more than or equal to 40 and less than 59; r < 39.
Another object of the present invention is to provide a safety intelligent control system for disabled drivers of civil aviation vehicle devices, which can discover fatigue, distraction or abnormal states in time by monitoring physiological conditions and expressions of drivers, thereby reducing risks of traffic accidents. The system can automatically cope with abnormal situations, and different measures are taken according to the state of a driver, including emergency stopping, so as to ensure the safety of the vehicle and passengers. The system monitors the state of the driver in real time, does not need to rely on the driver to perceive the problem, is favorable for taking intervention measures in advance, and reduces the occurrence of accidents. The system also records and analyzes driver status and behavior data, which can be used for performance improvement and accident investigation. The system can adjust the weight of the intervention according to the state of the driver to balance the safety and the comfort of the driver, and reduce false alarms and unnecessary interventions.
As a preferable scheme of the intelligent control system for disabled safety of the driving operator of the civil aviation vehicle equipment, the invention comprises the following steps: the system comprises a monitoring module, a control module, a judging module, a decision module, an executing module and a data recording module;
The monitoring module, the camera and the wearable equipment are used for detecting the contact surface of the steering wheel of a driver;
the control module is used for analyzing facial expressions and comparing heart rate parameters;
the judging module is used for judging the state of the driver according to the expression state interval;
the decision module takes different measures for different driver states;
the execution module is used for performing intervention control if the state of the driver is abnormal;
the execution data recording module records the state and behavior data of the driver.
A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any one of the safety intelligent control methods of disabled driving operators of a civil aviation vehicle device.
A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the method of any one of the civil aviation vehicle apparatus operator disabling safety intelligent control methods.
The invention has the beneficial effects that: according to the method, the facial expression and heart rate parameters of the driver are monitored by combining the camera and the wearable equipment, so that the physical state of the driver can be accurately estimated. The computing system can accurately detect the face position and extract the facial feature points, so that the expression state of the driver can be effectively judged, and the accuracy is further improved. And comparing the heart rate parameter with a preset threshold value, and combining the facial expression state interval to judge whether the driver is in a normal working state, so that the accident risk is reduced. According to different driver states, the automatic intervention is performed, emergency stop and help seeking functions are provided, and the safety of the driver and passengers is further ensured. The status and behavior data of the driver are recorded, contributing to performance improvement and accident investigation. The weight of the intervention can be adjusted to balance safety and driver comfort, providing a higher level of driving experience.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a method for controlling incapacitation safety and intelligence of a driving operator of a civil aviation vehicle device according to an embodiment of the invention.
Fig. 2 is a schematic flow chart of a civil aviation vehicle apparatus driver disabling safety intelligent control system according to an embodiment of the invention.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
Example 1
Referring to fig. 1, a first embodiment of the present invention provides a method for controlling disabled safety and intelligence of a driving operator of a civil aviation vehicle apparatus, comprising:
S1: capturing facial expression and heart rate parameters of a driver according to a camera and wearable equipment respectively;
It should be noted that the facial expressions include, face detection using deep learning, stepwise accurate detection of the face position is expressed as,
Y＝f(X；W)
Wherein X is an input image, W is a model parameter, Y is a detected face position, and f is a face position extracted from the input image X;
Facial feature point extraction was performed using the 68-point model in the Dlib library,
P＝h(Y；V)
Wherein P is a facial feature point, V is a model parameter, and h is a facial feature point extracted from a face position Y;
facial expression features are extracted by calculating euclidean distances and angles between feature points,
F＝d(P)
Where F is the calculated feature value and d is a feature related to the facial expression calculated from the extracted facial feature points P;
the expression classification is performed using a support vector machine,
E＝s(F；U)
Where U is a model parameter, E is an expression state interval, and s is a map for mapping the feature F to the expression state interval.
S2: comparing the heart rate parameter with a preset threshold value, and judging whether the driver is in a normal working state or not by combining the facial expression;
It should be noted that the comparing the heart rate parameter with the preset threshold includes, when H is equal to or less than H low and the duration exceeds T d, determining according to the expression state interval, if the expression state interval is a first interval and the duration exceeds T d, determining the driver state as A1, and if the expression state interval is a second interval and H is equal to or less than H low and the duration does not exceed T d, determining the driver state as A2;
When H is more than or equal to H high and the duration exceeds T d, judging according to the expression state interval, if the expression state interval is a first interval and the duration exceeds T d, judging the driver state as A1, and if the expression state interval is a second interval and the duration of H is more than or equal to H high does not exceed T d, judging the driver state as A2;
Wherein H low is a low heart rate threshold, H high is a high heart rate threshold, H is a driver heart rate, and T d is safe time.
S3: if the state of the driving operator is abnormal, the driving operator is judged and evaluated for a plurality of times, and if the abnormal state exists, the main system automatically takes over the operation, and the state and behavior data of the driving operator are recorded.
It should be noted that, the step of performing the driver state judgment and evaluation on the driving operator includes, when the physical state of the driving operator is A1, performing a third evaluation on the driver by detecting the contact surface of the steering wheel if WHR eye>HHReye f and the duration exceeds T d, and performing a third evaluation on the driver by detecting the contact surface of the steering wheel if WHR mouth>HHRmouth f and the duration exceeds T d;
The third evaluation of the driver by detecting the contact of the steering wheel comprises that when F (t) < F a (t), the system starts an emergency alarm, sends out an audio or visual warning signal, the main system requires the intervention of a decision module, decelerates and guides the vehicle to a parking state safely, the emergency communication module is automatically connected to a preset emergency contact network, sends out a help signal and provides vehicle position information, and the warning module is activated;
When the physical state of the driving operator is A2, the contact surface of the steering wheel is detected to evaluate the driver, if the driving operator F (t) > F a (t) is used, the system performs conventional vehicle monitoring and driver state monitoring, and if the driving operator F (t) < F a (t) is used, the holding power is recovered within a specified safety time and is larger than the self-adaptive holding power threshold value, the state of the driver is judged to be B1;
If the driver F (t) < F a (t), when the grip strength is recovered and is greater than the adaptive grip strength threshold value in the safety time and less than the dangerous time, judging that the driver state is B2;
wherein F (t) is the holding strength of the driver at the time of t, and F is the holding strength.
It should be noted that, the automatic take-over operation of the main system includes, when the system detects that the driver status is A1 or the driving status score is Q, immediately intervening in taking over the vehicle, starting the emergency mode system, adaptively adjusting the driving assistance level to be primary driving assistance according to the performance of the driver, automatically decelerating and starting the safety signal lamp, automatically guiding the vehicle to the safe parking area according to the actual road condition, simultaneously activating the internal and external safety protocols, notifying the emergency medical service, and implementing recording event and driver status data by the system;
When the system detects that the state of the driver is A2 or the score of the driving state is W, the system enhances the driving assistance function, the system adaptively adjusts the driving assistance level to be secondary driving assistance according to the performance of the driver, automatically keeps a lane, adjusts the speed of the vehicle and keeps a safe distance, and the warning system gives visual and audible warnings to the driver to remind the driver of abnormal driving and strengthen the detection of the state of the driver so as to take corresponding measures when the state of the driver changes, and the system records the state and behavior data of the driver in real time so as to perform long-term analysis and future preventive measures;
When the system detects that the state of the driver is B1 or the driving state score is E, the system enhances the monitoring of the driver, the system adaptively adjusts the driving assistance level to three-level driving assistance according to the performance of the driver, captures the behavior change which causes safety risk in real time, provides health and safety prompts through a user interface, automatically adjusts the vehicle setting according to the driving environment, adjusts the vehicle speed and the vehicle distance, ensures that the safety can be still maintained when the response capability of the driver is reduced, records the behavior data in the current state, and feeds back to the driver through the user interface;
When the system detects that the driver state is B2 or the driving state score R, the system provides light driving assistance, the system adaptively adjusts the driving assistance level to be four according to the performance of the driver, and state reminding is provided for the driver regularly.
The driving state score comprises Q is more than or equal to 80; w is more than or equal to 60 and less than 79; e is more than or equal to 40 and less than 59; r < 39.
It should be noted that the evaluation of the driver's face by detecting the contact of the steering wheel includes,
Fa(t)＝Fb-k·σn(t)
Wherein F b is the baseline force, k is the adjustment coefficient, F a (t) is the adaptive threshold, σ n (t) is the standard deviation calculated based on the last n measurements at time t, used to update the adaptive threshold, n is the number of the last measurements to calculate the current standard deviation.
It should be noted that the recorded status and behavior data of the driving operator include,
B＝{(H1,E1,F1,S1,T1),…,(Hi,Ei,Fi,Si,Ti)}
Wherein H i represents the ith acquired heart rate data, E i represents the ith acquired facial expression data, F i represents the ith acquired holding power data, S i represents the ith acquired state judgment data, and T i represents the ith acquired time stamp.
Example 2
For the second embodiment of the invention, a safe and intelligent control method for incapacitating a driver of a civil aviation vehicle device is provided, and scientific demonstration is carried out through experiments in order to verify the beneficial effects of the invention.
Model performance under different thresholds is compared. The accuracy, response time, false positive rate and false negative rate under different thresholds are shown by using a table form. As shown in table 1.
Table 1, data alignment table example
The my threshold setting reaches 92% above the other two sets of alternative thresholds. This means that the system is able to more accurately identify the status of the driver, thereby providing more timely assistance. Although the response time of the alternative threshold setting A is slightly shorter, the my threshold maintains a reasonable response time (1.2 seconds) while maintaining a higher accuracy, ensuring timely system response. The My threshold setting works well in both false positive and false negative rates, balancing the risk of excessive and insufficient intervention. In contrast, the false positive rate of alternative threshold setting a is higher, while the false negative rate of setting B is higher.
The setting of the My threshold is excellent in four key performance indexes of accuracy, response time, false alarm rate and false alarm rate, and the optimal balance of performance is realized. The high accuracy and reasonable response time ensure that the system can timely and accurately respond to the change of the state of the driver, and the safety risk is reduced. Meanwhile, the relatively low false alarm rate and false alarm rate reduce unnecessary interference to a driver, and the practicability of the system and the acceptance of the driver are improved.
In summary, the threshold combination (Q is greater than or equal to 80, W:60-79, E:40-59, R < 40) selected by the my effectively balances the false alarm rate and the false alarm rate while ensuring the accuracy and the responsiveness of the system, and provides the best comprehensive performance. The system is more reliable and effective in practical application, and can adapt to the requirements of different drivers, so that the overall driving safety and comfort are improved.
Example 3
A third embodiment of the present invention, which is different from the first two embodiments, is:
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Example 4
Referring to fig. 2, a fourth embodiment of the present invention provides a civil aviation vehicle apparatus driving operator disabling safety intelligent control system, which is characterized in that: the system comprises a monitoring module, a target detection module, a control module, a judging module, a decision module and an executing module;
The monitoring module, the camera and the wearable equipment are used for detecting the contact surface of the steering wheel of a driver;
a camera head: the facial expression capturing device is used for capturing the facial expression of a driver and carrying out face detection and facial feature point extraction.
Wearable device: the heart rate parameter of the driver is monitored in real time.
Steering operator steering wheel contact surface detection: the contact surface of the steering wheel held by the driver is detected through infrared induction on the steering wheel.
The control module analyzes facial expression: face detection is performed using deep learning, face positions are extracted, and then facial feature points are extracted using a 68-point model. Facial expression features are extracted by calculating euclidean distances and angles between feature points. Heart rate parameter comparison: and comparing the heart rate parameter monitored in real time with a preset threshold value to determine whether the heart rate parameter is in a normal working state.
And the judging module is used for carrying out expression classification by using a support vector machine and mapping the facial expression characteristics to the expression state interval. And judging the state of the driver according to the expression state interval.
The decision module, for different driver states (A1 or A2), the system will take different measures. If the driver's status is abnormal, an emergency alert will be initiated, an audible or visual warning signal will be issued, and a decision module intervention will be required. The decision module may reduce or increase the safety and comfort weights based on the driver status to determine the level of intervention of the system in the driving process.
The execution module is used for controlling the automatic steering and braking of the vehicle and guiding the vehicle to a parking state safely if the state of the driver is abnormal. Meanwhile, the emergency communication module is automatically connected to a preset emergency contact network, sends out a help signal and provides vehicle position information.
The execution data recording module records state and behavior data of a driver, including heart rate data, facial expression data, grip strength data, state judgment data and a time stamp. Such data may be used for subsequent analysis, improvement, and accident investigation.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered by the scope of the claims of the present invention.
Claims (10)
1. The intelligent control method for incapacitation of the driving operator of the civil aviation vehicle equipment is characterized by comprising the following steps of: comprising the steps of (a) a step of,
Detecting facial expression and heart rate parameters of a driver according to the camera and the wearable equipment respectively;
comparing the detected heart rate parameter with a preset threshold value, and predicting whether the driver is in a normal working state or not by combining the detected facial expression;
If the predicted result shows that the state of the driving operator is abnormal, the driving operator is judged and evaluated for multiple times, and if the abnormal state exists, the main system automatically takes over the operation, and the state and behavior data of the driving operator are recorded.
2. The intelligent control method for disabled driving operators of civil aviation vehicle equipment according to claim 1, characterized in that: the facial expression includes, face detection using deep learning, stepwise accurate detection of face positions expressed as,
Y＝f(X；W)
Wherein X is an input image, W is a model parameter, Y is a detected face position, and f is a face position extracted from the input image X;
Facial feature point extraction was performed using the 68-point model in the Dlib library,
P＝h(Y；V)
Wherein P is a facial feature point, V is a model parameter, and h is a facial feature point extracted from a face position Y;
facial expression features are extracted by calculating euclidean distances and angles between feature points,
F＝d(P)
Where F is the calculated feature value and d is a feature related to the facial expression calculated from the extracted facial feature points P;
the expression classification is performed using a support vector machine,
E＝s(F；U)
Where U is a model parameter, E is an expression state interval, and s is a map for mapping the feature F to the expression state interval.
3. The intelligent control method for disabled operation safety of a driving operator of a civil aviation vehicle apparatus according to claim 2, wherein: the comparison of the detected heart rate parameter with the preset threshold value comprises the steps of judging according to the expression state interval when H is smaller than or equal to H low and the duration exceeds T d, judging the driver state as A1 and recording the driver state as red if the expression state interval is a first interval and the duration exceeds T d, and judging the driver state as A2 if the expression state interval is a second interval and the duration of H is smaller than or equal to H low does not exceed T d;
When H is more than or equal to H high and the duration exceeds T d, judging according to the expression state interval, if the expression state interval is a first interval and the duration exceeds T d, judging the driver state as A1, and if the expression state interval is a second interval and the duration of H is more than or equal to H high does not exceed T d, judging the driver state as A2 and recording the driver state as a green state;
Wherein H low is a low heart rate threshold, H high is a high heart rate threshold, H is a driver heart rate, and T d is safe time.
4. A civil aviation vehicle apparatus driver disablement safety intelligent control method as claimed in claim 3, wherein: the multiple judging and evaluating the driving operator comprises the steps of performing third evaluating on the driving operator by detecting contact surface of the steering wheel if WHR eye>HHReye f and duration exceeds T d when the physical state of the driving operator is A1, and performing third evaluating on the driving operator by detecting contact surface of the steering wheel if WHR mouth>HHRmouth f and duration exceeds T d;
The third evaluation of the driver by detecting the contact of the steering wheel comprises that when F (t) < F a (t), the system starts an emergency alarm, sends out an audio or visual warning signal, the main system requires the intervention of a decision module, decelerates and guides the vehicle to a parking state safely, and the emergency communication module automatically connects to a preset emergency contact network and activates a warning module, sends out a help seeking signal and provides vehicle position information;
When the physical state of the driving operator is A2, the contact surface of the steering wheel is detected to evaluate the driver, if the driving operator F (t) > F a (t) is used, the system performs conventional vehicle monitoring and driver state monitoring, and if the driving operator F (t) < F a (t) is used, the holding power is recovered within a specified safety time and is larger than the self-adaptive holding power threshold value, the driver state is judged to be B1 and is recorded as a yellow state;
If the driver F (t) < F a (t), when the holding power is recovered and is greater than the self-adaptive holding power threshold value in the time period of being greater than the specified safety time period and less than the dangerous time period, judging the driver state as B2 and recording as a yellow state;
Where F (t) is the gripping strength of the driver at time t, F is the gripping strength, WHR eye is the eye aspect ratio, HHR eye f is the prescribed eye aspect ratio, WHR mouth is the mouth aspect ratio, and HHR mouth f is the prescribed mouth aspect ratio.
5. The intelligent control method for disabled operation safety of a civil aviation vehicle apparatus driver as claimed in claim 4, wherein: the automatic takeover operation of the main system comprises the steps that when the system detects that the state of a driver is A1 or the driving state score is Q, the system immediately intervenes in taking over the vehicle, an emergency mode system is started, the driving assistance level is adaptively adjusted to be primary driving assistance according to the performance of the driver, a safety signal lamp is automatically decelerated and started, the vehicle is automatically guided to a safe parking area according to the actual road condition, meanwhile, internal and external safety protocols are activated, emergency medical services are notified, and the system records events and driver state data;
When the system detects that the state of the driver is A2 or the score of the driving state is W, the system enhances the driving assistance function, the system adaptively adjusts the driving assistance level to be secondary driving assistance according to the performance of the driver, automatically keeps a lane, adjusts the speed of the vehicle and keeps a safe distance, and the warning system gives visual and audible warnings to the driver to remind the driver of abnormal driving and strengthen the detection of the state of the driver so as to take corresponding measures when the state of the driver changes, and the system records the state and behavior data of the driver in real time so as to perform long-term analysis and future preventive measures;
When the system detects that the state of the driver is B1 or the driving state score is E, the system enhances the monitoring of the driver, the system adaptively adjusts the driving assistance level to three-level driving assistance according to the performance of the driver, captures the behavior change which causes safety risk in real time, provides health and safety prompts through a user interface, automatically adjusts the vehicle setting according to the driving environment, adjusts the vehicle speed and the vehicle distance, ensures that the safety can be still maintained when the response capability of the driver is reduced, records the behavior data in the current state, and feeds back to the driver through the user interface;
When the system detects that the driver state is B2 or the driving state score R, the system provides light driving assistance, the system adaptively adjusts the driving assistance level to be four according to the performance of the driver, and state reminding is provided for the driver regularly.
6. The intelligent control method for disabled safety of a pilot of a civil aviation vehicle apparatus as claimed in claim 5, wherein said evaluating the driver by detecting the contact surface of the steering wheel comprises,
Fa(t)＝Fb-k·σn(t)
Wherein F b is the baseline force, k is the adjustment coefficient, F a (t) is the adaptive threshold, σ n (t) is the standard deviation calculated based on the last n measurements at time t, used to update the adaptive threshold, n is the number of the last measurements to calculate the current standard deviation.
7. The intelligent control method for disabled drivers of civil aviation vehicle equipment of claim 6, wherein the driving state score comprises, Q > 80; w is more than or equal to 60 and less than 79; e is more than or equal to 40 and less than 59; r < 39.
8. A system based on the intelligent control method for disabled safety of a driving operator of a civil aviation vehicle device according to any one of claims 1-7, which is characterized by comprising a monitoring module, a control module, a judging module, a decision module, an executing module and a data recording module;
The monitoring module, the camera and the wearable equipment are used for detecting the contact surface of the steering wheel of a driver;
the control module is used for analyzing facial expressions and comparing heart rate parameters;
the judging module is used for judging the state of the driver according to the expression state interval;
the decision module takes different measures for different driver states;
the execution module is used for performing intervention control if the state of the driver is abnormal;
the execution data recording module records the state and behavior data of the driver.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410188178.7A CN118062021A (en) | 2024-02-20 | 2024-02-20 | Intelligent control method and system for incapacitation of driving operator of civil aviation vehicle equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410188178.7A CN118062021A (en) | 2024-02-20 | 2024-02-20 | Intelligent control method and system for incapacitation of driving operator of civil aviation vehicle equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118062021A true CN118062021A (en) | 2024-05-24 |
Family
ID=91094763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410188178.7A Pending CN118062021A (en) | 2024-02-20 | 2024-02-20 | Intelligent control method and system for incapacitation of driving operator of civil aviation vehicle equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118062021A (en) |
-
2024
- 2024-02-20 CN CN202410188178.7A patent/CN118062021A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11117596B2 (en) | Intelligent driving control | |
| CN112389448B (en) | Abnormal driving behavior identification method based on vehicle state and driver state | |
| US9776644B2 (en) | On-vehicle situation detection apparatus and method | |
| CN108372785B (en) | Image recognition-based automobile unsafe driving detection device and detection method | |
| US9082285B2 (en) | System and method for determining vehicle driving status information-based driving concentration | |
| US8179243B2 (en) | Lane departure warning method and system thereof | |
| CN109977930B (en) | Fatigue driving detection method and device | |
| CN108839658B (en) | Fatigue driving monitoring system and method | |
| CN112373467A (en) | Intelligent obstacle avoidance system of unmanned automobile | |
| KR20160036968A (en) | Integrated assessment apparatus and method of drivers' drowsiness, inattention and workload | |
| CN112164228A (en) | Helmet-free behavior detection method for driving electric vehicle, electronic device and storage medium | |
| CN114987502B (en) | Driving state monitoring method and device, computer equipment and storage medium | |
| CN116572984A (en) | Dangerous driving management and control method and system based on multi-feature fusion | |
| CN110992709A (en) | Active speed limiting system based on fatigue state of driver | |
| KR20180059224A (en) | Apparatus for controlling stop of vehicle and method thereof | |
| CN210000130U (en) | automobile self-recognition deceleration system | |
| Mašanović et al. | Driver monitoring using the in-vehicle camera | |
| CN118062021A (en) | Intelligent control method and system for incapacitation of driving operator of civil aviation vehicle equipment | |
| KR20220014945A (en) | System and method for monitering driver | |
| CN116552542A (en) | Vehicle control method, device, electronic equipment and storage medium | |
| CN110459034B (en) | Fatigue driving early warning method and system | |
| KR101462067B1 (en) | Apparatus and method for detecting drowsy driving using vehicle information | |
| CN112810610A (en) | Curve driving control method, device and equipment and automobile | |
| CN115299948A (en) | Driver fatigue detection method and detection system | |
| CN114998874A (en) | Driver abnormal behavior detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |