CN107742399B - Method and device for sending alarm signal - Google Patents
Method and device for sending alarm signal Download PDFInfo
- Publication number
- CN107742399B CN107742399B CN201711140773.XA CN201711140773A CN107742399B CN 107742399 B CN107742399 B CN 107742399B CN 201711140773 A CN201711140773 A CN 201711140773A CN 107742399 B CN107742399 B CN 107742399B
- Authority
- CN
- China
- Prior art keywords
- fatigue
- data
- value
- attention
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1118—Determining activity level
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Dentistry (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Physiology (AREA)
- Medical Informatics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The embodiment of the application discloses a method and a device for sending out an alarm signal. One embodiment of the method comprises: acquiring driving state data in real time; importing the driving state data into a fatigue recognition model trained in advance to obtain a fatigue type and a fatigue value corresponding to the fatigue type; importing the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value; and judging the attention threshold range in which the attention value is positioned, and sending out an alarm signal according to the attention threshold range. This embodiment improves the accuracy of the fatigue level identification of the driver.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to the technical field of vehicle control, and particularly relates to a method and a device for sending out an alarm signal.
Background
The automobile expands the range of people going out, brings convenience to people going out and improves the quality of life of people. With the development and progress of science and technology, the unmanned vehicle controlled by the intelligent system can acquire more driving information than the manned vehicle, has higher safety and becomes an important trend of future vehicle development.
In the driving process of the vehicle, traffic accidents are easy to occur when a driver drives the vehicle in a fatigue mode. In order to reduce the occurrence of traffic accidents, the prior art determines whether the driver is tired by monitoring the eye movement of the driver and other methods.
Disclosure of Invention
The embodiment of the application aims to provide a method and a device for sending an alarm signal.
In a first aspect, an embodiment of the present application provides a method for issuing an alarm signal, where the method includes: acquiring driving state data in real time, wherein the driving state data comprises driver state data, driving environment data and vehicle state data, the driver state data comprises hand action data, mouth action data, head action data and eye action data, the driving environment data comprises lane line data and vehicle distance data, and the vehicle state data comprises speed data and direction data; importing the driving state data into a fatigue recognition model trained in advance to obtain a fatigue type and a fatigue value corresponding to the fatigue type, wherein the fatigue recognition model is used for determining the fatigue type and the fatigue value according to the driving state data, and the fatigue value is used for representing the degree corresponding to the fatigue type; importing the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value, wherein the attention value calculation model is used for calculating the attention value according to the fatigue type and the fatigue value, and the attention value is used for representing the fatigue degree of a driver; acquiring an attention threshold range, wherein the attention threshold range comprises a plurality of attention threshold sub-ranges formed by preset attention thresholds, each attention threshold sub-range corresponds to one alarm signal, responding to the attention threshold sub-range corresponding to the attention value and sending out the alarm signal corresponding to the attention threshold sub-range, the attention threshold range is determined by the preset attention threshold, and the attention threshold range is provided with the corresponding alarm signal.
In some embodiments, the method comprises the step of constructing a fatigue identification model, the step of constructing a fatigue identification model comprising: extracting temporally synchronized driver state data, driving environment data and vehicle state data from a driver state data set, a driving environment data set and a vehicle state data set, respectively; respectively extracting characteristics of the driver state data, the driving environment data and the vehicle state data to obtain driver state characteristic data, driving environment characteristic data and vehicle state characteristic data; determining a fatigue type according to the driver state characteristic data, wherein the fatigue type comprises at least one of the following items: eye closing type, yawning type, sight line deviation type and calling type; calculating a fatigue value through the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data; and training to obtain a fatigue recognition model by using a machine learning method and taking the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data as input and taking the fatigue type and a fatigue value of the corresponding fatigue type as output.
In some embodiments, the calculating the fatigue value from the driver state characteristic data, the driving environment characteristic data, and the vehicle state characteristic data includes: acquiring a time threshold range, a distance threshold range and a data variation threshold range, wherein the time threshold range comprises a plurality of time threshold sub-ranges formed by preset time thresholds, the distance threshold range comprises a plurality of distance threshold sub-ranges formed by preset distance thresholds, the data variation threshold range comprises a plurality of data variation threshold sub-ranges formed by preset data variation thresholds, each time threshold sub-range corresponds to a time weight value, each distance threshold sub-range corresponds to a distance weight value, and each data variation threshold sub-range corresponds to a data variation weight value; respectively measuring fatigue type duration time of each fatigue type corresponding to the driver state characteristic data, a distance change value corresponding to the driving environment characteristic data and a data change value corresponding to the vehicle state characteristic data, wherein the fatigue type duration time is represented by duration time of an action corresponding to a specified fatigue type corresponding to the driver state characteristic data, the distance change value is represented by variation of the driving environment characteristic data in unit time, and the data change value is represented by variation of the vehicle state characteristic data in unit time; determining a time weight value of a time threshold sub-range of a time threshold range, a distance weight value of a distance threshold sub-range of the distance threshold range and a data change weight value of a data change threshold sub-range of the data change threshold range, which correspond to the fatigue type duration, the distance change value and the data change value respectively; and weighting and summing the time weight value, the distance weight value and the data change weight value to obtain a fatigue value corresponding to the fatigue type.
In some embodiments, the method includes the step of constructing an attention value calculation model, the step of constructing an attention value calculation model including: dividing fatigue values in the fatigue value set into fatigue value subsets corresponding to the fatigue types according to the fatigue types; respectively setting a type weight value for each fatigue type, and taking the product of the type weight value and each fatigue value in a fatigue value subset corresponding to the type weight value as an attention component when the value of the corresponding fatigue type is the fatigue value; and training to obtain an attention value calculation model by using a machine learning method and taking the fatigue type and the fatigue value as inputs and the attention component as an output.
In some embodiments, the determining the attention threshold range in which the attention value is located, and sending the alarm signal according to the attention threshold range further includes: in response to the attention value not being within the attention threshold range, an automatic driving mode is initiated.
In a second aspect, an embodiment of the present application provides an apparatus for issuing an alarm signal, where the apparatus includes: the driving state data acquisition unit is used for acquiring driving state data in real time, wherein the driving state data comprise driver state data, driving environment data and vehicle state data, the driver state data comprise hand action data, mouth action data, head action data and eye action data, the driving environment data comprise lane line data and vehicle distance data, and the vehicle state data comprise speed data and direction data; a fatigue value obtaining unit, configured to import the driving state data into a fatigue recognition model trained in advance, to obtain a fatigue type and a fatigue value corresponding to the fatigue type, where the fatigue recognition model is configured to determine the fatigue type and the fatigue value according to the driving state data, and the fatigue value is used to represent a degree corresponding to the fatigue type; an attention value obtaining unit, configured to import the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value, where the attention value calculation model is configured to calculate an attention value according to the fatigue type and the fatigue value, and the attention value is used to represent a fatigue degree of a driver; the alarm unit comprises an attention threshold range acquisition sub-unit and an alarm sub-unit, wherein the attention threshold range acquisition sub-unit is used for acquiring an attention threshold range, the attention threshold range comprises a plurality of attention threshold sub-ranges formed by preset attention thresholds, each attention threshold sub-range corresponds to one alarm signal, the alarm sub-unit is used for responding to the attention threshold sub-range corresponding to the attention value and sending out the alarm signal corresponding to the attention threshold sub-range, the attention threshold range is determined by the preset attention threshold, and the attention threshold range is provided with the corresponding alarm signal.
In some embodiments, the apparatus comprises a fatigue recognition model building unit for building a fatigue recognition model, the fatigue recognition model building unit comprising: a data extraction subunit, configured to extract temporally synchronized driver state data, driving environment data, and vehicle state data from the driver state data set, the driving environment data set, and the vehicle state data set, respectively; the sign extraction subunit is used for respectively extracting the characteristics of the driver state data, the driving environment data and the vehicle state data to obtain the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data; a fatigue type determining subunit, configured to determine a fatigue type according to the driver state characteristic data, where the fatigue type includes at least one of: eye closing type, yawning type, sight line deviation type and calling type; the fatigue value calculating subunit is used for calculating a fatigue value according to the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data; and the fatigue recognition model construction subunit is used for training to obtain the fatigue recognition model by using a machine learning method and taking the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data as input and taking the fatigue type and a fatigue value of the corresponding fatigue type as output.
In some embodiments, the fatigue value calculating subunit includes: the threshold range obtaining module is used for obtaining a time threshold range, a distance threshold range and a data variation threshold range, wherein the time threshold range comprises a plurality of time threshold sub-ranges formed by preset time thresholds, the distance threshold range comprises a plurality of distance threshold sub-ranges formed by preset distance thresholds, the data variation threshold range comprises a plurality of data variation threshold sub-ranges formed by preset data variation thresholds, each time threshold sub-range corresponds to a time weight value, each distance threshold sub-range corresponds to a distance weight value, and each data variation threshold sub-range corresponds to a data variation weight value; the data measurement module is used for respectively measuring the fatigue type duration time of each fatigue type corresponding to the driver state characteristic data, the distance change value corresponding to the driving environment characteristic data and the data change value corresponding to the vehicle state characteristic data, wherein the fatigue type duration time is represented by the duration time of an action corresponding to a specified fatigue type corresponding to the driver state characteristic data, the distance change value is represented by the variation of the driving environment characteristic data in unit time, and the data change value is represented by the variation of the vehicle state characteristic data in unit time; the weight value determining module is used for determining a time weight value of a time threshold sub-range of a time threshold range, a distance weight value of a distance threshold sub-range of the distance threshold range and a data change weight value of a data change threshold sub-range of a data change threshold range, which correspond to the fatigue type duration, the distance change value and the data change value respectively; and the fatigue value calculation module is used for weighting and summing the time weight value, the distance weight value and the data change weight value to obtain a fatigue value corresponding to the fatigue type.
In some embodiments, the apparatus includes an attention value calculation model construction unit for constructing an attention value calculation model, the attention value calculation model construction unit including: the data dividing subunit is used for dividing the fatigue values in the fatigue value set into fatigue value subsets corresponding to the fatigue types according to the fatigue types; the attention component calculating subunit is used for respectively setting a type weight value for each fatigue type, and taking the product of the type weight value and each fatigue value in the fatigue value subset corresponding to the type weight value as the attention component of the corresponding fatigue type when the value is the fatigue value; and the attention value calculation model construction subunit is used for training to obtain the attention value calculation model by using a machine learning method and taking the fatigue type and the fatigue value as input and the attention component as output.
In some embodiments, the alarm unit further includes: in response to the attention value not being within the attention threshold range, an automatic driving mode is initiated.
In a third aspect, an embodiment of the present application provides a server, including: one or more processors; memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to perform the method for issuing an alert signal of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the method for issuing an alarm signal of the first aspect.
The method and the device for sending the warning signal, provided by the embodiment of the application, can be used for acquiring the driving state data in real time, wherein the driving state data comprises the driver state data, the driving environment data and the vehicle state data, so that the data pairs of the driver, the driving environment and the vehicle data are comprehensively considered, then the driving state data is led into a fatigue recognition model to obtain the fatigue type and the fatigue value corresponding to the fatigue type, the fatigue type and the fatigue value are led into a pre-trained attention value calculation model to obtain the attention value, finally the corresponding warning signal is sent according to the attention threshold value range where the attention value is located, and the accuracy of fatigue degree recognition of the driver is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a method for signaling an alert according to the present application;
FIG. 3 is a schematic diagram of one application scenario of a method for signaling an alert according to the present application;
FIG. 4 is a schematic block diagram illustrating one embodiment of an apparatus for signaling an alert according to the present application;
FIG. 5 is a block diagram of a computer system suitable for use in implementing a server according to embodiments of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows an exemplary system architecture 100 to which embodiments of the present method for issuing an alert signal or apparatus for issuing an alert signal may be applied.
As shown in fig. 1, the system architecture 100 may include a first camera 101, a second camera 102, a vehicle-mounted terminal 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the in-vehicle terminal 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The first camera 101 may be disposed outside the vehicle, and is configured to collect an image of the current vehicle during driving, calculate a distance between the current vehicle and another vehicle or a pedestrian on a driving road according to the collected image, and further calculate a distance between the current vehicle and a lane line; the second camera 102 may be disposed inside the vehicle for monitoring various motions of the driver, including hand motions, mouth motions, head motions, eye motions, and the like, and converting the motions into corresponding data; the vehicle-mounted terminal 103 may receive the first camera 101, the second camera 102 and the current vehicle state data (for example, speed data, direction data, etc.), process the received data, and send out a corresponding warning signal when the driver is driving fatigue.
The server 105 may be a server that provides various services, for example, a server that processes data transmitted from the in-vehicle terminal 103, and the server 105 may process data transmitted from the in-vehicle terminal 103 and transmit data such as the acquired attention value to the in-vehicle terminal 103.
It should be noted that the method for sending the alarm signal provided by the embodiment of the present application is generally executed by the vehicle-mounted terminal 103, and accordingly, the device for sending the alarm signal is generally disposed in the vehicle-mounted terminal 103.
It should be understood that the numbers of the first camera 101, the second camera 102, the in-vehicle terminal 103, the network 104, and the server 105 in fig. 1 are merely illustrative. There may be any number of first cameras 101, second cameras 102, in-vehicle terminals 103, networks 104, and servers 105, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a method for issuing an alert signal according to the present application is shown. The method for issuing an alarm signal comprises the following steps:
In the present embodiment, the electronic device (e.g., the in-vehicle terminal 103 shown in fig. 1) on which the method for issuing the warning signal operates may receive the driving state data through a wired connection manner or a wireless connection manner. The driving state data may include driver state data, driving environment data, and vehicle state data, the driver state data may include hand motion data, mouth motion data, head motion data, and eye motion data, the driving environment data may include lane line data and vehicle distance data, and the vehicle state data may include speed data and direction data. It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a uwb (ultra wideband) connection, and other wireless connection means now known or developed in the future.
When the driver drives the vehicle to run, the driver state data, the driving environment data and the vehicle state data can be acquired in real time. The driver state data can be collected by a second camera 102 arranged in the vehicle; the driving environment data can be collected by a first camera 101 arranged outside the vehicle; the vehicle state data may be obtained directly from the vehicle.
After the driving state data is acquired, the driving state data can be imported into a fatigue identification model, so that a fatigue type (which can be represented by a form of information) and a fatigue value corresponding to the fatigue type are obtained. The fatigue identification model is used for determining a fatigue type and a fatigue value through the driving state data, and the fatigue value is used for representing the degree of the corresponding fatigue type. The fatigue value can be expressed in various forms. For example, the fatigue value may range from 0 to 1. When the fatigue value is 0, the driver can be considered not to be fatigued; when the fatigue value is 1, the driver can be considered to be severely fatigued. The fatigue value can also be represented by other forms such as 0 to 100, and is not described in detail here.
In some optional implementations of this embodiment, the method of this embodiment may include a step of constructing a fatigue identification model, where the step of constructing the fatigue identification model may include the following steps:
in a first step, temporally synchronized driver state data, driving environment data and vehicle state data are extracted from a driver state data set, a driving environment data set and a vehicle state data set, respectively.
When the driver is in fatigue driving, not only the driver himself exhibits some fatigue characteristics, but also the vehicle is subjected to driving abnormality due to fatigue driving by the driver. Therefore, in determining whether the driver is fatigued, temporally synchronized driver state data, driving environment data, and vehicle state data may be extracted from the driver state data set, the driving environment data set, and the vehicle state data set, respectively. It should be noted that the driver state data, the driving environment data, and the vehicle state data are synchronized (temporally identical) as necessary in terms of time, so that an accurate judgment as to whether the driver is tired or not is made by integrating the various data.
And secondly, respectively carrying out feature extraction on the driver state data, the driving environment data and the vehicle state data to obtain the driver state feature data, the driving environment feature data and the vehicle state feature data.
Driver status data, driving environment data, and vehicle status data are typically collected directly. In order to determine whether the driver is tired, it is necessary to perform feature extraction on the driver state data, the driving environment data, and the vehicle state data to obtain the driver state feature data, the driving environment feature data, and the vehicle state feature data. That is, the driver state characteristic data, the driving environment characteristic data, and the vehicle state characteristic data are data related to driver fatigue driving. For example, the driver state characteristic data may be a time value of eye closure; the driving environment characteristic data may be a variation value of the vehicle distance; the vehicle state characteristic data may be a change value of the speed. The driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data can also be other types of data, depending on the actual needs.
And thirdly, determining the fatigue type according to the characteristic data of the state of the driver.
The driver state data may include data such as hand motion data, mouth motion data, head motion data, and eye motion data of the driver. When the driver presents one or more types of data, the corresponding action data can be represented by the driver state characteristic data. If a certain type or types of data are present, it is indicated that the driver exhibits a corresponding type of fatigue. The above fatigue types include at least one of: closed eye type, yawning type, line of sight offset type, and telephone type.
And fourthly, calculating a fatigue value according to the characteristic data of the state of the driver, the characteristic data of the driving environment and the characteristic data of the state of the vehicle.
After the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data are obtained, the fatigue value corresponding to the fatigue type can be obtained through a certain data processing mode. For example, the values of the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data corresponding to a certain fatigue type can be respectively calculated and then weighted to obtain the fatigue value of the fatigue type.
And fifthly, training to obtain a fatigue recognition model by using a machine learning method and taking the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data as input, and taking the fatigue type and a fatigue value of the corresponding fatigue type as output.
The fatigue identification model can be a corresponding relation table which is preset by technicians based on statistics of a large amount of driver state characteristic data, driving environment characteristic data, vehicle state characteristic data, fatigue types and fatigue values and stores corresponding relations of the driver state characteristic data, the driving environment characteristic data, the vehicle state characteristic data, the fatigue types and the fatigue values; the calculation formula may be a calculation formula which is preset by a technician based on statistics of a large amount of data and stored in the electronic device, and performs numerical calculation on the driver state characteristic data, the driving environment characteristic data, and the vehicle state characteristic data to obtain a calculation result of a fatigue value for characterizing the type of fatigue. For example, the calculation formula may be a formula for weighted summation of the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data, and the obtained result may be used to represent a fatigue value corresponding to the fatigue type.
In some optional implementations of the embodiment, the calculating the fatigue value through the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data may include:
the method comprises the following steps of firstly, acquiring a time threshold range, a distance threshold range and a data variation threshold range.
In general, the time threshold range, the distance threshold range, and the data change amount threshold range may be set in advance. The time threshold range can comprise a plurality of time threshold sub-ranges formed by preset time thresholds; the distance threshold range may include a plurality of distance threshold sub-ranges composed of preset distance thresholds; the data variation threshold range may include a plurality of data variation threshold sub-ranges composed of data variation thresholds set in advance. Each time threshold sub-range corresponds to a time weight value; each distance threshold sub-range corresponds to a distance weight value; each data change threshold sub-range corresponds to a data change weight value.
And secondly, respectively measuring the fatigue type duration of each fatigue type corresponding to the driver state characteristic data, the distance change value corresponding to the driving environment characteristic data and the data change value corresponding to the vehicle state characteristic data.
In order to quantitatively determine the correlation between various data and fatigue driving, the fatigue type duration, the distance variation value corresponding to the driving environment characteristic data, and the data variation value corresponding to the vehicle state characteristic data may be measured for each fatigue type corresponding to the driver state characteristic data, respectively. The fatigue type duration time can be represented by duration of an action corresponding to a designated fatigue type corresponding to the driver state characteristic data, the distance change value can be represented by variation of the driving environment characteristic data in unit time, and the data change value can be represented by variation of the vehicle state characteristic data in unit time.
And thirdly, determining a time weight value of a time threshold sub-range of a time threshold range, a distance weight value of a distance threshold sub-range of the distance threshold range and a data change weight value of a data change threshold sub-range of the data change threshold range, which correspond to the fatigue type duration, the distance change value and the data change value respectively.
After the fatigue type duration, the distance variation value and the data variation value are obtained, a time weight value of the fatigue type duration in a time threshold sub-range corresponding to the time threshold range, a distance weight value of the distance variation value in a distance threshold sub-range corresponding to the distance threshold range and a data variation weight value of the data variation value in a data variation threshold sub-range corresponding to the data variation threshold range can be respectively obtained through data comparison.
And fourthly, weighting and summing the time weight value, the distance weight value and the data change weight value to obtain a fatigue value corresponding to the fatigue type.
The correlation of the time weight value, the distance weight value, and the data change weight value to different fatigue types may be different. Therefore, different weights can be set for the time weight value, the distance weight value and the data change weight value according to the specific fatigue type, and the time weight value, the distance weight value, the data change weight value and the respective weight value are multiplied and then summed to obtain the fatigue value corresponding to the fatigue type. For example, the time weight value, the distance weight value, and the data change weight value may be 0.1, 0.3, and 0.5, respectively, the weight value corresponding to a certain fatigue type may be 0.2, 0.1, and 0.4, respectively, and the fatigue value corresponding to the certain fatigue type may be 0.1 × 0.2+0.3 × 0.1+0.5 × 0.4 — 0.25.
And step 203, importing the fatigue types and the fatigue values into a pre-trained attention value calculation model to obtain an attention value.
After the fatigue type and the fatigue value are obtained, the fatigue type and the fatigue value can be led into an attention value calculation model to obtain an attention value. Wherein the attention value calculation model is used for calculating an attention value through the fatigue type and the fatigue value, and the attention value is used for representing the fatigue degree of the driver. Similar to the fatigue value, the attention value may also range from 0 to 1. When the attention value is 0, the driver can be considered to be severely fatigued; when the attention value is 1, the driver can be considered not to be tired. The attention value can also be represented by other forms, which are not described in detail herein.
In some optional implementations of the present embodiment, the method of the present embodiment may include a step of constructing an attention value calculation model, and the step of constructing an attention value calculation model may include the following steps:
firstly, dividing fatigue values in the fatigue value set into fatigue value subsets corresponding to fatigue types according to the fatigue types.
As can be seen from the above description, different fatigue types correspond to respective fatigue values. Thus, the fatigue values in the set of fatigue values may be divided by the fatigue type into subsets of fatigue values corresponding to the fatigue type.
And secondly, respectively setting a type weight value for each fatigue type, and taking the product of the type weight value and each fatigue value in the fatigue value subset corresponding to the type weight value as the attention component of the corresponding fatigue type when the fatigue value is taken as the value.
According to the driving habits and the driving behaviors of different drivers, a type weight value can be set for each fatigue type so as to make a targeted judgment on whether the driver is tired according to the characteristics of the driver. When an attention class component of a certain fatigue type needs to be acquired, the product of the type weight value of the fatigue type and each fatigue value in the fatigue value subset corresponding to the type weight value can be used as the attention component when the value of the corresponding fatigue type is the fatigue value. I.e. how many fatigue values there are corresponding attention components. When the driver has multiple fatigue types, the attention components corresponding to the fatigue types can be summed to obtain the attention value of the driver. For example, if the driver has a fatigue type A, B, C, and the corresponding fatigue value subsets have fatigue values of 0.1, 0.2, and 0.3, respectively, and the corresponding type weights may be 0.2, and 0.6, respectively, then the attention value of the driver may be 0.1 × 0.2+0.2 × 0.2+0.3 × 0.6 — 0.24 when the fatigue values are 0.1, 0.2, and 0.3, respectively.
And thirdly, training to obtain an attention value calculation model by using a machine learning method and taking the fatigue type and the fatigue value as input and the attention component as output.
The attention value calculation model is a corresponding relation table which is preset by technicians based on statistics of a large number of fatigue types and fatigue values and stores corresponding relations of the fatigue types, the fatigue values and the attention components; or a calculation formula which is preset by a technician based on statistics of a large amount of data and stored in the electronic device, and is used for performing numerical calculation on the fatigue value to obtain a calculation result of the attention component for representing the fatigue type. For example, the calculation formula may be a formula for weighted summation of fatigue values, and the obtained result may be used to characterize the attention component corresponding to the fatigue type. Thereafter, the attention components of the respective fatigue types are summed to obtain an attention value.
It should be noted that the fatigue recognition model and the attention value calculation model described above can be obtained by training in advance through the server 105.
And step 204, judging the attention threshold range in which the attention value is positioned, and sending out an alarm signal according to the attention threshold range.
The in-vehicle terminal 103 may be preset with a plurality of attention thresholds, and an attention threshold range may be formed by the plurality of attention thresholds, that is, the above-mentioned attention threshold range is determined by a preset attention threshold. Then, a different warning signal may be set for each attention threshold range, and the different warning signals may represent the current attention of the driver. When the attention value belongs to a certain attention threshold range, an alarm signal corresponding to the attention threshold range is sent out.
In some optional implementations of this embodiment, the determining the attention threshold range in which the attention value is located, and sending the alarm signal according to the attention threshold range may include the following steps:
first, an attention threshold range is obtained.
Before an alarm signal is issued, the attention threshold range needs to be acquired first. Wherein, the attention threshold range includes a plurality of attention threshold sub-ranges formed by preset attention thresholds, each attention threshold sub-range corresponds to an alarm signal, and the alarm signal may include: an audio alert signal, a video alert signal, etc.
And secondly, responding to the attention threshold sub-range corresponding to the attention value, and sending out an alarm signal corresponding to the attention threshold sub-range.
For example, the attention threshold range may be constituted by attention thresholds 0, 0.5, and 1. When the attention threshold range of the attention value is between 0.5 and 1, the attention of the driver can be considered to be concentrated, correspondingly, the driver is not in a fatigue driving state, and the warning signal can be a voice for prompting the driver to pay attention to driving safety; when the attention value is in the attention threshold range of 0 to 0.5, the attention of the driver can be considered to be not concentrated, correspondingly, the driver is not in a light fatigue driving state, and the warning signal can be high-volume voice and an image signal to prompt the driver to pay attention to the driving safety.
In some optional implementation manners of this embodiment, the determining the attention threshold range in which the attention value is located, and sending the alarm signal according to the attention threshold range may further include: when the attention value is not within the above-mentioned attention threshold range, the automatic driving mode is started.
The in-vehicle terminal 103 may also automatically start the automatic driving mode in response to the vehicle having the automatic driving function and the attention value not being within the above-described attention threshold range. For example, when the attention value is in the attention threshold range of 0, the driver may be considered to lose attention, and at this time, the in-vehicle terminal 103 may directly start the automatic driving mode of the vehicle, so as to improve the safety of driving.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the method for issuing an alert signal according to the present embodiment. In the application scenario of fig. 3, after the vehicle-mounted terminal 103 acquires driving state data such as vehicle state data of a vehicle, driving environment data acquired by the first camera 101, and driver state data acquired by the second camera 102, the driving state data is imported into a fatigue recognition model to obtain a current fatigue type of the driver and a fatigue value corresponding to the fatigue type; then, importing the fatigue type and the fatigue value into an attention value calculation model to obtain an attention value; and finally, judging the attention threshold range in which the attention value is positioned, and sending out a warning signal 'please pay attention to driving safety' according to the attention threshold range.
The method provided by the embodiment of the application can acquire the driving state data in real time, the driving state data comprise driver state data, driving environment data and vehicle state data, the data pair of the driver, the driving environment and the vehicle data is comprehensively considered, then the driving state data are led into the fatigue recognition model to obtain the fatigue type and the fatigue value corresponding to the fatigue type, the fatigue type and the fatigue value are led into the pre-trained attention value calculation model to obtain the attention value, finally the corresponding warning signal is sent out according to the attention threshold value range where the attention value is located, and the accuracy of fatigue degree recognition of the driver is improved.
With further reference to fig. 4, as an implementation of the method shown in the above-mentioned figures, the present application provides an embodiment of an apparatus for issuing an alarm signal, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 4, and the apparatus may be applied to various electronic devices.
As shown in fig. 4, the apparatus 400 for issuing an alarm signal of the present embodiment may include: a driving state data acquisition unit 401, a fatigue value acquisition unit 402, an attention value acquisition unit 403, and a warning unit 404. The driving state data acquiring unit 401 is configured to acquire driving state data in real time, where the driving state data includes driver state data, driving environment data, and vehicle state data, the driver state data includes hand motion data, mouth motion data, head motion data, and eye motion data, the driving environment data includes lane line data and vehicle distance data, and the vehicle state data includes speed data and direction data; the fatigue value obtaining unit 402 is configured to import the driving state data into a fatigue recognition model trained in advance, to obtain a fatigue type and a fatigue value corresponding to the fatigue type, where the fatigue recognition model is configured to determine the fatigue type and the fatigue value according to the driving state data, and the fatigue value is used to represent a degree corresponding to the fatigue type; the attention value obtaining unit 403 is configured to import the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value, where the attention value calculation model is configured to calculate an attention value according to the fatigue type and the fatigue value, and the attention value is used to represent a fatigue degree of a driver; the alarm unit 404 is configured to determine an attention threshold range in which the attention value is located, and send an alarm signal according to the attention threshold range, where the attention threshold range is determined by a preset attention threshold.
In some optional implementations of the present embodiment, the apparatus 400 for issuing an alarm signal of the present embodiment may include a fatigue recognition model building unit (not shown in the figure) for building a fatigue recognition model, where the fatigue recognition model building unit may include: a data extraction subunit (not shown), a feature extraction subunit (not shown), a fatigue type determination subunit (not shown), a fatigue value calculation subunit (not shown), and a fatigue recognition model construction subunit (not shown). The data extraction subunit is used for respectively extracting temporally synchronous driver state data, driving environment data and vehicle state data from the driver state data set, the driving environment data set and the vehicle state data set; the characteristic extraction subunit is used for respectively extracting the characteristics of the driver state data, the driving environment data and the vehicle state data to obtain the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data; the fatigue type determining subunit is used for determining a fatigue type according to the driver state characteristic data, wherein the fatigue type comprises at least one of the following items: eye closing type, yawning type, sight line deviation type and calling type; the fatigue value calculation operator unit is used for calculating a fatigue value through the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data; the fatigue recognition model construction subunit is used for training to obtain a fatigue recognition model by using a machine learning method and taking the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data as input and taking the fatigue type and a fatigue value of the corresponding fatigue type as output.
In some optional implementations of the present embodiment, the fatigue value calculating subunit may include: a threshold range obtaining module (not shown), a data measuring module (not shown), a weight value determining module (not shown) and a fatigue value calculating module (not shown). The threshold range obtaining module is used for obtaining a time threshold range, a distance threshold range and a data variation threshold range, wherein the time threshold range comprises a plurality of time threshold sub-ranges formed by preset time thresholds, the distance threshold range comprises a plurality of distance threshold sub-ranges formed by preset distance thresholds, the data variation threshold range comprises a plurality of data variation threshold sub-ranges formed by preset data variation thresholds, each time threshold sub-range corresponds to a time weight value, each distance threshold sub-range corresponds to a distance weight value, and each data variation threshold sub-range corresponds to a data variation weight value; the data measurement module is used for respectively measuring the fatigue type duration time of each fatigue type corresponding to the driver state characteristic data, the distance change value corresponding to the driving environment characteristic data and the data change value corresponding to the vehicle state characteristic data, wherein the fatigue type duration time is represented by the duration time of an action corresponding to a specified fatigue type corresponding to the driver state characteristic data, the distance change value is represented by the variation of the driving environment characteristic data in unit time, and the data change value is represented by the variation of the vehicle state characteristic data in unit time; the weight value determining module is used for determining a time weight value of a time threshold sub-range of a time threshold range, a distance weight value of a distance threshold sub-range of the distance threshold range and a data change weight value of a data change threshold sub-range of a data change threshold range, which correspond to the fatigue type duration, the distance change value and the data change value respectively; and the fatigue value calculation module is used for weighting and summing the time weight value, the distance weight value and the data change weight value to obtain a fatigue value corresponding to the fatigue type.
In some optional implementations of the present embodiment, the apparatus 400 for issuing an alert signal of the present embodiment may include an attention value calculation model building unit (not shown in the figure) for building an attention value calculation model, and the attention value calculation model building unit may include: a data division subunit (not shown in the figure), an attention component calculation subunit (not shown in the figure), and an attention value calculation model construction subunit (not shown in the figure). The data dividing unit is used for dividing fatigue values in the fatigue value set into fatigue value subsets corresponding to the fatigue types according to the fatigue types; the attention component calculating subunit is configured to set a type weight value for each fatigue type, and use a product of the type weight value and each fatigue value in the fatigue value subset corresponding to the type weight value as an attention component of the corresponding fatigue type when the value is the fatigue value; and the attention value calculation model construction subunit is used for training to obtain the attention value calculation model by using a machine learning method and taking the fatigue type and the fatigue value as inputs and the attention component as an output.
In some optional implementations of this embodiment, the alarm unit 404 may include: an attention threshold range acquisition subunit (not shown in the figure) and an alarm subunit (not shown in the figure). The attention threshold range obtaining sub-unit is configured to obtain an attention threshold range, where the attention threshold range includes multiple attention threshold sub-ranges formed by preset attention thresholds, each attention threshold sub-range corresponds to an alarm signal, and the alarm signal includes: sound alarm signal, image alarm signal; the alarm sub-unit is used for responding to the attention threshold sub-range corresponding to the attention value and sending out an alarm signal corresponding to the attention threshold sub-range.
In some optional implementations of this embodiment, the alarm unit 404 may further include: in response to the attention value not being within the attention threshold range, an automatic driving mode is initiated.
The present embodiment further provides a server, including: one or more processors; a memory for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to perform the above-described method for issuing an alert signal.
The present embodiment also provides a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the above-mentioned method for issuing an alarm signal.
Referring now to FIG. 5, a block diagram of a computer system 500 suitable for use in implementing a server according to embodiments of the present application is shown. The server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the method of the present application when executed by the Central Processing Unit (CPU) 501.
It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a driving state data acquisition unit, a fatigue value acquisition unit, an attention value acquisition unit, and an alarm unit. Where the names of the elements do not in some cases constitute a limitation of the elements themselves, for example, an alarm element may also be described as an "element for issuing an alarm signal".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: acquiring driving state data in real time, wherein the driving state data comprises driver state data, driving environment data and vehicle state data, the driver state data comprises hand action data, mouth action data, head action data and eye action data, the driving environment data comprises lane line data and vehicle distance data, and the vehicle state data comprises speed data and direction data; importing the driving state data into a fatigue recognition model trained in advance to obtain a fatigue type and a fatigue value corresponding to the fatigue type, wherein the fatigue recognition model is used for determining the fatigue type and the fatigue value according to the driving state data, and the fatigue value is used for representing the degree corresponding to the fatigue type; importing the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value, wherein the attention value calculation model is used for calculating the attention value according to the fatigue type and the fatigue value, and the attention value is used for representing the fatigue degree of a driver; and judging the attention threshold range in which the attention value is positioned, and sending out an alarm signal according to the attention threshold range, wherein the attention threshold range is determined by a preset attention threshold.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (10)
1. A method for signaling an alarm, the method comprising:
acquiring driving state data in real time, wherein the driving state data comprises driver state data, driving environment data and vehicle state data, the driver state data comprises hand action data, mouth action data, head action data and eye action data, the driving environment data comprises lane line data and vehicle distance data, and the vehicle state data comprises speed data and direction data;
the driving state data are imported into a fatigue recognition model trained in advance to obtain a fatigue type and a fatigue value corresponding to the fatigue type, the fatigue recognition model is used for determining the fatigue type and the fatigue value through the driving state data, and the fatigue value is determined after being weighted by corresponding weighting coefficients based on the driving state characteristic data, the driving environment characteristic data and the vehicle state characteristic data under the corresponding fatigue type; wherein the type of fatigue comprises at least one of: eye closing type, yawning type, sight line deviation type and calling type;
importing the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value, wherein the attention value calculation model is used for calculating the attention value according to the fatigue type and the fatigue value, and the attention value is used for representing the fatigue degree of a driver;
the method comprises the steps of obtaining an attention threshold range, wherein the attention threshold range comprises a plurality of attention threshold sub-ranges formed by preset attention thresholds, each attention threshold sub-range corresponds to one warning signal, responding to the attention threshold sub-range corresponding to an attention value, sending out the warning signal corresponding to the attention threshold sub-range, responding to the attention value not within the attention threshold range, starting an automatic driving mode, wherein the attention threshold range is determined by the preset attention threshold, and the attention threshold range is provided with the corresponding warning signals.
2. The method of claim 1, comprising the step of constructing a fatigue identification model, the step of constructing a fatigue identification model comprising:
extracting temporally synchronized driver state data, driving environment data and vehicle state data from a driver state data set, a driving environment data set and a vehicle state data set, respectively;
respectively extracting characteristics of the driver state data, the driving environment data and the vehicle state data to obtain driver state characteristic data, driving environment characteristic data and vehicle state characteristic data;
determining the fatigue type according to the driver state characteristic data;
calculating a fatigue value through the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data;
and training to obtain a fatigue recognition model by using a machine learning method and taking the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data as input and taking the fatigue type and a fatigue value of the corresponding fatigue type as output.
3. The method of claim 2, wherein calculating the fatigue value from the driver state characteristic data, the driving environment characteristic data, and the vehicle state characteristic data comprises:
acquiring a time threshold range, a distance threshold range and a data variation threshold range, wherein the time threshold range comprises a plurality of time threshold sub-ranges formed by preset time thresholds, the distance threshold range comprises a plurality of distance threshold sub-ranges formed by preset distance thresholds, the data variation threshold range comprises a plurality of data variation threshold sub-ranges formed by preset data variation thresholds, each time threshold sub-range corresponds to a time weight value, each distance threshold sub-range corresponds to a distance weight value, and each data variation threshold sub-range corresponds to a data variation weight value;
respectively measuring fatigue type duration time of each fatigue type corresponding to the driver state characteristic data, a distance change value corresponding to the driving environment characteristic data and a data change value corresponding to the vehicle state characteristic data, wherein the fatigue type duration time is represented by duration time of an action corresponding to a specified fatigue type corresponding to the driver state characteristic data, the distance change value is represented by variation of the driving environment characteristic data in unit time, and the data change value is represented by variation of the vehicle state characteristic data in unit time;
determining a time weight value of a time threshold sub-range of a time threshold range, a distance weight value of a distance threshold sub-range of the distance threshold range and a data change weight value of a data change threshold sub-range of the data change threshold range, which correspond to the fatigue type duration, the distance change value and the data change value respectively;
and weighting and summing the time weight value, the distance weight value and the data change weight value to obtain a fatigue value corresponding to the fatigue type.
4. The method of claim 1, comprising the step of constructing an attention value calculation model, the step of constructing an attention value calculation model comprising:
dividing fatigue values in the fatigue value set into fatigue value subsets corresponding to the fatigue types according to the fatigue types;
respectively setting a type weight value for each fatigue type, and taking the product of the type weight value and each fatigue value in a fatigue value subset corresponding to the type weight value as an attention component when the value of the corresponding fatigue type is the fatigue value;
and training to obtain an attention value calculation model by using a machine learning method and taking the fatigue type and the fatigue value as inputs and the attention component as an output.
5. An apparatus for signaling an alarm, the apparatus comprising:
the driving state data acquisition unit is used for acquiring driving state data in real time, wherein the driving state data comprise driver state data, driving environment data and vehicle state data, the driver state data comprise hand action data, mouth action data, head action data and eye action data, the driving environment data comprise lane line data and vehicle distance data, and the vehicle state data comprise speed data and direction data;
the fatigue value acquisition unit is used for importing the driving state data into a fatigue recognition model trained in advance to obtain a fatigue type and a fatigue value corresponding to the fatigue type, the fatigue recognition model is used for determining the fatigue type and the fatigue value according to the driving state data, and the fatigue value is determined based on the driving state characteristic data, the driving environment characteristic data and the vehicle state characteristic data under the corresponding fatigue type after being weighted by corresponding weighting coefficients; wherein the type of fatigue comprises at least one of: eye closing type, yawning type, sight line deviation type and calling type;
an attention value obtaining unit, configured to import the fatigue type and the fatigue value into a pre-trained attention value calculation model to obtain an attention value, where the attention value calculation model is configured to calculate an attention value according to the fatigue type and the fatigue value, and the attention value is used to represent a fatigue degree of a driver;
the warning unit comprises an attention threshold range acquisition sub-unit and a warning sub-unit, wherein the attention threshold range acquisition sub-unit is used for acquiring an attention threshold range, the attention threshold range comprises a plurality of attention threshold sub-ranges formed by preset attention thresholds, each attention threshold sub-range corresponds to one warning signal, the warning sub-unit is used for responding to the attention threshold sub-range corresponding to the attention value, sending out the warning signal corresponding to the attention threshold sub-range, and starting an automatic driving mode when the attention value is not in the attention threshold range, the attention threshold range is determined by the preset attention threshold, and the attention threshold range is provided with the corresponding warning signals.
6. The apparatus according to claim 5, characterized in that the apparatus comprises a fatigue recognition model construction unit for constructing a fatigue recognition model, the fatigue recognition model construction unit comprising:
a data extraction subunit, configured to extract temporally synchronized driver state data, driving environment data, and vehicle state data from the driver state data set, the driving environment data set, and the vehicle state data set, respectively;
the characteristic extraction subunit is used for respectively extracting the characteristics of the driver state data, the driving environment data and the vehicle state data to obtain the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data;
the fatigue type determining subunit is used for determining the fatigue type according to the characteristic data of the state of the driver;
the fatigue value calculating subunit is used for calculating a fatigue value according to the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data;
and the fatigue recognition model construction subunit is used for training to obtain the fatigue recognition model by using a machine learning method and taking the driver state characteristic data, the driving environment characteristic data and the vehicle state characteristic data as input and taking the fatigue type and a fatigue value of the corresponding fatigue type as output.
7. The apparatus of claim 6, wherein the fatigue value calculating subunit comprises:
the threshold range obtaining module is used for obtaining a time threshold range, a distance threshold range and a data variation threshold range, wherein the time threshold range comprises a plurality of time threshold sub-ranges formed by preset time thresholds, the distance threshold range comprises a plurality of distance threshold sub-ranges formed by preset distance thresholds, the data variation threshold range comprises a plurality of data variation threshold sub-ranges formed by preset data variation thresholds, each time threshold sub-range corresponds to a time weight value, each distance threshold sub-range corresponds to a distance weight value, and each data variation threshold sub-range corresponds to a data variation weight value;
the data measurement module is used for respectively measuring the fatigue type duration time of each fatigue type corresponding to the driver state characteristic data, the distance change value corresponding to the driving environment characteristic data and the data change value corresponding to the vehicle state characteristic data, wherein the fatigue type duration time is represented by the duration time of an action corresponding to a specified fatigue type corresponding to the driver state characteristic data, the distance change value is represented by the variation of the driving environment characteristic data in unit time, and the data change value is represented by the variation of the vehicle state characteristic data in unit time;
the weight value determining module is used for determining a time weight value of a time threshold sub-range of a time threshold range, a distance weight value of a distance threshold sub-range of the distance threshold range and a data change weight value of a data change threshold sub-range of a data change threshold range, which correspond to the fatigue type duration, the distance change value and the data change value respectively;
and the fatigue value calculation module is used for weighting and summing the time weight value, the distance weight value and the data change weight value to obtain a fatigue value corresponding to the fatigue type.
8. The apparatus according to claim 5, characterized in that the apparatus includes an attention value calculation model construction unit for constructing an attention value calculation model, the attention value calculation model construction unit including:
the data dividing subunit is used for dividing the fatigue values in the fatigue value set into fatigue value subsets corresponding to the fatigue types according to the fatigue types;
the attention component calculating subunit is used for respectively setting a type weight value for each fatigue type, and taking the product of the type weight value and each fatigue value in the fatigue value subset corresponding to the type weight value as the attention component of the corresponding fatigue type when the value is the fatigue value;
and the attention value calculation model construction subunit is used for training to obtain the attention value calculation model by using a machine learning method and taking the fatigue type and the fatigue value as input and the attention component as output.
9. A server, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method recited by any of claims 1-4.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711140773.XA CN107742399B (en) | 2017-11-16 | 2017-11-16 | Method and device for sending alarm signal |
| PCT/CN2018/099158 WO2019095733A1 (en) | 2017-11-16 | 2018-08-07 | Method and device used to emit alarm signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711140773.XA CN107742399B (en) | 2017-11-16 | 2017-11-16 | Method and device for sending alarm signal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107742399A CN107742399A (en) | 2018-02-27 |
| CN107742399B true CN107742399B (en) | 2022-02-22 |
Family
ID=61234884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711140773.XA Active CN107742399B (en) | 2017-11-16 | 2017-11-16 | Method and device for sending alarm signal |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107742399B (en) |
| WO (1) | WO2019095733A1 (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107742399B (en) * | 2017-11-16 | 2022-02-22 | 百度在线网络技术（北京）有限公司 | Method and device for sending alarm signal |
| CN109582529A (en) * | 2018-09-29 | 2019-04-05 | 阿里巴巴集团控股有限公司 | A kind of setting method and device of alarm threshold value |
| CN109191789A (en) * | 2018-10-18 | 2019-01-11 | 斑马网络技术有限公司 | Method for detecting fatigue driving, device, terminal and storage medium |
| CN110191011A (en) * | 2019-04-15 | 2019-08-30 | 厦门科灿信息技术有限公司 | Smart machine monitoring method, device and equipment based on data center's monitoring system |
| CN110458191B (en) * | 2019-07-05 | 2024-04-12 | 中国平安财产保险股份有限公司 | Fatigue state judging method, device, computer equipment and storage medium |
| CN110705502B (en) * | 2019-10-14 | 2023-07-28 | 首约科技(北京)有限公司 | Optimization method of driver monitoring equipment |
| CN111062300A (en) * | 2019-12-11 | 2020-04-24 | 深圳市赛梅斯凯科技有限公司 | Driving state detection method, device, equipment and computer readable storage medium |
| CN112365680A (en) * | 2020-10-29 | 2021-02-12 | 福信富通科技股份有限公司 | Active safety early warning method and system based on AI identification |
| CN113034895A (en) * | 2021-02-04 | 2021-06-25 | 招商局公路网络科技控股股份有限公司 | ETC portal system, and expressway fatigue driving early warning method and device |
| CN113744499B (en) * | 2021-08-12 | 2023-05-30 | 科大讯飞股份有限公司 | Fatigue early warning method, glasses, system and computer readable storage medium |
| CN113460060B (en) * | 2021-08-20 | 2022-12-09 | 武汉霖汐科技有限公司 | Driver fatigue degree evaluation system, control method, and storage medium |
| CN114312800B (en) * | 2022-02-14 | 2022-11-15 | 深圳市发掘科技有限公司 | Vehicle safe driving method and device, computer equipment and storage medium |
| CN114493046A (en) * | 2022-03-17 | 2022-05-13 | 安波福电子(苏州)有限公司 | Fatigue driving prediction method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408879A (en) * | 2014-11-19 | 2015-03-11 | 湖南工学院 | Method, device and system for processing fatigue driving early warning |
| CN104794855A (en) * | 2014-01-22 | 2015-07-22 | 径卫视觉科技（上海）有限公司 | Driver's attention comprehensive assessment system |
| CN105922993A (en) * | 2015-02-27 | 2016-09-07 | 本田技研工业株式会社 | Attention calling system for vehicle |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6927694B1 (en) * | 2001-08-20 | 2005-08-09 | Research Foundation Of The University Of Central Florida | Algorithm for monitoring head/eye motion for driver alertness with one camera |
| CN102509418B (en) * | 2011-10-11 | 2013-11-13 | 东华大学 | Fatigue driving estimation and early-warning method and device of multi-sensor information fusion |
| CN104112334B (en) * | 2013-04-16 | 2017-12-22 | 百度在线网络技术（北京）有限公司 | Fatigue driving method for early warning and system |
| CN103606245B (en) * | 2013-11-08 | 2015-11-18 | 北京工业大学 | Fatigue driving based on Bluetooth brain wave earphone and Android mobile phone detects early warning system |
| US10354539B2 (en) * | 2015-06-08 | 2019-07-16 | Biofli Technologies, Inc. | Situational awareness analysis and fatigue management system |
| CN107742399B (en) * | 2017-11-16 | 2022-02-22 | 百度在线网络技术（北京）有限公司 | Method and device for sending alarm signal |
-
2017
- 2017-11-16 CN CN201711140773.XA patent/CN107742399B/en active Active
-
2018
- 2018-08-07 WO PCT/CN2018/099158 patent/WO2019095733A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794855A (en) * | 2014-01-22 | 2015-07-22 | 径卫视觉科技（上海）有限公司 | Driver's attention comprehensive assessment system |
| CN104408879A (en) * | 2014-11-19 | 2015-03-11 | 湖南工学院 | Method, device and system for processing fatigue driving early warning |
| CN105922993A (en) * | 2015-02-27 | 2016-09-07 | 本田技研工业株式会社 | Attention calling system for vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019095733A1 (en) | 2019-05-23 |
| CN107742399A (en) | 2018-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107742399B (en) | Method and device for sending alarm signal | |
| US20200007638A1 (en) | Method and apparatus for pushing information | |
| CN109902575B (en) | Anti-walking method and device based on unmanned vehicle and related equipment | |
| US11003926B2 (en) | Method and apparatus for recognizing boundary of traffic sign | |
| WO2019047656A1 (en) | Method and apparatus for use in controlling driverless vehicle | |
| US11328518B2 (en) | Method and apparatus for outputting information | |
| US10949687B2 (en) | Method and apparatus for outputting information | |
| CN110889351A (en) | Video detection method and device, terminal equipment and readable storage medium | |
| CN112465049A (en) | Method and device for generating anomaly detection model and method and device for detecting anomaly event | |
| CN110119725B (en) | Method and device for detecting signal lamp | |
| US20200005091A1 (en) | Method and apparatus for image processing for vehicle | |
| CN110097600B (en) | Method and device for identifying traffic sign | |
| CN116932919A (en) | Information pushing method, device, electronic equipment and computer readable medium | |
| CN108334085A (en) | Smart collaboration method, apparatus, system, intelligent terminal and storage medium | |
| CN110853364B (en) | Data monitoring method and device | |
| CN110807394A (en) | Emotion recognition method, test driving experience evaluation method, device, equipment and medium | |
| CN110516760A (en) | Situation identification method, device, terminal and computer readable storage medium | |
| CN110135517B (en) | Method and device for obtaining vehicle similarity | |
| CN110059619B (en) | Automatic alarm method and device based on image recognition | |
| CN115209374B (en) | Motor vehicle alarm system based on third party call center | |
| CN107844777B (en) | Method and apparatus for generating information | |
| CN109410582B (en) | Traffic condition analysis method and terminal equipment | |
| CN112889035B (en) | Test method, test device and terminal equipment | |
| CN111611804A (en) | Danger identification method and device, electronic equipment and storage medium | |
| CN115081545B (en) | Driver rotation identification method and identification model construction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |