CN113312958A - Driver state-based order dispatching priority adjusting method and device - Google Patents
Driver state-based order dispatching priority adjusting method and device Download PDFInfo
- Publication number
- CN113312958A CN113312958A CN202110304274.XA CN202110304274A CN113312958A CN 113312958 A CN113312958 A CN 113312958A CN 202110304274 A CN202110304274 A CN 202110304274A CN 113312958 A CN113312958 A CN 113312958A
- Authority
- CN
- China
- Prior art keywords
- driver
- state
- image data
- information
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000008451 emotion Effects 0.000 claims abstract description 39
- 230000008921 facial expression Effects 0.000 claims abstract description 38
- 230000002996 emotional effect Effects 0.000 claims abstract description 27
- 230000001815 facial effect Effects 0.000 claims abstract description 19
- 230000002159 abnormal effect Effects 0.000 claims abstract description 17
- 238000004891 communication Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000000284 resting effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 abstract description 18
- 206010039203 Road traffic accident Diseases 0.000 abstract description 8
- 206010000117 Abnormal behaviour Diseases 0.000 abstract description 6
- 210000003128 head Anatomy 0.000 description 27
- 238000012544 monitoring process Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 230000002829 reductive effect Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 210000001747 pupil Anatomy 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 208000003464 asthenopia Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000008909 emotion recognition Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000004399 eye closure Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06311—Scheduling, planning or task assignment for a person or group
- G06Q10/063114—Status monitoring or status determination for a person or group
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06316—Sequencing of tasks or work
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0645—Rental transactions; Leasing transactions
-
- G06Q50/40—
Abstract
The invention discloses a driver state-based order dispatching priority adjusting method, which comprises the steps of obtaining video data, wherein the video data comprises human eye image data and facial image data of a driver; calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter; carrying out facial expression recognition on the facial image data to obtain facial expression information; judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state; adjusting a driver's dispatch priority strategy based on the fatigue driving state and/or the negative emotional state. The method comprises the steps of acquiring real-time video data in the vehicle ordering service process of the network appointment vehicle, and identifying the state, abnormal behavior and the like of a driver of a vehicle ordering in the vehicle ordering service process based on the real-time video data; the method and the device can feed back and adjust the priority of the order delivery in time when an abnormal state occurs, help to discover hidden risks as soon as possible, and avoid traffic accidents and the like.
Description
Technical Field
The invention belongs to the technical field of network appointment vehicle dispatching, and particularly relates to a dispatching priority adjusting method and device based on a driver state.
Background
The taxi booking service is a taxi booking service which is constructed based on the internet technology, is accessed to vehicles and drivers meeting conditions, and provides taxi booking service by integrating supply and demand information. With the increasing favor of the public in the trip of taking a car, how to improve the service quality of safe driving of a driver also becomes a problem of key attention of a service provider.
The quality of the network booking service is greatly related to the state of a driver, such as the health state of the driver, and whether fatigue directly influences the driving safety. And monitoring and intelligent assessment driver's state help discovering early and hide the risk, avoids the emergence of traffic accident.
The applicant researches and discovers that the prior art has the following defects:
in the prior art, only monitor whether tired to the driver, it shoots driver's eyes through the camera, monitors whether tired to the driver. Unilateral monitoring means have limited reference value, and other information of the driver which is not suitable for continuous service is difficult to identify.
Disclosure of Invention
The present invention is directed to solving the above-mentioned problems, and provides a method and an apparatus for adjusting the priority of a dispatch based on the status of a driver.
In order to solve the problems, the invention is realized according to the following technical scheme:
in a first aspect, the invention provides a driver status-based dispatch priority adjustment method, including:
acquiring video data, wherein the video data comprises human eye image data and facial image data of a driver;
calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter;
carrying out facial expression recognition on the facial image data to obtain facial expression information; judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state;
adjusting a driver's dispatch priority strategy based on the fatigue driving state and/or the negative emotional state.
With reference to the first aspect, the present invention provides a first possible implementation manner of the first aspect, wherein the video data further includes head pose image data of a driver; before the calculation of the driving fatigue parameter PERCOLOS value and the facial expression recognition, the method further comprises the following steps:
performing head posture recognition on the head posture image data to obtain head posture information of a driver;
when the head posture information is judged to be one of the head-up posture, the head-tilted posture and the head-lowering posture, calculating the time under the current posture;
and if the time is greater than the preset time threshold, outputting an abnormal alarm of the driver.
With reference to the first aspect, the present invention provides a first possible implementation manner of the second aspect, where the video data includes voice data, and the method further includes:
recognizing the voice data to obtain emotion information of a driver;
extracting text information from the voice data, and identifying whether the text information contains sensitive words or not; if the text information contains sensitive words, outputting a service abnormity alarm;
adjusting a driver's dispatch priority strategy based on the fatigue driving state, the negative emotional state, and/or the driver emotional information.
With reference to the first aspect, the present invention provides a third possible implementation manner of the first aspect, where the adjusting of the driver's dispatch priority policy includes:
when a fatigue driving state and/or a negative emotional state exists, reducing the dispatch priority of the driver; when a service order exists currently, pushing information for resting after the order is finished; if no service order exists at present, pushing off-line rest information;
and increasing driver's dispatch priority when a fatigue driving state and/or a negative emotional state is not present.
With reference to the first aspect, the present invention provides a first possible implementation manner of the fourth aspect, before acquiring the video data, further including:
and when an online service request of a driver is received, establishing communication with a terminal corresponding to the driver to acquire video data.
In a second aspect, the present invention further provides a dispatch priority adjustment device based on a driver status, including an obtaining module, a processing module and an adjusting module:
the acquisition module is used for acquiring video data, and the video data comprises human eye image data and facial image data of a driver;
the processing module is used for calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter;
performing facial expression recognition on the facial image data to obtain facial expression information; judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state;
the adjusting module is used for adjusting the order dispatching priority strategy of the driver based on the fatigue driving state and/or the negative emotion state.
With reference to the second aspect, the present invention provides a second possible implementation manner of the second aspect, further comprising a recognition module, wherein the video data further comprises head pose image data of a driver; before calculation of a driving fatigue parameter PERCOLOS value and facial expression recognition;
the recognition module is used for recognizing the head posture of the head posture image data to obtain the head posture information of the driver;
when the head posture information is judged to be one of the head-up posture, the head-tilted posture and the head-lowering posture, calculating the time under the current posture;
and if the time is greater than the preset time threshold, outputting an abnormal alarm of the driver.
In combination with the second aspect, the present invention provides a third possible implementation manner of the second aspect, wherein the video data includes voice data;
the processing device is used for identifying the voice data to obtain emotion information of the driver;
extracting text information from the voice data, and identifying whether the text information contains sensitive words or not; if the text information contains sensitive words, outputting a service abnormity alarm;
the adjusting module is used for adjusting the driver's dispatch priority strategy based on the fatigue driving state, the negative emotion state and/or the driver emotion information.
In combination with the second aspect, the present invention provides a third possible implementation manner of the second aspect, wherein the adjusting module is configured to adjust the driver's dispatch priority policy according to the following steps:
when a fatigue driving state and/or a negative emotional state exists, reducing the dispatch priority of the driver; when a service order exists currently, pushing information for resting after the order is finished; if no service order exists at present, pushing off-line rest information;
and increasing driver's dispatch priority when a fatigue driving state and/or a negative emotional state is not present.
With reference to the second aspect, the present invention provides a fourth possible implementation manner of the second aspect, further including a communication module, where the communication module is configured to establish communication with a terminal corresponding to a driver when an online service request of the driver is received before the video data is acquired, so as to acquire the video data.
Compared with the prior art, the invention has the beneficial effects that:
the method comprises the steps of acquiring real-time video data in the vehicle ordering service process of the network appointment vehicle, and identifying the state, abnormal behavior and the like of a driver of a vehicle ordering in the vehicle ordering service process based on the real-time video data; and when the abnormal states such as fatigue driving states, negative emotion states and the like are determined to be contained in the real-time video data, the driver's dispatch priority strategy is adjusted. Based on the video data obtained in the taxi taking service process, the multi-aspect analysis is carried out so as to feed back and adjust the priority of the dispatching order in time when the abnormal state occurs, thereby being beneficial to discovering hidden risks as soon as possible and avoiding traffic accidents and the like.
Drawings
Embodiments of the invention are described in further detail below with reference to the attached drawing figures, wherein:
FIG. 1 is a schematic diagram of a driver status-based dispatch priority adjustment method and apparatus according to the present invention;
FIG. 2 is a flow chart diagram of a method of driver status based dispatch priority adjustment of the present invention;
FIG. 3 is a flow chart of head pose recognition of the present invention;
FIG. 4 is a flow chart illustrating the calculation of the driving fatigue parameter PERCOLOS value according to the present invention;
FIG. 5 is a block diagram of the driver status based dispatch priority adjustment mechanism of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
The described embodiments are only some embodiments of the present application and not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
To enable those skilled in the art to utilize the present disclosure, the following embodiments are presented in conjunction with a specific application scenario, "net appointment taxi taking scenario". It will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the application. Although the present application is described primarily in the context of a "net appointment taxi taking scenario," it should be understood that this is only one exemplary embodiment. The application can be applied to any other transportation means, and can comprise a taxi, a private car, a windward vehicle, a bus and the like, or any combination thereof. The present application may also include any service system for network taxi appointment.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
In order to solve at least one technical problem in the background of the present application, an embodiment of the present application provides a method and an apparatus for adjusting a dispatch priority based on a driver status, which can identify a status, an abnormal behavior, and the like of a driver using a vehicle order in a vehicle order service process by acquiring real-time video data in a vehicle order service process of a network appointment; and when the abnormal states such as fatigue driving states, negative emotion states and the like are determined to be contained in the real-time video data, the driver's dispatch priority strategy is adjusted. Based on the video data obtained in the taxi taking service process, the method and the system perform multi-aspect analysis to timely feed back and adjust the order dispatching priority when the abnormal state occurs, help to discover hidden risks as soon as possible and avoid traffic accidents and the like, and explain the technical scheme of the application through possible implementation modes.
All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Example one
Fig. 1 is a schematic diagram of a dispatch priority adjustment method and device based on driver status according to the present application. For example, the system for driver status based dispatch priority adjustment may be an online transportation service platform relied upon for transportation services such as a net appointment, taxi, designated driving service, express service, carpooling service, bus service, driver rental service, or shift service, or a combination of any of the above.
The system for adjusting the dispatch priority based on the driver state can comprise a service end, a network, a service request terminal (user end), a service providing terminal (driver end), wherein the service end can comprise a processor for executing instruction operation, and the like.
In addition, the system for adjusting the dispatching priority based on the driver state can further comprise a monitoring terminal, wherein the monitoring terminal can be a vehicle data recorder with an image acquisition function, or a video shooting device such as a camera arranged in the vehicle, or a smart phone provided with a transportation service platform application program. The monitoring terminal can be in communication connection with the server so as to send the collected video data to the server.
In addition, the service end in the system can be in communication connection with a customer service platform, and the customer service platform can be a background for providing travel customer service. While one possible example of the system for monitoring and warning the abnormal behavior of the networked car booking driver shown in fig. 1 is shown, in other possible embodiments, the system for monitoring and warning the abnormal behavior of the networked car booking driver may include only one of the components shown in fig. 1 or may also include other components.
In some embodiments, the server may be a single server or a server group. The service end can access and store the information in a service request terminal (user end), a service providing terminal (driver end) and a monitoring terminal through a network. In some embodiments, the server may be implemented on a cloud platform; by way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud (community cloud), a distributed cloud, an inter-cloud, a multi-cloud, and the like, or any combination thereof.
In some embodiments, the service request terminal (user terminal), the service providing terminal (driver terminal) may be a smartphone or a tablet computer.
In one embodiment, the service request terminal (user terminal) is used for placing a vehicle order. The service providing terminal (a driver terminal) or the monitoring terminal is used for acquiring real-time video data in the vehicle order service process; the service providing terminal (the driver end) or the monitoring terminal uploads the real-time video data to the server end through the network. The server side obtains real-time video data, and identifies the state, abnormal behavior and the like of a driver using the order in the order service process based on the real-time video data. And when the video data is determined to contain the fatigue driving state, the negative emotion state, the driver emotion information and the driver abnormity, adjusting the driver's dispatch priority strategy or outputting the driver abnormity alarm. Sending the driver exception alert may include sending to a customer service request terminal (user side), a service providing terminal (driver side), or a customer service platform.
In another embodiment, the service request terminal (user side), the service providing terminal (driver side) comprises a processor. The processor may process information and/or data from the in-vehicle order service process to perform one or more of the functions described herein.
Example two
Fig. 2 is a flowchart illustrating a method for adjusting a driver-state-based dispatch priority according to the present application, which can be applied to the server or the driver end.
It should be understood that in other embodiments, the order of some steps in the method for adjusting the dispatch priority based on the driver status according to the embodiment may be interchanged according to actual needs, or some steps may be omitted or deleted. The detailed steps of the method for driver status based dispatch priority adjustment are described below.
The application aims to provide a method for acquiring video data in a driver service process so as to detect whether the driver of a vehicle has abnormal states, behaviors and the like. And timely feedback and adjustment of the order dispatching priority are carried out when an abnormal state occurs, so that the hidden risk can be found as soon as possible, and traffic accidents and the like can be avoided. And prompt information can be sent to a driver end, a user end and a customer service platform of the vehicle service so as to achieve the warning effect on a vehicle driver and the reminding effect on a user, and the customer service platform timely contacts the user to confirm the safety of the user and make a response measure. When the risk degree is higher, still can be automatic outside relevant department propelling movement alarm information.
The second dispatch priority adjustment method based on the driver state provided by the embodiment of the application comprises the following steps:
s100: video data including human eye image data and facial image data of a driver is acquired.
S200, calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter.
In the invention, the eye state of the driver is recognized by collecting the eye image data of the driver, the driving fatigue parameter PERCOLOS value is calculated by using the eye state data, and the fatigue driving state is judged when the driving fatigue parameter PERCOLOS value is larger than the preset parameter. The visual fatigue detection mode is used for detecting the eye fatigue state of the driver, so that the interference on the driving behavior of the driver can be reduced; and detecting the fatigue driving state of the driver in real time through video streaming.
S300, carrying out facial expression recognition on the facial image data to obtain facial expression information; and judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state.
The traffic accident occurs due to various causes, generally, human factors, vehicle factors, road and environmental factors, etc., in which the human factors are the top. The human factor is most important, and the driver factor is the most important factor. The bad emotion of the driver is an important cause of the traffic accident. When stimulated, the general people show through the face, actions, eye spirit and the like. In the driving process of a vehicle, a driver can feel happy and satisfied, and the driving vehicle feels comfortable and psychological comfort, has positive promotion effect on observation and judgment of things, often shows strong sensibility, is easy to observe, has quick response, accurate judgment and quick action, and is favorable for the driving safety of the vehicle. On the other hand, if emotional fluctuation such as worry, fear, worries, anger and the like occurs, the feeling force is reduced, the energy is dispersed, the driver is reluctant to observe thinking and drive to get into the bold atmosphere, and when an emergency and an emergency happen, the adverse behaviors such as slow response, misjudgment and the like occur. Therefore, it is very important to ensure safe driving, detect changes in the face of the driver in time to determine emotional changes, prompt the driver to maintain emotional stability, and apply corresponding vehicle control.
In the invention, whether the emotion of the driver is stable is judged based on the micro expression of the driver, and the emotion judgment is realized by acquiring the facial expression of the driver, so that the current emotion of the driver is reflected more accurately. Furthermore, the current driving state of a driver can be reflected together with voice data, unsafe factors in the driving process can be timely and accurately captured by combining the data in the two aspects, evasive measures can be timely taken, potential safety hazards are reduced, and driving safety is guaranteed.
In one implementation, the server may perform recognition processing on the facial image data through a pre-established machine learning recognition model to obtain facial expression information representing emotion. Such as no expression, happy feeling, extreme happy feeling, angry, extreme angry, fright, extreme fright, fear, extreme fright, disgust, extreme disgust, sadness, extreme sadness, etc.
Specifically, the video data includes a plurality of image frames, and the probability values of a plurality of facial expression information obtained in the corresponding machine learning identification model take the facial expression information with the maximum probability value as the target emotion. It should be noted that the identification may be performed according to the time sequence in which the face image data is acquired.
Specifically, in this embodiment, preferably, the video data includes voice data, and the method for adjusting the dispatch priority based on the driver status further includes:
and S310, recognizing the voice data to obtain the emotional information of the driver.
The voice data is analyzed by acquiring the current voice data of the driver. When the order is executed, the current voice data of the driver can be obtained, and the current inclination information of the driver, such as excitement, anger, joy, sadness and the like, is reflected through the voice data. In the actual service process, the voice data may be communication between the driver and the passenger during the service process or other conversation contents of the driver.
In one specific implementation, the speech data can be identified by using an algorithm strategy of acoustic feature classification and identification stored in a server, and the emotion state of the current user is discriminated by comprehensively analyzing feature values of tone, volume, speed, vocal tract features and the like in the speech. Are prior art and may be implemented by those skilled in the art without undue experimentation.
S320: extracting text information from the voice data, and identifying whether the text information contains sensitive words or not; and if the text information contains sensitive words, outputting a service abnormity alarm.
In the embodiment, the voice data is converted into the text information, and sensitive words such as violence tendency, unhealthy colored words or non-civilized words in the text information are identified through the keywords. When the detection is identified, the service abnormity alarm is output to a customer service platform, and the alarm is given by using a service request terminal (user terminal) and a service providing terminal (driver terminal).
Furthermore, the emotion recognition method based on the facial features and the voice features can improve the accuracy of emotion recognition of the driver, so that a foundation is laid for reminding the driver of driving reasonably and avoiding potential traffic accidents.
S400: adjusting a driver's dispatch priority strategy based on the fatigue driving state, the negative emotional state, and/or the driver emotional information.
In the present invention, the strategy for adjusting the driver's dispatch priority includes:
reducing driver's dispatch priority when a fatigue driving state, a negative emotional state, and/or driver emotional information is present; when a service order exists currently, pushing information for resting after the order is finished; if no service order exists at present, pushing off-line rest information; and increasing driver's dispatch priority when a fatigue driving state, a negative emotional state, and/or driver emotional information is not present.
Specifically, the adjustment of the priority of the dispatch for increasing or decreasing the design is continued until the driver is off-line from the service providing terminal (driver terminal) or within a specified time period after the driver is off-line, and no service is provided.
Specifically, before acquiring the video data, the method further includes:
and when an online service request of a driver is received, establishing communication with an intelligent terminal corresponding to the driver to acquire video data.
In this embodiment, the intelligent terminal may be a service providing terminal (a driver end), a camera, a vehicle-mounted terminal, or the like.
EXAMPLE III
As shown in fig. 3, the present application further provides an implementation of a dispatch priority adjustment method based on a driver status, wherein the video data further includes head pose image data of the driver; before the calculation of the driving fatigue parameter PERCOLOS value and the facial expression recognition, the method further comprises the following steps:
s10: and performing head posture recognition on the head posture image data to obtain the head posture information of the driver.
In one embodiment, the two-dimensional contour and position information of the above parts are obtained by recognizing two-dimensional contour and position information of a plurality of key facial features of the driver in the head state image data, wherein the plurality of facial features includes a left eye part, a right eye part, a nose part, a left ear part, a right ear part, a face contour, and the like.
Furthermore, the two-dimensional outline and the position information of a plurality of facial features of the driver are compared with the preset standard face information, and then a comparison result can be obtained. The head posture information includes a front view state, a head-up posture, a head-tilted posture, and a head-down posture.
Specifically, the preset standard face information is a standard head state in front of the face front view.
And in advance, the face information of the preset standard is stored in the server.
S20: when it is determined that the head posture information is one of the head-up posture, the head-tilt posture, and the head-tilt posture, a time at the current posture is calculated.
When the head posture information of the driver is detected to be one of the head-up posture, the head-tilt posture and the head-down posture, the staying time in the head posture is counted. Through the design, abnormal alarm triggered by normal head movement of a driver is avoided, and detection precision and efficiency are improved. Meanwhile, when the driver lowers or raises his head, the server can not detect the fatigue driving state in the video data any more, and the problem that the driver can not recognize the fatigue driving state exists.
S30: and if the time is greater than the preset time threshold, outputting an abnormal alarm of the driver.
In one implementation, in the service process, the running data of the service providing terminal (driver terminal) is obtained, and when the vehicle is judged to run based on the running data, the time under the current posture is calculated: when the head-up posture stays for more than 5s, outputting abnormal alarm of the driver to the customer service platform, and giving an alarm by using a service request terminal (user terminal), a service providing terminal (driver terminal) and the like. When the head-bending posture stays for more than 5s, outputting abnormal alarm of the driver to a customer service platform, and giving an alarm by using a service request terminal (a user terminal), a service providing terminal (a driver terminal) and the like. When the head-lowering posture stays for more than 3s, an abnormal alarm of the driver is output to the customer service platform, and the alarm is given by using a service request terminal (a user terminal), a service providing terminal (a driver terminal) and the like. The travel data includes speed, position information, and the like.
Example four
As shown in fig. 4, an embodiment of the present application further provides an implementation manner of a dispatch priority adjustment method based on a driver state, where the human eye image data is subjected to a driving fatigue parameter PERCOLOS value calculation, and when the driving fatigue parameter PERCOLOS value is greater than a preset parameter, it is determined that the driver state is a fatigue driving state, and the method specifically includes:
s211: processing the video data to obtain human eye image data;
in a preferred implementation, the image frames of the video data are cropped to cut out the image data of the eyes. The human eye image data is an image frame of the eyes. By clipping, the throughput of the video stream can be reduced.
S212: it is recognized that the eyes in the image data of the human eyes are open or closed.
Specifically, the collected human eye image data is classified into two types of open and closed according to the degree of opening of the eyes. Wherein, the opening degree of the pupil of the eye is set to be more than 20 percent and the opening degree of the pupil of the eye is set to be less than or equal to 20 percent and the opening degree of the pupil of the eye is set to be closed.
S213: and calculating a driving fatigue parameter PERCOLOS value of the human eye image data.
The eye closure is a measurement of the ratio of the time that the eye is closed to the time that the eye is open over time, which is calculated from the time of image processing for each frame. The calculation of the values of the driving fatigue parameters PERCOLOS is known to those skilled in the art and will not be described herein in too much.
EXAMPLE five
As shown in fig. 5, the invention further provides a dispatch priority adjusting device based on the driver status, which comprises an obtaining module, a processing module, an adjusting module and a communication module.
The acquisition module is used for acquiring video data, wherein the video data comprises human eye image data, facial image data and voice data of a driver;
the processing module is used for calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter;
performing facial expression recognition on the facial image data to obtain facial expression information; judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state;
the adjusting module is used for adjusting the order dispatching priority strategy of the driver based on the fatigue driving state and/or the negative emotion state;
the communication module is used for establishing communication with a terminal corresponding to a driver when an online service request of the driver is received before the video data is acquired so as to acquire the video data.
The processing device is used for identifying the voice data to obtain emotion information of a driver;
extracting text information from the voice data, and identifying whether the text information contains sensitive words or not; if the text information contains sensitive words, outputting a service abnormity alarm;
the adjusting module is used for adjusting the driver's dispatch priority strategy based on the fatigue driving state, the negative emotion state and/or the driver emotion information.
The embodiment preferably comprises a recognition module, and the video data further comprises head pose image data of a driver; before calculation of a driving fatigue parameter PERCOLOS value and facial expression recognition;
the recognition module is used for recognizing the head posture of the head posture image data to obtain the head posture information of the driver;
when the head posture information is judged to be one of the head-up posture, the head-tilted posture and the head-lowering posture, calculating the time under the current posture;
and if the time is greater than the preset time threshold, outputting an abnormal alarm of the driver.
Further, the adjusting module is configured to adjust the driver's dispatch priority policy according to the following steps, including:
when a fatigue driving state and/or a negative emotional state exists, reducing the dispatch priority of the driver; when a service order exists currently, pushing information for resting after the order is finished; if no service order exists at present, pushing off-line rest information;
and increasing driver's dispatch priority when a fatigue driving state and/or a negative emotional state is not present.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, so that any modification, equivalent change and modification made to the above embodiment according to the technical spirit of the present invention are within the scope of the technical solution of the present invention.
Claims (10)
1. A driver state-based dispatch priority adjustment method is characterized by comprising the following steps:
acquiring video data, wherein the video data comprises human eye image data and facial image data of a driver;
calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter;
carrying out facial expression recognition on the facial image data to obtain facial expression information; judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state;
adjusting a driver's dispatch priority strategy based on the fatigue driving state and/or the negative emotional state.
2. The driver status-based dispatch priority adjustment method of claim 1, wherein the video data further comprises driver head pose image data; before the calculation of the driving fatigue parameter PERCOLOS value and the facial expression recognition, the method further comprises the following steps:
performing head posture recognition on the head posture image data to obtain head posture information of a driver;
when the head posture information is judged to be one of the head-up posture, the head-tilted posture and the head-lowering posture, calculating the time under the current posture;
and if the time is greater than the preset time threshold, outputting an abnormal alarm of the driver.
3. The driver status-based dispatch priority adjustment method as claimed in claim 1, wherein said video data comprises voice data, further comprising:
recognizing the voice data to obtain emotion information of a driver;
extracting text information from the voice data, and identifying whether the text information contains sensitive words or not; if the text information contains sensitive words, outputting a service abnormity alarm;
adjusting a driver's dispatch priority strategy based on the fatigue driving state, the negative emotional state, and/or the driver emotional information.
4. The driver status-based dispatch priority adjustment method of any one of claims 1-3, wherein the adjusting the driver's dispatch priority policy comprises:
when a fatigue driving state and/or a negative emotional state exists, reducing the dispatch priority of the driver; when a service order exists currently, pushing information for resting after the order is finished; if no service order exists at present, pushing off-line rest information;
and increasing driver's dispatch priority when a fatigue driving state and/or a negative emotional state is not present.
5. The driver status-based dispatch priority adjustment method of any one of claims 1-3, further comprising, prior to obtaining video data:
and when an online service request of a driver is received, establishing communication with an intelligent terminal corresponding to the driver to acquire video data.
6. The utility model provides a group's priority adjusting device based on driver's state which characterized in that, includes acquisition module, processing module and adjusting module:
the acquisition module is used for acquiring video data, and the video data comprises human eye image data and facial image data of a driver;
the processing module is used for calculating a driving fatigue parameter PERCOLOS value of the human eye image data, and judging the human eye image data to be in a fatigue driving state when the driving fatigue parameter PERCOLOS value is larger than a preset parameter;
performing facial expression recognition on the facial image data to obtain facial expression information; judging whether the facial expression information contains negative emotions or not, and if so, determining the facial expression information to be in a negative emotion state;
the adjusting module is used for adjusting the order dispatching priority strategy of the driver based on the fatigue driving state and/or the negative emotion state.
7. The driver status-based dispatch priority adjustment device of claim 6, further comprising a recognition module, the video data further comprising driver head pose image data; before calculation of a driving fatigue parameter PERCOLOS value and facial expression recognition;
the recognition module is used for recognizing the head posture of the head posture image data to obtain the head posture information of the driver;
when the head posture information is judged to be one of the head-up posture, the head-tilted posture and the head-lowering posture, calculating the time under the current posture;
and if the time is greater than the preset time threshold, outputting an abnormal alarm of the driver.
8. The driver status-based dispatch priority adjustment device of claim 6, wherein the video data comprises voice data;
the processing device is used for identifying the voice data to obtain emotion information of the driver;
extracting text information from the voice data, and identifying whether the text information contains sensitive words or not; if the text information contains sensitive words, outputting a service abnormity alarm;
the adjusting module is used for adjusting the driver's dispatch priority strategy based on the fatigue driving state, the negative emotion state and/or the driver emotion information.
9. The driver status-based dispatch priority adjustment device of any one of claims 6-8, wherein the adjustment module is configured to adjust the driver's dispatch priority policy according to the following steps, comprising:
when a fatigue driving state and/or a negative emotional state exists, reducing the dispatch priority of the driver; when a service order exists currently, pushing information for resting after the order is finished; if no service order exists at present, pushing off-line rest information;
and increasing driver's dispatch priority when a fatigue driving state and/or a negative emotional state is not present.
10. The driver status-based dispatch priority adjustment device as claimed in any one of claims 6 to 8, further comprising a communication module, wherein the communication module is configured to establish communication with a terminal corresponding to the driver to obtain the video data when an online service request of the driver is received before the video data is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110304274.XA CN113312958B (en) | 2021-03-22 | 2021-03-22 | Method and device for adjusting dispatch priority based on driver state |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110304274.XA CN113312958B (en) | 2021-03-22 | 2021-03-22 | Method and device for adjusting dispatch priority based on driver state |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113312958A true CN113312958A (en) | 2021-08-27 |
| CN113312958B CN113312958B (en) | 2024-04-12 |
Family
ID=77372164
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110304274.XA Active CN113312958B (en) | 2021-03-22 | 2021-03-22 | Method and device for adjusting dispatch priority based on driver state |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113312958B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863636A (en) * | 2022-03-25 | 2022-08-05 | 吉林云帆智能工程有限公司 | Emotion recognition algorithm for rail vehicle driver |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028068A1 (en) * | 2016-08-12 | 2018-02-15 | 深圳市元征科技股份有限公司 | Fatigue driving monitoring method and cloud server |
| CN107918826A (en) * | 2017-11-13 | 2018-04-17 | 北京航空航天大学 | The driver's evaluation and dispatching method that a kind of driving environment perceives |
| WO2019232973A1 (en) * | 2018-06-04 | 2019-12-12 | 上海商汤智能科技有限公司 | Vehicle control method and system, and in-vehicle intelligent system, electronic device and medium |
| CN112183245A (en) * | 2020-09-11 | 2021-01-05 | 广州宸祺出行科技有限公司 | Method and device for monitoring abnormal behaviors of taxi appointment driver of network and giving alarm and electronic equipment |
| CN112288286A (en) * | 2020-10-30 | 2021-01-29 | 上海仙塔智能科技有限公司 | Secure dispatching method, secure dispatching system and readable storage medium |
-
2021
- 2021-03-22 CN CN202110304274.XA patent/CN113312958B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028068A1 (en) * | 2016-08-12 | 2018-02-15 | 深圳市元征科技股份有限公司 | Fatigue driving monitoring method and cloud server |
| CN107918826A (en) * | 2017-11-13 | 2018-04-17 | 北京航空航天大学 | The driver's evaluation and dispatching method that a kind of driving environment perceives |
| WO2019232973A1 (en) * | 2018-06-04 | 2019-12-12 | 上海商汤智能科技有限公司 | Vehicle control method and system, and in-vehicle intelligent system, electronic device and medium |
| CN112183245A (en) * | 2020-09-11 | 2021-01-05 | 广州宸祺出行科技有限公司 | Method and device for monitoring abnormal behaviors of taxi appointment driver of network and giving alarm and electronic equipment |
| CN112288286A (en) * | 2020-10-30 | 2021-01-29 | 上海仙塔智能科技有限公司 | Secure dispatching method, secure dispatching system and readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 蔡木生;周航;: "汽车疲劳驾驶的检测与监视系统设计", 单片机与嵌入式系统应用, no. 06 * |
| 郭孜政;刘仙;陈瑞雅;张骏;: "疲劳影响驾驶员心理旋转能力的ERP试验研究", 中国安全科学学报, no. 10 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863636A (en) * | 2022-03-25 | 2022-08-05 | 吉林云帆智能工程有限公司 | Emotion recognition algorithm for rail vehicle driver |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113312958B (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11249544B2 (en) | Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness | |
| CN109664891A (en) | Auxiliary driving method, device, equipment and storage medium | |
| CN110395260B (en) | Vehicle, safe driving method and device | |
| CN110143202A (en) | A kind of dangerous driving identification and method for early warning and system | |
| CN103871200A (en) | Safety warning system and method used for automobile driving | |
| CN111027478A (en) | Driver and passenger behavior analysis and early warning system based on deep learning | |
| CN110766912A (en) | Driving early warning method, device and computer readable storage medium | |
| CN109448317A (en) | Taxi automatic alarm system | |
| Lashkov et al. | Driver dangerous state detection based on OpenCV & dlib libraries using mobile video processing | |
| EP4042322A1 (en) | Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness | |
| CN113312958A (en) | Driver state-based order dispatching priority adjusting method and device | |
| CN114987500A (en) | Driver state monitoring method, terminal device and storage medium | |
| CN112550306A (en) | Vehicle driving assistance system, vehicle including the same, and corresponding method and medium | |
| US11881036B2 (en) | Devices and methods for detecting drowsiness of drivers of vehicles | |
| KR20120086119A (en) | Visual attention apparatus and control method based on mind aware and display apparatus used visual attention apparatus | |
| CN112562267A (en) | Vehicle-mounted safety robot and safe driving assistance method | |
| JP2014238707A (en) | Driver state determination system | |
| CN116588115A (en) | Vehicle safety system based on driver state analysis | |
| CN113561988A (en) | Voice control method based on sight tracking, automobile and readable storage medium | |
| Wang et al. | Enhancing YOLOv7-Based Fatigue Driving Detection through the Integration of Coordinate Attention Mechanism | |
| WO2021262166A1 (en) | Operator evaluation and vehicle control based on eyewear data | |
| Berri et al. | A 3D vision system for detecting use of mobile phones while driving | |
| CN110838027A (en) | Method and device for determining vehicle use satisfaction degree, storage medium and computing equipment | |
| US11961395B2 (en) | Driving assistance device, driving assistance method, and storage medium in which program is stored | |
| EP4332885A1 (en) | Electronic device, control method for electronic device, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |