DE102020004513A1 - Method for identifying specific behaviors of drivers of vehicles on the basis of camera images - Google Patents

Abstract

The invention relates to a method for identifying specific behaviors of drivers (10) of vehicles (20) on the basis of camera images (40), comprising the steps:
- Acquisition of camera images (40) of a driver (10) of a vehicle (20) by means of a camera (1),
- Identifying postures of the driver (10) in the camera images (40) by a computing device (30);
- Categorizing the body postures on the basis of provided categories by the computing device (30);
- Provision by the computing device (6) of respectively different numerical letter sequences for the categories;
- Creation of a data record (50) by combining the numerical letter sequences of the categories for the body postures by the computing device (30);
- Transmission of the data set (50) for the purpose of analyzing the data set (50) from the computing device to an analysis device (60);
- Analysis of the data set (50) for the purpose of identifying specific behavior patterns of the driver (10) by the analysis device (60).

Description

The invention relates to a method for identifying specific behaviors of drivers of vehicles, a computing device and a Com puterprogram mprodukt.

The US2015186714A discloses a system for monitoring postures of a driver of a vehicle.

• - Erfassen von Kamerabildern eines Fahrers eines Fahrzeugs mittels einer Kamera,
• - Identifizieren von Körperhaltungen des Fahrers in den Kamerabildern durch eine Recheneinrichtung;
• - Kategorisieren der Körperhaltungen anhand von bereitgestellten Kategorien durch die Recheneinrichtung;
• - Bereitstellen von jeweils unterschiedlichen Zahlenbuchstabenfolgen für die Kategorien durch die Recheneinrichtung;
• - Erstellen eines Datensatzes durch Zusammenfügen der Zahlenbuchstabenfolgen der Kategorien für die Körperhaltungen durch die Recheneinrichtung;
• - Übermitteln des Datensatzes zwecks Analyse des Datensatzes von der Recheneinrichtung an eine Analyseeinrichtung;
• - Analysieren des Datensatzes zwecks Identifikation von spezifischen Verhaltensweisen des Fahrers durch die Analyseeinrichtung.

A method according to the invention for identifying specific behaviors of drivers of vehicles on the basis of camera images comprises the steps:

• - Acquisition of camera images of a driver of a vehicle by means of a camera,
• - Identifying postures of the driver in the camera images by a computing device;
• - Categorizing the postures on the basis of provided categories by the computing device;
• - Provision of different numerical letter sequences for the categories by the computing device;
• - Creating a data record by combining the numerical letter sequences of the categories for the postures by the computing device;
• Transmission of the data record for the purpose of analyzing the data record from the computing device to an analysis device;
• - Analysis of the data set for the purpose of identifying specific behavior patterns of the driver by the analysis device.

Because the data record includes categories for categorizing the body postures, the data record includes indirectly categorized body postures of the driver.

This means that the data set is a simple and anonymous representation of the categorized postures of the driver.

The fact that the data record is generated by means of the categorized body postures enables the data record to be analyzed quickly and precisely by the analysis device, which is designed as a neural network, for example. This is made possible without personal data of the driver, such as the camera images, having to be transmitted to the analysis unit.

Furthermore, it is thus made possible that a large number of body postures can be categorized by a large number of drivers. This wide variety of categorized postures can be used to train analytical devices such as neural networks. The use of the analysis devices trained in this way then in turn enables a precise and rapid analysis of the categorized body postures.

Postures include lines of sight, opening corners, eye movements, lip movements, shoulder movements and / or torso movements.

Drivers include vehicle operators and / or vehicle occupants.

The fact that the data record is generated by joining combinations of numbers and letters enables the data record to have a small data size. This enables the data set to be transported simply and easily via mobile networks.

Number and letter sequences preferably include Ascii character sets, in particular Arabic numbers and Latin letters with or without special characters. This enables the data set to be easily processed by a large number of computers used internationally.

The fact that the data set is structured in this way enables analysis devices such as neural networks to process it particularly easily. This in turn enables the driver's postures to be analyzed easily, as a result of which potentially dangerous behavior on the part of the driver can be quickly identified. This enables an increase in safety for the driver and other road users.

Behaviors include a driver's level of awareness of the current traffic situation, an activity of the driver such as eating, drinking and conducting conversations, a level of drowsiness of the driver and / or a level of activity of the driver. Dangerous behaviors are, for example, those that result in an increased risk to the driver and / or other road users.

This enables specific and / or dangerous behavior patterns of the driver to be identified quickly and precisely. This enables an alarm to be triggered at an early stage in the event that dangerous behavior has been identified.

A computing device according to the invention for identifying specific behaviors of drivers of vehicles on the basis of Camera images includes a camera for capturing camera images of a driver of a vehicle. The computing device is designed and set up to identify the driver's postures in the camera images, to categorize the postures on the basis of the categories provided, to create a data record by combining the numerical and letter sequences provided for the categories for the postures and / or to transmit the data record for the purpose of analyzing the data record from the computing device to an analysis device.

A computer program product according to the invention comprises a program which, when it is executed by a computer, causes the computer to carry out a method according to the invention for the automated creation of a data set about specific postures of drivers of vehicles.

The advantages of the method according to the invention result in the same way for the computing device according to the invention and the computer program product according to the invention.

The dependent claims describe advantageous embodiments of the invention.

Preferred exemplary embodiments are explained in more detail with reference to the following figure.

1 shows an embodiment of a computing device according to the invention 30th .

The computing device 30th is a smartphone.

Commercially available smartphones usually include cameras and computers and are available inexpensively in a variety of designs. This enables the computing device according to the invention 5 can only be produced using software adaptations to a standard smartphone.

The computing device 30th includes a camera 1 for capturing camera images 40 of a driver 10 of a vehicle 2 .

In this embodiment, the camera is 1 oriented so that a head of the driver 10 for example in a center of the camera images 40 you can see. This enables a direct recording of the driver's face 10 . The camera images 40 are video recordings of the driver 10 . Alternatively, the acquisition of individual images is also conceivable.

The computing device 30th comprises an infrared camera in this exemplary embodiment 1 , trained and set up to detect light of the infrared spectrum. In this embodiment, the camera 1 and the infrared camera 1 identical. This makes it possible for the driver to move even in dark light conditions 10 with the camera 1 capture. The computing device 30th also includes an infrared light source 2 , trained and set up to emit light in the infrared spectrum. The infrared light source 2 is arranged such that light emitted from the infrared light source 2 from the driver 10 is reflected and off the camera 1 can be captured. This enables the driver to be illuminated 10 with radiation that is visually imperceptible to the driver and therefore not disruptive.

In this exemplary embodiment, the computing device comprises 30th a calculator 5 . The computer 5 is trained and set up, the driver's postures 10 in the camera images 40 to identify.

A first external source provides categories for categorizing the postures.

The categories include lines of sight, eye opening angles, eye movements, lip movements, shoulder movements and / or torso movements.

The computer 5 categorizes the postures based on the categories provided.

In this exemplary embodiment, the categorized body postures include an angle specification for a pupil position, an eye opening angle, a pupil movement trajectory, lip positions, shoulder positions and torso positions. In doing so, markers are provided for the corresponding body parts which enable the named positions to be identified.

A second external source provides different sequences of numbers and letters for the categories. In this embodiment, the first external and the second external source are identical.

In this exemplary embodiment, the number letter sequence provided for the opening angle of the eye is ÖWA1. The identified opening angle is then assigned to this category. In this exemplary embodiment, the opening angle is 105 °.

The computer 5 creates a record 50 by merging the sequence of numbers and letters of the categories for the postures.

In this exemplary embodiment, the external source provides a standardized rule for combining the data record 50 ready. The order in which the individual categories appear is specified. The identified posture is always placed behind it. This results in a standardized data record 50 .

The record 50 becomes an analysis facility 60 for the purpose of analyzing the data set 50 and / or for the purpose of training the analysis device 60 transfer.

The computing device 30th includes a second camera. The second camera is opposite the first camera 1 arranged. This enables that in addition to the images of the driver 10 also pictures of a traffic situation of the vehicle 20th captured at the same time. In this exemplary embodiment, the traffic situation includes a road on which the vehicle is located 20th currently located, as well as other road users who are within 500 m of the vehicle 20th are located. In this exemplary embodiment, categories for categorizing the traffic situation are provided. The recorded traffic situation is categorized based on these categories. The traffic situation categorized in this way is also included in the data record 50 written. This enables the record 50 additionally includes information on the traffic situation, thereby determining the specific behavior of the driver 10 can be brought into connection with the traffic situation. This is an extension of conventional ADAS, which is used by the vehicle 20th are included.

The analysis facility 60 is spatially separated from the vehicle 20th arranged. This analysis facility 60 is a neural network.

This enables the analysis facility 60 is designed as a particularly powerful neural network. This enables precise identification of specific driver behaviors 10 .

This also enables the identification of dangerous behavior on the part of the driver 10 . In the event that dangerous behavior by the driver 10 are identified, the responsible operating units of the driver 10 alarmed. This enables the dangerous behavior to be ended promptly by the driver 10 .

In this exemplary embodiment, specific behaviors include a degree of awareness of the driver for the current road traffic, an activity of the driver such as eating, drinking and having conversations, a degree of drowsiness of the driver and / or a degree of activity of the driver.

In this exemplary embodiment, the neural network analyzes the data set 50 with regard to specific driver behaviors 1 0. The behaviors include a degree of awareness of the driver for the current road traffic, an activity of the driver such as eating, drinking and conducting conversations, a degree of drowsiness of the driver and / or a degree of activity of the driver.

From the entirety of the data set 50 analyzes the neural network that driver in this record 50 is attentive. Thus, the analysis does not reveal any dangerous behavior.

In an alternative exemplary embodiment, the analysis shows that the driver is inattentive. An alarm is then initially triggered on the driver himself. If this does not result in an increase in attention, a responsible superior will be informed of a professional driver. This will increase the driver's awareness 20th enables. In another alternative embodiment, the driver is 10 a young driver whose responsible operations unit is the young driver's parents.

In an alternative embodiment, the analysis device is 60 trained as software. The software is based on a neural network, which is used to identify specific driver behaviors 10 was trained.

This enables the use of low-performance computers 5 to identify specific driver behaviors 10 . Since the software is based on a neural network trained for this purpose, despite the weak computer 5 precise identification of specific driver behaviors 10 enables. By having the calculator 5 Is poor performance, there will be a low power consumption and a small size for the computer 5 enables. This in turn enables problem-free integration of the computer 5 in the vehicle 20th .

The computing device 30th includes a CAN connection device 4th for determining, sending and / or receiving CAN bus data via a CAN bus, the CAN bus being an in-vehicle CAN bus. The CAN bus data is temporally related to the camera images 40 . In this exemplary embodiment, there is a temporal connection when the time at which the camera images are captured 40 does not deviate by more than 5 minutes from the time the CAN bus data was generated.

In this exemplary embodiment, the CAN bus data from the CAN bus are also included in the data record 50 written. These are initially via the CAN connection facility 4th to the computer 5 transfer. The CAN bus data each include a description. For example, the description for the steering angle is “steering angle”. This description also forms a category. The associated data is then categorized accordingly.

The establishment 30th comprises a satellite navigation device 3 to determine the current positions of the vehicle 20th . In this embodiment, the current positions of the vehicle 20th also in the data set 50 recorded. A categorization is carried out using the "Position" category.

A computer program product according to the invention, not shown, comprises a program which, when it is executed by a computer, causes the computer to provide a method according to the invention for identifying specific behaviors of drivers 10 of vehicles 20th based on camera images 40 .

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 2015186714 A [0002]

Claims (10)

Method for identifying specific behaviors of drivers (10) of vehicles (20) on the basis of camera images (40), comprising the steps: - Acquisition of camera images (40) of a driver (10) of a vehicle (20) by means of a camera (1), - Identifying postures of the driver (10) in the camera images (40) by a computing device (30); - Categorizing the body postures on the basis of provided categories by the computing device (30); - Provision by the computing device (6) of respectively different numerical letter sequences for the categories; - Creating a data record (50) by combining the numerical letter sequences of the categories for the body postures by the computing device (30); - Transmission of the data set (50) for the purpose of analyzing the data set (50) from the computing device to an analysis device (60); - Analyzing the data set (50) for the purpose of identifying specific behavior patterns of the driver (10) by the analysis device (60). Procedure according to Claim 1 wherein the analysis device (60) is a neural network. Method according to one of the preceding claims, wherein the analysis device (60) is arranged spatially separated from the vehicle (20). Method according to one of the preceding claims, wherein the analysis device (60) is designed as software, and wherein the software is based on a neural network which has been trained to identify specific behaviors of the driver (10). Procedure according to Claim 4 , wherein the software runs on the computing device (60). Method according to one of the preceding claims, wherein the computing device (60) and / or the camera (1) is a smartphone. Method according to one of the preceding claims, comprising the step: - Acquisition of second camera images of a traffic situation of the vehicle (20) by means of a second camera. Method according to one of the preceding claims, comprising the steps: - Determination of CAN bus data of the vehicle (20) by a determination device (4), the CAN bus data being chronologically related to the camera images (40); - Transmission of the CAN bus data to the computing device (30) for the purpose of taking the CAN bus data into account when creating the data record (50). Computing device (30) for identifying specific behaviors of drivers (10) of vehicles (20) on the basis of camera images (40), comprising - A camera (1) for capturing camera images (40) of a driver (10) of a vehicle (20), - The computing device (30) being designed and set up to identify postures of the driver (10) in the camera images (40), to categorize the postures on the basis of provided categories, to create a data record (50) by combining provided numerical letter sequences of the categories for the postures and / or for transmitting the data set (50) for the purpose of analyzing the data set (50) from the computing device to an analysis device (60). Computer program product, comprising a program which, when executed by a computer, causes the computer to implement a method according to the invention according to one of the Claims 1 to 5 perform.

Images