DE102021005092A1 - Method and device for recognizing emotional states of occupants of a vehicle - Google Patents

Abstract

The invention relates to a method for recognizing emotional states of occupants of a vehicle, in which, in a predetermined period of time while the vehicle (1) is being driven by an interaction, emotions of a vehicle occupant (15, 17, 19) triggered optically and / or acoustically and / or haptically detected and analyzed. In one method, activities of several vehicle occupants (15, 17, 19) in the vehicle interior (7) are recorded and the emotion processes of the individual vehicle occupants (15, 17, 19) determined over the specified period of time, the sensor-recorded changes in the vehicle interior (7) are recorded for the individual vehicle occupants (15, 17, 19).

Description

The invention relates to a method for recognizing emotional states of occupants of a vehicle, in which emotions of a vehicle occupant triggered by an interaction are optically and / or acoustically and / or haptically detected and analyzed in a predetermined period of time while the vehicle is driving the vehicle, as well as a device for Implementation of the procedure.

the DE 10 2020 100 497 A1 discloses systems and methods of real-time vehicle-based analysis and uses thereof. In this method, at least images, videos and audios of a user are collected while the vehicle is being operated. These images, videos, and audios are analyzed to determine the user's emotions and feelings when interacting with one or more features of the vehicle. User actions are identified that precede a point in time at which the emotion or feeling of the user was recognized. When these emotions and feelings are negative, their correlating vehicle interactions are interpreted by vehicle designers as potential areas for vehicle improvement.

The object of the invention is to specify a method and a device for recognizing emotional states and their temporal changes and relationships between the changes in the emotional states of vehicle occupants, in which the described solution is improved.

The invention results from the features of the independent claims. Advantageous further developments and refinements are the subject matter of the dependent claims. Further features, possible applications and advantages of the invention emerge from the following description and the explanation of exemplary embodiments of the invention which are shown in the figures.

The object is achieved with the subject matter of claim 1.

In the method explained at the outset, activities of several vehicle occupants in the vehicle interior are recorded and the emotion processes of the individual occupants determined over the specified period of time with sensor-recorded changes in the vehicle interior are assigned to the individual vehicle occupants. This has the advantage that the interaction between the vehicle occupants is improved and thus an increase in the well-being of the vehicle occupants is achieved.

Advantageously, the emotion profiles together with the changes recorded by sensors are determined anonymously by means of statistical methods and / or methods of machine learning and the anonymized data are assigned to the vehicle occupants. The evaluation methods used allow a reliable assignment of the determined emotion profiles to the respective vehicle occupants. By anonymizing the data obtained, data protection is also taken into account.

In one embodiment, interrelationships between the vehicle occupants and their cause and / or dependency are determined by means of the emotion profiles ascertained in the predetermined period of time. On the basis of the identification of the cause, it is possible to work towards improving the well-being of the vehicle occupants in a targeted manner.

In one variant, on the basis of the identified interrelationships between the vehicle occupants, vehicle-internal and / or vehicle-external applications are adapted. Automatic changes to vehicle applications make it particularly easy to increase the well-being of the vehicle occupants.

In one embodiment, the vehicle occupants are informed of the identified interrelationships. The vehicle occupants now know that the vehicle properties will be automatically adapted to improve their wellbeing in a timely manner, so that manual intervention is not necessary.

Another aspect of the invention relates to a device for recognizing emotional states of occupants of a vehicle, comprising a control unit which is connected to an emotion recognition module for detecting the current emotional state of at least one vehicle occupant at predetermined times / or optical and / or haptic sensors is coupled. In one device, to improve the well-being of the vehicle occupants, an activity detection module for detecting a physical action by the vehicle occupants is routed to the control unit to determine time-dependent emotional states of the individual vehicle occupants and their interrelationships.

It is advantageous if the control unit receives further signals from a central vehicle control unit to compare the changes in the determined emotional states with general vehicle functions and their changes. Thus, the determined emotional states can correspond to the current one Vehicle functions are assigned, from which an analysis is possible, which vehicle functions lead to which emotional states of the individual vehicle occupants.

In a further embodiment, a memory unit for the central storage of the anonymized emotion profiles of the vehicle occupants is part of the control unit or positioned outside the vehicle and connected wirelessly to the control unit. This ensures that information once obtained for setting the vehicle functions can be called up by the vehicle occupants during further journeys.

Further advantages, features and details emerge from the following description, in which at least one exemplary embodiment is described in detail - possibly with reference to the drawing. Described and / or graphically represented features can form the subject matter of the invention individually or in any meaningful combination, possibly also independently of the claims, and in particular can also be the subject matter of one or more separate applications. Identical, similar and / or functionally identical parts are provided with the same reference symbols.

• 1 ein Ausführungsbeispiel einer erfindungsgemäßen Vorrichtung zur Durchführung des erfindungsgemäßen Verfahrens.

It shows:

• 1 an embodiment of a device according to the invention for performing the method according to the invention.

In 1 an embodiment of a device according to the invention is shown, such as this in a vehicle 1 for the detection of emotional states of vehicle occupants 1 is used. The device comprises a control unit 3 that with an emotion detection module 5 is connected, which in turn with in the vehicle interior 7th arranged optical sensors 9 , acoustic sensors 11 and haptic sensors 13th is coupled, which is close to the vehicle occupants 15th , 17th , 19th , the number of which can be variable, are positioned. The optical sensors 9 , for example a camera, the facial expressions and gestures of the vehicle occupants 15th , 17th , 19th capture while the acoustic sensors 11 the intonation of the voices of the vehicle occupants 15th , 17th , 19th to the control unit 3 can transfer. Haptic sensors 13th can transfer physiological measured values, which for example determine the skin resistance of the vehicle occupants, to the control unit 3 forward onto.

Furthermore, an activity detection module performs 21 to the control unit 3 which, for example, indicate the viewing directions, a physical interaction or expressions of the vehicle occupants 15th , 17th , 19th recorded. At the same time is a central vehicle control unit 23 with the control unit 3 for the transmission of additional further input signals by the vehicle occupants 15th , 17th , 19th tied together. The control unit 3 is wireless with an external storage unit 25th , for example in the form of a cloud, coupled with access to external vehicle functions 27 Has.

The control unit 3 continuously detects the signals from the emotion recognition module 5 , the activity detection module 21 and the central vehicle control unit 23 , where the emotion detection module 5 the current emotional state of the various vehicle occupants based on the sensor signals 15th , 17th , 19th in a given period of time while the vehicle is in motion 1 certainly. The control unit uses these input signals to determine 3 the connection between the changes in the emotional states of the vehicle occupants 15th , 17th , 19th , ie the time course of the change in an emotion of a vehicle occupant 15th becomes with the interactions of other vehicle occupants 17th , 19th in the vehicle 1 as well as the changes in emotional states of other vehicle occupants 17th , 19th and general vehicle functions and their changes. These history patterns are anonymized centrally in the storage unit 25th saved. By means of clustering methods and distance measures, significant values are determined, from which it is derived which with the recorded activity recognition module 21 determined activities and / or changes in the emotional states of the vehicle occupants 15th , 17th , 19th and / or further input parameters of the central vehicle control unit 23 Reason for the change in emotions of the vehicle occupant 15th was. Based on these results, external vehicle functions 27 or in-vehicle functions via the central vehicle control unit 23 be adjusted.

In detail, the control unit has 3 using a technical process that records emotional changes and using additional input parameters of the activity determination module 21 and the central vehicle control unit 23 evaluates with regard to their interaction. Methods of machine learning, deep learning, root cause analysis or causal interference can be used. In particular, it is thus possible to monitor signals and / or time courses from the central vehicle control unit 23 and / or the activity detection module 21 to recognize which are decisive for the emotional change of the vehicle occupant 15th was.

For example, two vehicle occupants are driving 15th , 17th in a vehicle 1 , their respective emotional states E1, E2 are recorded over time t. If the states E1, E2 change negatively or positively, the control unit determines 3 the temporal course of emotions E1 (t), E2 (t) as well as at the same time or at Parameters of the activity detection module that took place at a previous point in time 21 and the central vehicle control unit 23 to detect connections and correlations. In particular, it can be recorded here whether, for example, there is an interaction between the two vehicle occupants 15th , 17th have led to their emotional states E1, E2 having changed.

An example of this is a scenario for a chauffeur service that uses separate services to access functions external to the vehicle 27 applied application is bookable. This is the control unit 3 able to recognize a pattern in the event that the driver of the chauffeur service assumes a certain driving behavior, which is via the central vehicle control unit 23 and the activity detection module 21 is recognized.

This leads to the emotional state of the chauffeured vehicle occupant 17th changed positively or negatively over the course of time t.

Because these changes in the storage unit 25th are stored, they can thus be evaluated centrally, whereby causal or correlating relationships between the driver and the person being chauffeured 17th can be derived. For future journeys, the chauffeur service application can, for example, determine whether the combination of driver and vehicle occupant is being chauffeured 17th to positive or negative emotional changes in the state of the respective vehicle occupant 17th leads.

Viewed generalized, preferences of the vehicle occupant can thus 17th recorded and suggested via the Chaffeurservice application or preferences of the vehicle occupant 17th recorded and via the central vehicle control unit 23 are proposed or relationships between different vehicle occupants 15th , 17th , 19th can be recognized and via the application or the central vehicle control unit 23 to the vehicle occupants 15th , 17th , 19th be communicated.

List of reference symbols

1

vehicle

3

Control unit

5

Emotion detection module

7th

Vehicle interior

9

optical sensors

11

acoustic sensors

13th

haptic sensors

15th

Vehicle occupant

17th

Vehicle occupant

19th

Vehicle occupant

21

Activity detection module

23

central vehicle control unit

25th

Storage unit

27

external vehicle functions

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102020100497 A1 [0002]

Claims (8)

Method for recognizing emotional states of occupants of a vehicle, in which, in a predetermined period of time while the vehicle (1) is being driven, emotions of a vehicle occupant (15, 17, 19) triggered by an interaction are optically and / or acoustically and / or haptically detected and analyzed are, characterized in that activities of several vehicle occupants (15, 17, 19) in the vehicle interior (7) are recorded and the emotion processes of the individual vehicle occupants (15, 17, 19) determined over the specified period with changes in the vehicle interior (7) recorded by sensors individual vehicle occupants (15, 17, 19) are assigned. Procedure according to Claim 1 , characterized in that the emotion profiles together with the changes recorded by sensors are determined anonymously by means of statistical processes and / or methods of machine learning and the anonymized data are assigned to the vehicle occupants (15, 17, 19). Procedure according to Claim 1 or 2 , characterized in that interrelationships between the vehicle occupants (15, 17, 19) and their cause and / or dependency are determined by means of the emotion profiles determined in the predetermined period of time. Procedure according to Claim 1 , 2 or 3 , characterized in that vehicle-internal and / or vehicle-external applications are adapted on the basis of the identified interrelationships between the vehicle occupants (15, 17, 19). Method according to at least one of the preceding claims, characterized in that the vehicle occupants (15, 17, 19) are informed of the identified interrelationships. Device for recognizing emotional states of occupants of a vehicle, comprising a control unit (3) which is connected to an emotion recognition module (5) for acquiring the current emotional state of at least one vehicle occupant (15, 17, 19) at predetermined times, for which purpose the emotion recognition module ( 5) is coupled to acoustic and / or optical and / or haptic sensors (9, 11, 13) directed at the vehicle occupants (15, 17, 19), characterized in that an activity detection module (21) for detecting a physical activity of the vehicle occupants (15, 17, 19) on the determining control unit (3) for determining time-dependent emotional states of the individual vehicle occupants (15, 17, 19) and their interrelationships. Device according to Claim 6 , characterized in that the control unit (3) receives further signals from a central vehicle control unit (23) for comparing the changes in the determined emotional states with general vehicle functions and their changes. Device according to Claim 6 or 7th , characterized in that a storage unit (25) for the central storage of the anonymized emotion profiles of the vehicle occupants (15, 17, 19) is part of the control unit (3) or is positioned outside the vehicle and connected wirelessly to the control unit (3).

Images