DE102023004448A1 - Method for determining a user's linguistic interaction with a language assistance system - Google Patents

Abstract

The invention relates to a method for determining a user's (2) linguistic interaction with a voice assistance system (3) in a vehicle (1), wherein the user (2) of the vehicle (1) is identified. According to the invention, a voice interaction of the user (2) with the voice assistance system (3), with occupants (4) of the vehicle (1) and with people (5) outside the vehicle (1) is analyzed and quantized via telecommunications, - the inmates (4) and people (5) are each assigned to a group (G1, G2), - the voice interactions for the respective group (G1, G2) are recorded and evaluated over time, with voice information of the respective group (G1, G2) are evaluated using determined feature values (M1 to Mn), - is automatically recognized when an interaction pattern of the user (2) with the voice assistance system (3) approaches an interaction pattern of one of the groups (G1, G2) or when an interaction pattern approaches of the user (2) with one of the groups (G1, G2) approaches the interaction pattern of the user (2) with the voice assistance system (3) and - if such an approach is recognized, the user (2) is informed of his change in behavior.

Description

The invention relates to a method for determining a user's linguistic interaction with a voice assistance system in a vehicle, whereby the user of the vehicle is identified.

From the US 2020/0279553 A1 a method for operating a conversation agent is known. The method includes receiving an audio input representing speech from a user; a recognition of a content of the speech; determining a style of speech; generating a response dialog based on the content of the speech; and modifying the response dialogue based on the style of the speech.

The invention is based on the object of specifying a method for determining a user's linguistic interaction with a language assistance system.

The object is achieved according to the invention by a method which has the features specified in claim 1.

Advantageous embodiments of the invention are the subject of the subclaims.

• - eine Sprachinteraktion des Nutzers mit dem Sprachassistenzsystem, mit Insassen des Fahrzeuges und mit Personen außerhalb des Fahrzeuges über eine Telekommunikation analysiert und quantisiert wird,
• - die Insassen und Personen jeweils einer Gruppe zugeordnet werden,
• - die Sprachinteraktionen für die jeweilige Gruppe über die Zeit aufgezeichnet und ausgewertet werden, wobei Sprachinformationen der jeweiligen Gruppe unter Verwendung ermittelter Merkmalswerte ausgewertet werden,

• - automatisch erkannt wird, wenn sich ein Interaktionsmuster des Nutzers mit dem Sprachassistenzsystem einem Interaktionsmuster einer der Gruppen annähert oder wenn sich ein Interaktionsmuster des Nutzers mit einer der Gruppen dem Interaktionsmuster des Nutzers mit dem Sprachassistenzsystem annähert und
• - wenn eine solche Annäherung erkannt wird, der Nutzer auf seine Verhaltensänderung hingewiesen wird.

According to the invention, a method for determining a user's linguistic interaction with a voice assistance system in a vehicle, whereby the user of the vehicle is identified, provides that

• - a voice interaction of the user with the voice assistance system, with occupants of the vehicle and with people outside the vehicle is analyzed and quantized via telecommunications,
• - the inmates and people are each assigned to a group,
• - the speech interactions for the respective group are recorded and evaluated over time, with speech information of the respective group being evaluated using determined characteristic values,

• - is automatically recognized when an interaction pattern of the user with the voice assistance system approaches an interaction pattern of one of the groups or when an interaction pattern of the user with one of the groups approaches the interaction pattern of the user with the voice assistance system and
• - if such an approach is detected, the user is informed of his change in behavior.

By using the method, it is possible to create or prevent an emotional bond between the user of the vehicle and the vehicle, in particular the voice assistance system.

Using the method, the experience and well-being of the user of the vehicle can be increased using digital means. The process serves to increase so-called digital luxury.

Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

• 1 schematisch ein Fahrzeug mit einer Vorrichtung zur Durchführung eines Verfahrens zur Ermittlung eines sprachlichen Umganges eines Nutzers mit einem Sprachassistenzsystem,
• 2 schematisch ein erstes Diagramm mit einer zweidimensionalen Darstellung einer Analyse von Sprachinteraktionen,
• 3 schematisch ein zweites Diagramm mit einer zweidimensionalen Darstellung einer weiteren Aufteilung von Gesprächspartnern,
• 4 schematisch ein drittes Diagramm mit überlappenden Gruppen in Bezug auf Sprachinteraktionen und
• 5 schematisch ein viertes Diagramm mit einer Annäherung der Gruppen in Bezug auf Sprachinteraktionen.

Show:

• 1 schematically a vehicle with a device for carrying out a method for determining a user's linguistic interaction with a language assistance system,
• 2 schematically a first diagram with a two-dimensional representation of an analysis of language interactions,
• 3 schematically a second diagram with a two-dimensional representation of a further division of interlocutors,
• 4 schematically a third diagram with overlapping groups related to language interactions and
• 5 schematically a fourth diagram with an approximation of the groups in terms of language interactions.

Corresponding parts are provided with the same reference numbers in all figures.

1 shows a vehicle 1 with a device for carrying out a method for determining a linguistic interaction of a user 2 with a voice assistance system 3, with another occupant 4 being located in the vehicle 1. In addition, a person 5 located outside the vehicle 1 is shown as the conversation partner of the user 2.

The device comprises the voice assistance system 3, a telecommunications device 6, by means of which the user 2 communicates with the person 5 as a conversation partner outside the vehicle 1, at least one microphone 7 and a control unit 8 connected to the microphone 7.

Within the vehicle 1, communication between the user 2, i.e. a driver of the vehicle 1, with the voice assistance system 3 and the other occupant 4 as well as by means of the telecommunications device 6 is carried out Communication between the user 2 and the person 5 is recorded and evaluated and analyzed by the control unit 8.

In general, it is known that digital language assistance systems 3, which are based in particular on machine learning language models, enable a generation of language and dialogues with a user 2, whereby the language and dialogues are often difficult to recognize as computer-generated. This can create a problem that the user 2 humanizes a voice assistance system 3 and attributes emotions or consciousness to the voice assistance system 3, for example. In particular, the problem can arise that the user 2 prefers an interaction with the voice assistance system 3 to an interaction with a real person, for example with the other occupant 4 in the vehicle 1, or even stops interacting with him.

A method carried out using the device is described below, by means of which an increasingly humanized perception of the voice assistance system 3 by the user 2 is automatically recognized and enables an adapted interaction control of the user 2 with the voice assistance system 3 of the vehicle 1.

First, the method provides that the user 2 is recognized and identified in the vehicle 1, for example using facial recognition, a personal identification number, a personalized vehicle key, etc.

Voice interactions of the user 2 with the voice assistance system 3, with the other occupant 4 of the vehicle 1 and with the person 5 with whom the user 2 communicates via the telecommunications device 6 are analyzed and quantized.

In particular, a content, a manner of a guided conversation recorded by means of the microphone 7 in the vehicle 1, a frequency of voice interactions, meta information, such as time of day, weather, etc., as voice interactions of the user 2 with the voice assistance system 3, etc Inmates 4 and/or the person 5 are analyzed and quantized by means of the control unit 8.

For example, communication is analyzed using statistical methods, whereby a choice of words is determined based on words used, word combinations, phrases, etc. and content is determined based on the topic areas addressed and a variance between topic areas. In particular, subject areas relating to personal information, political and/or religious and/or philosophical topics are identified. A labor law subject area, antitrust law and other legal subject areas can also be identified.

A conversational pattern of language interaction can be recognized using statistical methods in terms of speech proportions and question-answer schemes. In addition, it is possible to recognize an even distribution of the amount of speech by the conversation partners involved.

Language interactions can be analyzed using pre-trained neural models for machine language processing.

Methods such as topic modeling allow various granularities of content analysis of guided dialogues. For example, it can be determined which topic areas (e.g. food versus relativity theory) are being discussed.

Using methods such as sentiment analysis, dialogues can be annotated with an attitude and/or mood conveyed in them (bans versus reinforcing messages). In this way, it can be determined whether the user 2 has a positive or negative attitude towards a topic and in which tone of voice the user 2 speaks.

Using methods such as style classification, dialogues can be analyzed with regard to a style of formulation, for example in relation to poems, in relation to argumentative texts, in relation to literature, in relation to dialogues, etc. This determines how the speaking user 2 expresses himself, in particular whether he gives brief commands or complex explanations.

Furthermore, methods for classifying speech melodies of the user 2 can be used when carrying out a speech interaction.

In one embodiment of the method, groups for the occupants 4 in the vehicle 1 and the people 5 as conversation partners can be recorded individually. For example, a voice interaction between the user 2 and his mother may differ from a voice interaction between the user 2 and a financial service provider. Each conversation partner is assigned to a group, for example a family or a group of financial service providers, so that the analyzes of the voice interactions can be aggregated.

In particular, the language interactions for the respective group are recorded, evaluated and analyzed over time, i.e. over a certain period of time.

Speech information from a respective group G1 to G3 shown in the following figures, also referred to as a cluster, is evaluated, with characteristic values M1 to Mn determined for this purpose being used for further features described above, for example the style of the formulation, speech melody, etc.

2 shows a first diagram D1 with a two-dimensional representation of an analysis of voice interactions of the user 2 of the vehicle 1. In particular shows 2 a communication analysis carried out on the basis of two feature values M1, M2 for voice interactions of a first group G1, in particular other occupants 4 of the vehicle 1, a second group G2, in particular of people 5 with whom the user 2 communicates via the telecommunications device 6, and for voice interactions of the User 2 with a third group G3, in particular the voice assistance system 3.

3 shows a second diagram D2, with the granularity of the groups G1 to G3 being further increased. In particular, the second group G2 of people with whom the user 2 of the vehicle 1 communicates via the telecommunications device 6 is differentiated, as explained above with reference to the mother and the financial service provider.

According to the in 2 In the exemplary embodiment shown, the second group G2 is divided into a first subgroup G2.1, in particular business partners of the user 2, and into a second subgroup G2.2, in particular friends of the user 2.

With regard to the evaluation, respective focal points or mean values of group elements are determined in a feature space spanned by the feature values M1 to Mn in order to obtain, as it were, prototypical interactions for the respective group G1 to G3.

Such a value is determined in an initial phase and re-determined at regular time intervals, so that changes and, above all, approximations of prototypical interaction behavior of the user 2 to other occupants 4 and/or people 5 can be recognized.

Furthermore, individual language interactions of the user 2, for example a conversation with his mother-in-law, can be analyzed and a similarity or distance to individual prototypical language interaction behavior can be determined. This makes it possible, for example, to determine whether this voice interaction is more like a conversation with a friend or a machine.

For this purpose an in 2 distance measure D shown, such as a so-called Manhattan distance, for example in the form of a Mannheim metric, a taxi or city block metric, a Euclidean distance or a Chebyshev distance or the like.

Furthermore, group differences can be automatically evaluated, for example using a silhouette score. The evaluation of the group differences will be compared with group differences of other vehicles, not shown in detail, of a common vehicle fleet, for example a vehicle manufacturer, in particular if a value of the evaluation is aggregated across the vehicles 1 of the vehicle fleet in a central computer unit connected in terms of data to the respective vehicles 1 of the vehicle fleet .

In addition, group differences or cluster differences or the distance measure D can be determined, in addition to representing groups G1 to G3 through prototypical language interactions, using metrics known from the literature, for example single linkage, average linkage, complete linkage, using individual group examples.

Additionally or alternatively, a supervised learning machine learning approach may be used to train a model based on sample data of the respective feature values M1 to Mn of previous speech interactions to classify the known group or a target group membership value. After training, in particular after a first phase, such a user-specific model can be used in a second phase for interference or prediction, with the new model being used to predict a new language interaction and class or group membership.

An approach of an interaction pattern between the user 2 and the voice assistance system 3 is then automatically recognized with an interaction pattern between the user 2 and the other occupant 4 and/or the person 5. In particular, such an approach is automatically recognized if the user 2 does so a real occupant 4 and/or a real person 5 speaks, such as with the voice assistance system 3. There is a risk here that the user 2 humanizes the voice assistance system 3.

In addition, automatic recognition occurs when the interaction pattern between the user 2 and the other occupant 4 and/or the person 5 approaches the interaction pattern between the user 2 and the voice assistance system 3, that is, when the user 2 speaks to a human in this way , as with the voice assistance system 3. There is a risk here that the user 2 will speak to people, as with an emotionless voice assistance system 3. Such an approach is possible using arrows P in 5 shown.

4 shows potentially problematic user behavior, with groups G1 to G3 overlapping.

5 shows the possibly problematic user behavior based on the approximation of at least one of the groups G1, G2 to the voice interactions of the user 2 with the voice assistance system 3 over time, shown by the arrows P.

In one embodiment of the method, an input can be expanded from a speech analysis to an emotion analysis, in particular taking voice tone and gestures into account. For example, emotion recognition of the user 2 can be carried out based on facial, speech and/or physical signals using a corresponding sensor system, in particular on the vehicle. For this purpose, for example, the control unit 8 analyzes image data from an interior camera or several interior cameras and/or recorded data from vital sensors installed on the vehicle in real time. An approach, for example based on neural networks, of machine vision makes it possible to classify facial expressions and body movements of the user 2 and/or the other occupants 4 of the vehicle 1 and to automatically recognize and classify emotions. In addition to an analysis based on visual data, sensors in the seat can also indicate jerky, for example startling, movements. An analysis of audio data can indicate the recognition of moments of shock and/or stress by the user 2. Furthermore, vital sensors can be used, for example to detect abnormalities in the ECG, in heart rate variability, in relation to systolic blood pressure, oxygen saturation in the blood, pupil size, etc., to monitor the well-being of the user 2 and/or the other occupant 4 of vehicle 1 to quantize.

Depending on the above-mentioned approach, the user 2 of the vehicle 1 can be made aware of his change in behavior, in particular, upon request, be made aware of possible risks of such a voice interaction, determine to what extent he would like to be made aware of such an approach in the future, determine to what extent he would like to be made aware in the future depending on shared, possibly personal content.

Alternatively or additionally, a change in behavior can at least be suggested to the user 2.

For this purpose, for example, during a conversation, i.e. during a voice interaction, this can be analyzed and assigned to a prototypical interaction behavior and communicated to the user 2, for example when the user speaks to his superior, such as with a child or with a friend, or with the voice assistance system 3. The user 2 can be warned in such a case so that the user 2 can adapt his behavior appropriately during the voice interaction, i.e. during the conversation.

Alternatively or additionally, an output of the voice assistance system 3 in the vehicle 1 can be adjusted.

For example, the voice assistant 3 can be parameterized in such a way that a more formal language or a machine-like language is used if the user 2 has a voice interaction with the voice assistance system 3 that is too human and there is thus a risk that the user 2 thinks with a real person to communicate.

Alternatively or additionally, the voice assistant 3 can be parameterized so that a more natural language or a less machine-like language is used when the user 2 has an increasingly human voice interaction with the voice assistance system 3 and the user 2 builds a more emotional connection with his voice assistance system 3 in the vehicle 1 where he thinks he is communicating with a real person.

The voice assistant 3 can also be controlled in such a way that content and general interaction patterns are more similar to the content and general interaction patterns of the user 2 with friends, in that content of the voice interaction of the user 2 with a friend is captured by the voice assistant 3 via the microphone 7 and the control unit 8 . For example, the user 2 regularly talks to a friend about football and beer, so that the voice assistant 3 can also use this conversation content and in one embodiment can continue a previous conversation about these topics of the user 2 and / or interaction patterns between User 2 and friend as well as user 2 and voice assistant 3 can be approximated by optimizing the parameterization of the voice assistance system 3 in such a way that differences between the interaction pattern user 2 and friend as well as user 2 and voice assistant 3 are minimized.

Reference symbol list

1

vehicle

2

User

3

Voice assistance system

4

inmate

5

person

6

Telecommunication device

7

microphone

8th

Control unit

D

Distance measure

G1 to G3

group

M1 to Mn

Feature value

P

Arrow

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 20200279553 A1 [0002]

Claims (3)

Method for determining the linguistic interaction of a user (2) with a voice assistance system (3) in a vehicle (1), wherein the user (2) of the vehicle (1) is identified, characterized in that - a voice interaction of the user (2) is analyzed and quantized via telecommunication with the voice assistance system (3), with occupants (4) of the vehicle (1) and with people (5) outside the vehicle (1), - the occupants (4) and people (5) one each Group (G1, G2) are assigned, - the speech interactions for the respective group (G1, G2) are recorded and evaluated over time, with speech information of the respective group (G1, G2) being evaluated using determined characteristic values (M1 to Mn). , - is automatically recognized when an interaction pattern of the user (2) with the voice assistance system (3) approaches an interaction pattern of one of the groups (G1, G2) or when an interaction pattern of the user (2) approaches one of the groups (G1, G2 ) approximates the interaction pattern of the user (2) with the voice assistance system (3) and - if such an approach is recognized, the user (2) is informed of his change in behavior. Procedure according to Claim 1 , characterized in that a language of the language assistance system (3) is changed depending on the recognized approximation of the interaction patterns. Procedure according to Claim 1 or 2 , characterized in that the user (2) determines to what extent he will be made aware of his change in behavior.

Images