WO2019202056A1

WO2019202056A1 - User communication on board a motor vehicle

Info

Publication number: WO2019202056A1
Application number: PCT/EP2019/060034
Authority: WO
Inventors: Christian Suess; Viviane GHADERI; Thomas Hubregtsen
Original assignee: Bayerische Motoren Werke Aktiengesellschaft
Priority date: 2018-04-19
Filing date: 2019-04-18
Publication date: 2019-10-24
Also published as: DE102018206015A1

Abstract

The invention relates to a system for supporting a user on board a motor vehicle, comprising a scanning device on board the motor vehicle, which is configured to scan an utterance of the user, and a processing device. The processing device is configured to select the analysis service as a function of an identification of the user from a plurality of predetermined analysis services, and to pass on the scanned utterance to an analysis service, wherein the selected analysis service is configured to determine a user's intent based on the scanned utterance.

Description

User communication on board a motor vehicle

The invention relates to the support of a user on board a

Motor vehicle. In particular, the invention relates to the determination of a user intention based on an utterance of the user.

In order to control a predetermined function, a user on board a motor vehicle can express his intention in a classical manner by the operation of a dedicated scanning device. The intention may include a function of the motor vehicle and concern, for example, a driving function, an automatic route guidance or an entertainment function. The scanning device may include a button, switch or lever or a touch screen or a touchpad.

In order to better assist the user, in particular a driver of the motor vehicle, an advanced operating concept may be provided, for example a voice or gesture control. The user can be scanned acoustically or optically on board the motor vehicle and a

Sampling result can be performed by a predetermined analysis service. For example, there are various digital assistants known as Siri, Alexa or Cortana for acoustic analysis.

DE 10 2016 004 287 A1 relates to a method for speech recognition in a motor vehicle.

DE 10 2008 028 477 A1 proposes to a user by means of a

Voice control system to offer help.

It has been shown that not all analysis techniques achieve equally good results under all circumstances encountered in a motor vehicle. It is therefore an object of the present invention to provide an improved technique for analyzing a user's utterance. The invention solves this problem by means of the subjects of the independent claims. Subclaims give preferred

Embodiments again. According to a first aspect, a system for supporting a

User on board a motor vehicle, a scanning device on board the motor vehicle, which is adapted to scan a utterance of the user, and a processing device. In this case, the processing device is set up to select the analysis service as a function of an identification of the user from a plurality of predetermined analysis services and to forward the scanned utterance to an analysis service, wherein the selected

Analysis service is arranged to determine a user intention based on the scanned utterance.

As a result, it is always possible to use that analysis service which is optimal in a given situation. The quality of the analysis result can be improved. An analysis service can be switched off or replaced without endangering the function of the system. An additional one

Analysis service can be added with little effort. Optionally, a chain use of analysis services is possible by a

Analysis result of a first analysis service serves as input for a second analysis service. The improved analysis may also allow to control a function that could hitherto not be controlled aboard the motor vehicle in the manner described, due to a high level of user attention required for communication.

One type of utterance may include a gesture, an articulated sound, an inarticulate sound, a facial expression, a look, and / or a physical activity, the processing means being adapted to

Select analysis service depending on the type of utterance. Thus, innovative communication concepts can be used to improve communication with the user. The user may also make several utterances of different types, one or more

Analysis services can be employed with the analysis of each type.

The selection of the analysis service can be done depending on a driving situation of the motor vehicle. For this purpose, the driving situation can be determined in particular by the processing device. The driving situation can

For example, a driving speed, a traffic density, a ride in the city, over land or on a highway, under different Include weather conditions, times of day and sunsets. The

Processing device can be connected to determine the driving situation by means of an interface with another device on board the motor vehicle, such as a navigation system or a longitudinal or lateral control, for example, to determine a driving condition.

The driving situation can also take into account a situation in the motor vehicle, for example whether a conversation is taking place, if the user is involved in the conversation, if an entertainment system is active, etc. On the basis of the driving situation it can be decided in particular whether a scanned utterance of the user with a User intent is linked or the user has given the utterance rather unintentionally. For example, a predetermined hand movement for controlling a volume of an audio system may be provided. If the motor vehicle passes through a fast cornering combination in which the user, as the driver, has to grasp both hands in rapid succession on a steering wheel, the recognized one is

Hand movement probably more random. In other embodiments, the same statement may be made in different driving situations

be associated with different user intentions.

To select the analysis service, different approaches may be considered, one or more of which may be performed in combination with each other.

The analysis service can be selected, for example, as a function of a user setting assigned to the identification. For example, the user may make a selection depending on his or her own experience or on costs associated with using the analysis service. Several analysis services can also be named by the user, which can be used for different types of utterances, for example. Further, the user can perform a predetermined order of

Specify analytics services from which to select the highest-ranked analytic service available at a given moment.

The utterance may have a predetermined property, the analysis service depending on its processing capability for an utterance can be selected with a predetermined property. The property may include, for example, ambient noise, such as rain or higher

Speed, a dialect in an articulated utterance of the user or wearing a pair of glasses of the user in the utterance by means of a

Include facial expression or a look. For a variety

different circumstances such as one or more predetermined analysis services can be provided.

In this case, the processing device can be set up to determine the property of the utterance. For this purpose, the processing device to signals of one or more sensors on board the motor vehicle

fall back, for example, an interior and especially the user can scan. The property may also be associated with a situation on board the motor vehicle, in particular a driving situation. In a further embodiment, the property may also be assigned to the user. For example, it may be known that a particular user occasionally speaks dialect, so that not only an analysis result of a verbal utterance of the user must wait to determine this.

The processing device may be further configured to

Analysis service depending on the success of a previous analysis. In particular, a success-based learning may be implemented that determines under what circumstances which analysis service will provide the best results. For example, success can be achieved through a

User input or a user behavior are determined. If the user reruns a very similar statement in response to a particular user intent, the analysis is likely to be inaccurate.

Alternatively, the user may agree to a particular one

User intention also, for example, by pressing a button or the

Express speech of a keyword. In yet another

Embodiment, a certain confidence can be used to determine the quality. The confidence can be determined by the analysis service

be provided and express the probability with which the analysis result is correct. The analysis service determined in this way can preferably be commissioned for the following user expressions. In yet another embodiment, the utterance is forwarded in parallel to a plurality of analysis services, and one of a plurality of user intentions determined by the analysis services is selected. In particular, if there is still little experience with one or more available analysis services, their analytical capabilities can be rapidly evaluated in this way. In one embodiment, the result that arrives first may also be used. This can be useful, for example, in triggering a security-relevant or otherwise urgent or time-critical function.

In a further development, the processing device is set up to determine an analysis service to be selected in the future preferably on the basis of the selection made. So can a once selected

Analysis service will be given the possibility of customization to the user, a situation or another recurring circumstance.

The system may further include another sensing device on board the

Motor vehicle, wherein the further scanning device is adapted to determine an identification of the user. The further scanning device can be operated manually, for example by the user. Alternatively, the scanner may be a personal item such as an electronically encoded key or a corresponding one

Include subject. The user can also be identified by means of his mobile phone, for example by means of near field communication (NFC). An identification of the user by the mobile phone can be accepted.

In yet another embodiment, the user may be identified by a biometric feature, such as a camera or a fingerprint reader. Also an identification based on an entered code is possible. Further, the user may be identified based on a personalized setting, such as the adjustment of a seat, exterior or interior mirror.

The processing device is preferably set up to control a predetermined function depending on the specific user intention. The function preferably affects the user and may have a function aboard the Motor vehicle include, for example, the opening or closing of a window or the activation of a navigation destination. Alternatively, the function can also be effected only on board the motor vehicle, for example reading aloud an e-mail or establishing a telephone connection.

The execution of the function by an execution service can be initiated by the processing device. For this purpose, one of several predetermined services offering the function can also be selected. For example, in a first step, the utterance of the

User include a voice instruction concerning the reading of an e-mail. The service performing the function may be identical to or different from the analyzing service. Thus, different services can be better combined, for example, one for the

To increase user usable functionality.

According to a second aspect, a motor vehicle comprises a scanning device on board the motor vehicle, for scanning a user's utterance, and a processing device. In this case, the processing device is to

arranged to select the analysis service in response to an identification of the user from a plurality of predetermined analysis services and forward the scanned utterance to an analysis service, wherein the selected

In a first variant, the processing device is located on board the motor vehicle, as indicated above at various points. In a second variant, the processing device is outside the motor vehicle and is for example by means of a wireless

Communication device connected to the motor vehicle. The

Processing device may be part of another service that

in particular via a communication network such as the mobile network or the Internet can be reached.

In a third aspect, a method of assisting a user on board a motor vehicle includes steps of scanning a user's statement on board the motor vehicle; of selecting one Analysis service of a plurality of predetermined analysis services in response to an identification of the user; and forwarding the scanned utterance to the selected analysis service, the selected one

The method can be present in whole or in part in the form of a computer program product with program code means. The mentioned herein

Processing device can in particular a programmable

Microcomputers include and be adapted to perform the method or a part thereof. The processing device may be mounted on board the motor vehicle. Features or advantages of the method can be related to the system or the motor vehicle and vice versa.

The invention will now be described in more detail with reference to the accompanying drawings, in which:

Figure 1 shows a system with a motor vehicle; and

Figure 2 illustrates a flowchart of a method.

FIG. 1 shows a system 100 with a motor vehicle 105, on whose shelf a user 110 is located. In addition, there is a device 115 on board the motor vehicle 105, which comprises at least one scanning device 120, which is adapted to scan an utterance of the user 1 10, furthermore a processing device 125, and preferably a communication device 130, which is set up for wireless communication. Optionally, a further scanning device 135 is provided, which is configured to provide an identification of the user 1 10. The scanner 135 may be part of a security system of the motor vehicle 105 or a digital assistant of the user 110. The identity of the user 110 may also be determined by hints, such as a picture of the user, settings he makes on the motor vehicle 105, or his driving style if he is driving the motor vehicle 105 himself. The scanning device 120 may include, for example, a camera (in the visible or invisible wavelength range), a microphone, an ultrasonic sensor, a touchpad or a touchscreen. A task of

Sampling device 120 is to sample an utterance of user 110, which utterance may particularly include a component which person 110 may also use in social interaction with another person, such as a gesture, a look, speech, touch, facial expression , Tonality, a non-verbal sound, etc. It is also possible for several such signals, which may be in a sense relationship with one another, to be scanned simultaneously by the person 110.

The utterance is more preferably an expression of the user 110 for a predetermined intention. The utterance may be from a predetermined repertoire, for example, intended to control intentions in the given context of the motor vehicle 105. For example, a look at an exterior mirror can select an exterior mirror adjustment and another look can adjust the exterior mirror in a corresponding direction. Preferably, however, the repertoire is not limited in this way, but includes elements that can also be used for interpersonal communication. For example, the exterior mirror adjustment of the example could be activated by the person 110 moving to the left

Outside mirror looks and "next up", "up", "higher" or one

meaningful statement formulated linguistically.

Outside the motor vehicle 105 is preferably another wireless

Communication device 140 is provided which is adapted to

To exchange information with the communication device 130. The communication device 140 is preferably with one or more

Analysis services 145, including a network 150 may be provided, which may be wired, wireless or mixed and may include a private or public section. In addition, one or more execution services 155 may be communicatively connected to the communication device 150. The services 145, 155 can each be realized as a server, as a locally abstracted service, in particular in a cloud, or in another way. For example, a plurality of analysis services 145 may be provided, each having a user intent based on a scan of a user verbal utterance of the user 110. Typically, the analysis services 145 differ in certain characteristics, such as a rate of determination, resistance to noise, knowledge of different languages or dialects, a destination accuracy, etc. Properties of analysis services 145 may be stored in a data store 160 that may be connected to the processing device 125 , A control service 155 is preferably configured to perform a predetermined control on the basis of a corresponding input, ie to achieve an effect intended by the user 110. This effect can be of a physical nature, such as a

mechanical device is controlled on board or outside of the motor vehicle 105, or for example information technology nature, for example, made a telephone connection, read an e-mail or a calendar entry is created. The control service 155 may also implement, for example, a booking, a payment transaction, or an order, or provide information such as a weather, a local time, a sports score, or a stock price.

In the illustrated embodiment, the processing device 125 is aboard the motor vehicle 105; In another embodiment, the processing device 125 may also be outside the motor vehicle 105

be provided; In this case, a further processing device for signal processing and / or communication on board the motor vehicle 105 may be provided. In the following, only one processing device 125 is assumed, which is arranged on board the motor vehicle 105 purely by way of example.

An utterance of the user 110 may be by means of at least one

Sampling device 120 is scanned and passed by the processing means 125 to one of the analysis services 145. The analysis service 145 is configured to determine a user intention based on the scanned utterance, and preferably to return it to the processing device 125. For example, an analysis service 145 may perform a voice-to-text conversion (or vice versa), a translation from one language or dialect to another language or dialect, and so forth. On the io

Based on the particular user intention may then be a predetermined

Control task to be executed, as described in more detail below.

It is proposed that the processing device 125 of the

Analysis Services 145 selects one to determine its underlying user intent based on a present sample of an utterance of the user 110. Preferably, the selection is based on additional information that takes into account a current situation of the user 110, a property or capability of an analysis service 145, and / or other factors. For this purpose, in particular information stored in the data memory 160 and situational information, which likewise can be determined by means of a scanning device 120, can be correlated with one another.

Furthermore, it is preferred that the selection of the analysis service 145 can be adapted, in particular on the basis of a history in order to repeat a once successful selection as possible, a current availability of analysis services, a price incurred for the analysis or another parameter, the may also change quickly, so that the selection of analysis service 145 can be very dynamic.

There may also be multiple analysis services 145 for processing

different aspects of an utterance of the user 110, whereby the quality of a pairing of analysis services 145 can also be included in a future selection. Each analysis service 145 may be sent an aspect of the utterance for processing, one aspect may also be processed by multiple analysis services 145. In a cooperative

In parallelization, results of the analysis services 145 can be combined with each other; in a competitive parallelization, the result whose quality, for example with regard to a processing speed, a specific confidence or a degree of detail, was determined to be the best can be used. Further, multiple analysis services 145 may be functionally concatenated with each other so that a result of one or more analysis services 145 represents an input of one or more other analysis services 145. For example, in an example of chaining analysis services 145, a speech-to-text conversion, a natural language understanding (NLU), a dialog management, a natural language generation (NLG), and a text-to-speech may be successively performed Speech conversion (text to speech, TTS). A similar concatenation could also be carried out, for example, for face recognition and viewing direction recognition. An order of processing can also be varied.

Also at the chaining can the quality of a mating different

Analysis services 145 can be used as a basis for a later selection of the analysis services 145. For example, the flexible selection of analysis services 145 may allow one analytic service 145 or a combination of parallel and / or serially concatenated analysis services 145 against another

Analysis Service 145 - or another combination of analysis services 145 - to replace.

An analysis service 145 may also be inserted into an existing chain of analysis services 145. For example, a sequence of a

Speech recognition and a downstream generation of natural language can be changed by performing a semantic analysis in between. Recursive nesting of analysis services 145 may also be possible.

In another example, a chain of analysis services 145 for speech analysis may be flexibly augmented by introducing appropriate analysis services 145 for other modularities, such as gesture recognition, face recognition, touch recognition, etc., into Natural Interaction Processing (NIP). Here, depending on the task or availability of modalities and this

necessary hardware (e.g., cameras, touchscreens, etc.), flexible NIP systems are configured together.

An example of this would be a speech recognition with a recognition of natural language, which is parallel to a viewing direction detection, a graphical user guidance and possibly another combination of Analysis Services 145 works. The further combination can be one

Touch detection, followed by one or more

cascaded analysis services 145 and a natural

Include speech production with conversion into spoken language.

It should be noted that in one embodiment, one or more other services, particularly control services 155, may be included in such a chain, so that virtually any meshes parallel and / or serially interconnected services 145, 15 are generated and modified as needed can.

FIG. 2 shows a flow chart of a method 200 that can be carried out in particular on the system 100 of FIG. In particular, the method 200 may be executed in whole or in part on the processing device 125.

In a first step 205, an identification of the user 1 10 on board the motor vehicle 105 can be determined. For this purpose, it can also be determined whether the user 110 controls the motor vehicle 105 himself or is only a passenger. In a step 210, an utterance of the user 110 is scanned by means of at least one scanning device 120, as described in more detail above.

In a step 215, an analysis service 145 is selected, wherein the selection is made preferably on the basis of a situation of the user 1 10, predetermined properties of various analysis services 145 and optionally further information. The scan or a portion of the scan, and optionally other information such as a particular

Driving situation of the user 1 10 and the motor vehicle 105 are then preferably transmitted to the selected analysis service 145.

In a step 220, on the basis of the transmitted information, the selected analysis service 145 determines a user intention that may be the basis of the scanned utterance. In certain cases, it may also be determined that the scanned utterance is unlikely to be motivated by a recognizable intention of the user. In a step 225, the result of the analysis can be evaluated, for example, based on a confidence value also provided or on the basis of a compatibility of the specific user intention with the previously determined situation. This step may also be performed at a later time and feedback from the user may include how well he senses his intention through the analysis.

In a step 230, a function may be determined that is associated with the intent or that may be inferred from the particular intent. The function can be controlled directly in a step 235, in particular by the aboard the motor vehicle 105 brought off

Processing device 125, or in a step 240, a

Control service 155, which performs the function by transmitting a corresponding request and any required information to the selected control service 155. The

Function control by means of the control service 155 may be useful, in particular, if the function does not directly affect the motor vehicle 105 or is influenced by the fact that the user 110 is on board the motor vehicle. The control service 155 can also be used to advantage when processing an output of the analysis service 145 directly. For example, the

Analysis service 145 includes a speech-to-text (STT) conversion and the control service 155 may send or store text at a predetermined location. The method 200 may also be varied, particularly in the order or frequency of execution of the described steps, or by any of the other embodiments, variants, options, or capabilities described herein. reference numeral

100 system

105 motor vehicle

1 10 users

115 device

120 scanning device

125 processing device

130 communication device

135 scanning device

140 communication device

145 Analysis service

150 network

155 execution service

160 storage device

200 procedures

205 Determine Identification User

210 Sampling of the user

215 selections Analysis service; Submit / forward

220 Analyze

225 Evaluate the result

230 Determine function

235 tax function

240 selections control service; Submit / forward

Claims

claims

A system (100) for assisting a user (110) on board a motor vehicle (105), comprising:

a scanner (120) on board the motor vehicle (105) for sampling an utterance of the user (110);

processing means (125) arranged to pass the scanned utterance to an analysis service (145); wherein the analysis service (145) is adapted to determine user intention based on the scanned utterance;

characterized in that

the processing device (125) is adapted to the

Analysis service (145) in response to an identification of the user (110) from a plurality of predetermined analysis services (145)

select.

The system (100) of claim 1, wherein a type of utterance comprises a gesture, an articulated sound, an inarticulate sound, a facial expression, a look, and / or a physical operation, and the

Processing means (125) is adapted to select the analysis service (145) depending on the type of utterance.

3. System (100) according to claim 1 or 2, wherein the processing device (125) is adapted to determine a driving situation of the motor vehicle (105) and to select the analysis service (145) depending on the driving situation.

4. System (100) according to one of the preceding claims, wherein the

Processing device (125) is adapted to the analysis service (145) in response to an identification associated

User setting (1 10).

The system (100) of any of the preceding claims, wherein the utterance has a predetermined property and the

Processing device (125) is adapted to the analysis service

(145) depending on its processability for an utterance having a predetermined property.

The system (100) of claim 5, wherein the processing means (125) is arranged to determine the property of the utterance.

7. System (100) according to one of the preceding claims, wherein the

Processing means (125) is arranged to select the analysis service (145) in response to a success of a previous analysis.

8. System (100) according to one of the preceding claims, wherein the

Processing means (125) is adapted to forward the utterance to a plurality of analysis services (145) and to select one of a plurality of specific user intentions.

The system (100) of claim 8, wherein the processing means (125) is arranged to determine an analysis service (145) to be selected preferably in the future based on the selection made.

10. System (100) according to one of the preceding claims, further comprising a further scanning device (135) on board the motor vehicle (105), wherein the further scanning device is adapted to a

Identification of the user (1 10) to determine.

1 1. A system (100) according to any one of the preceding claims, wherein the

Processing means (125) is adapted to control a predetermined function depending on the particular user intention.

The system (100) of claim 1 1, wherein the processing means (125) is arranged to cause the execution of the predetermined function by an execution service (155).

13. Motor vehicle (105), comprising:

a scanner (120) for sampling an utterance of a user (110) on board the motor vehicle (105);

characterized in that

the processing device (125) is adapted to the

select.

14. Method (200) for supporting a user (110) on board a motor vehicle (105), comprising the following steps:

Scanning (210) an utterance of the user (110) on board the motor vehicle (105);

Selecting (215) an analysis service (145) from several

predetermined analysis services (145) depending on a

Identification of the user (1 10);

Forwarding (215) the sampled utterance to the selected analysis service (145);

wherein the selected analysis service (145) is adapted to determine a user intention based on the scanned utterance.

The method (200) of claim 14, wherein the utterance is linguistic with

is a dialect, and the processing means (125) is arranged to select the analysis service (145) based on its ability to process speech with the dialect (215).