DE102019105588A1 - Minimally invasive integration of messaging services in vehicles - Google Patents

Abstract

The present disclosure relates to an instant messaging method for use in a vehicle of a user. The method comprises determining one or more context parameters that describe the context of a use of instant messaging by the user; Receiving one or more incoming messages; Determining, based on the one or more context parameters, whether the one or more input messages are to be transmitted to the user; and, if the one or more input messages are to be transmitted to the user, transmitting the one or more input messages to the user. The present disclosure further relates to a system that is configured to carry out the method according to the invention, and to a vehicle that comprises the system.

Description

The disclosure relates to systems and methods for integrating messaging services in vehicles. The disclosure relates in particular to systems and methods for the minimally invasive, context-sensitive and multimodal integration of messaging services in vehicles.

State of the art

Messaging (also instant messaging or chat) generally refers to the timely sending of mostly text messages between different users or user groups, preferably via wireless networks. For this purpose, special programs (apps) are typically used on mobile devices, whereby the users communicating with each other must use the same app (e.g. WhatsApp, Facebook Messenger, iMessage, ICQ) in order to be able to connect with each other. At least the apps used must support the same communication platform or the same messaging service (e.g. IRC, XMPP).

In the prior art, at least some of the integration of messaging services in vehicles is known, which is implemented on devices integrated in the vehicle, for example infotainment systems. Manufacturer-specific (e.g. OEM-proprietary) solutions are typically used here. Apps are also known that run on mobile devices, for example smartphones, and can be linked to infotainment systems in the vehicle via appropriate interfaces (e.g. Google Android Car, Apple Car Play). There are also largely manufacturer-independent approaches (e.g. Mirror Link) for connecting different platforms, for example for connecting different mobile devices with infotainment systems from different manufacturers, so that the use of apps developed for mobile devices via the vehicle's user interface (HMI) is possible regardless of the manufacturer.

The operation of apps on mobile devices is problematic for the user of a vehicle (hereinafter also referred to as the "driver", unless expressly specified otherwise), at least while driving, and the immediate use of the mobile device (e.g. without a hands-free device) by the driver due to impaired attention and distraction prohibited in most countries. Various apps provide appropriate functions to take into account situations in which messages are received and answered or otherwise processed while driving. For example, the driver can select and send roughly standardized text modules provided by the respective app with relatively little effort (eg "I'm on the road", "Urgent?", "Contact me later!") Without the need for a correspondingly complex text entry becomes necessary. In some cases, the apps can also be configured to send a previously selected standard response (e.g. "In a meeting, contact me later"). However, direct or manual operation of the app is often necessary anyway.

In addition, certain services have additional functions in some cases, for example for the analysis of chat messages (e.g. Facebook Messenger) or e-mails (e.g. Google Smart Reply) in order to be able to enter and receive appropriate input or output based on the chat history or the e-mail message. Suggested answers variable, ie not just static, to provide. Other functions include functions known from e-mail or calendar programs, for example "out-of-office replies", display of conflicts for suggested appointments, or e-mail autoresponders.

The pamphlet US 2007/0162550 A1 describes an instant messaging system for communication from a first vehicle to a second vehicle, wherein the communication is addressed at least partially based on a spatial proximity of the two vehicles. In this way, it should be possible to design communication with other road users who are actually unknown - instead of generic (e.g. horn or headlight flasher) - so that, for example, a warning signal can be given specifically to a road user or an overtaking can be announced without other road users feel addressed. The described system also allows a series of predefined messages (eg overtaking request, general warning, keep your distance) to be sent, for example by means of additional buttons on the steering wheel. Instant messaging is not discussed in the publication.

The pamphlet U.S. 9,947,319 describes a virtual or mobile assistant that can record and save information about a user's status in a first session. In a second subsequent session, reference can then be made to the stored state by selecting from a large number of predetermined words, phrases or statements. This processing of the status of a user is intended to give the assistant a more social or human component, so that the interaction with the assistant is more like an interaction with a human counterpart. The context of use "vehicle" is not taken into account in the publication.

Known systems usually only support one, possibly proprietary, communication platform, so that cross-platform communication is not possible or only possible to a limited extent. Although a user can switch between several apps under certain circumstances, each operating step for switching the app and / or for operating different (graphical) user interfaces entails a corresponding input effort and impairment of attention on the part of the user. If the user is the driver of the vehicle, the attention required to operate the communication must be reduced to a minimum in order to ensure safe driving of the vehicle in traffic. In addition, simultaneous or coupled communication between partners on different platforms is not possible. Furthermore, in some vehicles there is generally no chat client or the chat client for a specific chat platform that a partner outside of the vehicle would like to use to communicate with the driver.

There is therefore a need for a system that enables platform-independent and cross-platform communication with different communication partners, whereby the system, in particular its operation, should be completely transparent for the user and independent of the apps or communication platforms used in each case, uniform and clear . This has the advantage that different platforms can be used transparently by the driver, in particular that platforms for which no integration with the infotainment system present in the vehicle is offered can also be used. Furthermore, the means and options available in the vehicle for interaction (e.g. acoustically, haptically) with the driver should be used, so that direct operation of the app is no longer necessary. This maintains or increases the driver's attention to the traffic. Furthermore, the means and options available in the vehicle for interaction (e.g. acoustically, haptically) with the driver should be used, so that an app (e.g. a chat client) does not have to be available. For example, a “hello” message can be implemented by vibrating the steering wheel and a response to this can be implemented by tapping your finger on the steering wheel.

Known systems or apps typically only handle messages or responses originating from the driver. A processing or preparation of incoming messages does not take place or takes place only rudimentarily. For example, the driver of a vehicle cannot participate in a group chat via which a large number of participants can send and receive a large number of messages in a short time, or only to a very limited extent. Since incoming messages are not processed or edited, the driver is often overwhelmed by the amount of messages and cannot extract the essential information from the chat without manually reviewing the messages in detail. However, this would require the driver to neglect the attention of the traffic and dangerous situations could result.

There is therefore a need for a system which processes incoming messages in such a way that a user is made accessible only to him or her relevant and up-to-date information, in particular in a manner that does not or does not significantly impair attention to the traffic. Here it is desirable that outdated and irrelevant information is reduced or completely filtered out and current and relevant information is bundled and made available to the user at the right time. For example, it often happens in group chats that appointments are made, the outcome of which is only determined at the end of the conversation, for example, when several dates are offered and discussed for a meeting before the participants finally agree on one of the options discussed. It would not make sense here to convey the entire course of the discussion to a user if most of the content is of no or at best subordinate importance. This has the advantage that the driver's attention to the traffic is maintained or further increased. A summary can be made textually or graphically. It can take place within one of the given chat client apps in the vehicle or via the vehicle HMI or other actuators in the vehicle, for example analogous to a Morse code by vibrating the seat, steering wheel or the like.

There is also a need for a system that takes into account the current state of the driver or the vehicle and the traffic situation. The context of a driving situation can include the state of the driver (e.g. alert, tired, relaxed, aggressive, calm, hectic), the driving situation (e.g. city, cross-country drive, motorway, speed, driving style, control systems, assistance systems) and other factors (e.g. location , Time, date, weather). Such a system should preferably output messages in situations in which the driver only has to pay little or no attention to the traffic, for example when standing still in a traffic jam, in stop & go traffic, when driving with assistance systems (e.g. stop & go, longitudinal and lateral guidance), or when you wait (longer) at traffic lights or at level crossings, and the like. Likewise, avoid distracting the user in situations in which heightened attention is required, for example while driving in heavy traffic, at increased speed on motorways, in urban areas, in bad weather conditions or when control systems intervene (e.g. skidding of the vehicle on slippery roads). Furthermore, the system should be able to have a positive effect on the condition of the driver, for example, calming down if the driver appears excited or aggressive, or encouraging if the driver can be assumed to be tired. Communication or a lack of communication can have a corresponding effect here. This has the advantage that the driver's communication is controlled in such a way that driving safety is not impaired or improved.

After all, known systems usually only provide a very limited number of predefined text modules or answers that often do not fit the communication context and / or subliminally convey to the communication partner that the communication is paid little or no attention (e.g. "OK", Yes "," No ”as answers). Furthermore, only simple text answers are often available, but no other media (e.g. photos, audio / video recordings).

There is therefore a need for a system that provides context-dependent individual vocabulary, answer sentences and / or statements that correspond to the other communication behavior of the user, which he or she also uses in other situations (e.g. not in the vehicle). For example, with certain communication partners (e.g. friends, family) an informal, relaxed tone is more appropriate, while with others (e.g. colleagues, strangers) a more formal, more polite language may be required. The system should be able to capture different contexts and adapt processing to them. This has the advantage that the driver is provided with customizable communication options.

In addition, in certain situations it can be advantageous if the driver can communicate unnoticed by passengers or other occupants, for example in the case of chauffeur- or taxi-like applications (e.g. Uber, Lyft, taxis). Voice input or output of text messages, for example, are often not suitable in such a context, since both incoming and outgoing messages can typically be heard by everyone in the vehicle. Here the system should have modalities that allow exclusive communication with the driver that cannot be perceived by other people in the vehicle.

Disclosure of the invention

It is an object of the present disclosure to provide instant messaging methods and systems for use in a vehicle which avoid one or more of the aforementioned disadvantages and / or enable one or more of the described advantages.

This object is achieved by the subject matter of the independent claims. Advantageous refinements are given in the subclaims.

In a first aspect according to embodiments of the present disclosure, an instant messaging method for use in a vehicle is specified. The method comprises determining one or more context parameters that describe the context of a use of instant messaging by the user; Receiving one or more incoming messages; Determining, based on the one or more context parameters, whether the one or more input messages are to be transmitted to the user; and, if the one or more input messages are to be transmitted to the user, transmitting the one or more input messages to the user.

In a second aspect according to aspect 1, the method further comprises determining, based on the one or more context parameters, whether the one or more input messages are to be signaled to the user; and, if the one or more input messages are to be signaled to the user, signaling to the user that one or more input messages have been received. Preferably, the signaling comprises one or more of the following signals: one or more optically perceptible signals, preferably wherein the one or more optically perceptible signals include one or more of the following: signals that are output via a head-up display (HUD); Signals displayed via a central instrument cluster; and signals that are displayed on a central information display (CID); one or more acoustically perceptible signals, preferably wherein the one or more acoustically perceptible signals include one or more of the following: signals that are output via a loudspeaker system of the vehicle; Signals that are output via an acoustic signal generator; and one or more tactilely perceptible signals, preferably wherein the one or more tactilely perceptible signals include one or more of the following: vibration or knocking signals that are output via control elements of the vehicle; Vibration or knocking signals that are output via a seat in the vehicle.

In a third aspect according to one of the aspects 1 or 2, the method further comprises processing the one or more input messages using natural language processing (NLP). The processing preferably comprises: content-neutral reduction of a number and / or length of the one or more input messages; and / or removal of obsolete and / or for the user irrelevant parts of the one or more input messages. A sequence, order and / or sorting present within the one or more input messages is preferably retained during the reduction and / or removal.

In a fourth aspect according to one of aspects 1 to 3, the method further comprises generating a message from the user based on the one or more input messages and the one or more context parameters, the message preferably including a response to one or more incoming messages and / or the message includes one or more of text, emoticon, image, audio, and video; and preferably automatically sending the user's message without any action on the part of the user.

In a fifth aspect according to aspect 4, the method further comprises determining, based on the one or more context parameters, whether the user's message should be sent automatically; and, if the user's message is to be sent automatically, the user's message is automatically sent without any action on the part of the user.

In a sixth aspect according to aspect 4, the method further comprises generating a plurality of alternative messages from the user based on the one or more input messages and the one or more context parameters; Delivering the multiple alternative messages to the user; Receiving input from the user to select at least one of the plurality of alternative messages; and sending the selected at least one message.

In a seventh aspect according to aspect 6, receiving an input from the user for selecting at least one of the plurality of alternative messages comprises: detecting a tactile input via a sensor in the steering wheel, preferably the sensor being configured to detect vibrations or shocks; Detecting an acoustic input from the user; and detecting activation of a control element of the vehicle by the user.

In an eighth aspect according to one of the aspects 1 to 7, the context parameters include one or more of the following parameters: one or more of location, time and date; one or more weather parameters that describe a local weather in relation to the vehicle, preferably including one or more of temperature, precipitation, air pressure, air pressure profile, air humidity and visibility; one or more vehicle parameters that describe a state of a vehicle, preferably including one or more of the speed of the vehicle, state of driver assistance systems, state of control systems, direction of travel of the vehicle, range of the vehicle, number of vehicle occupants, currently traveled route in a navigation system of the vehicle Type of a currently traveled road based on map material available in the navigation system, degree of the user's attention available for use of instant messaging, and an estimated time of arrival of the vehicle; one or more user parameters that describe a state of the user of the vehicle, preferably including one or more of the state of mind of the user, state of alertness of the user, length of stay of the user in the vehicle, response or reaction speed of the user and activity of the user; and one or more environmental parameters that describe the surroundings of the vehicle, preferably including the type and number of other road users in the surroundings of the vehicle, the type of road being traveled on and the state of the road being traveled on.

In a ninth aspect, an instant messaging system for use in a vehicle of a user is specified. The system comprises a control unit which is configured to carry out the method according to one of aspects 1 to 8.

In a tenth aspect, a vehicle is specified. The vehicle comprises an instant messaging system according to aspect 9.

The methods and systems disclosed herein provide one or more of the advantages noted below.

Based on the methods and systems disclosed here, the user of a vehicle or other occupants of the vehicle can participate in instant messaging, the incoming and outgoing messages taking into account the current context of use. The steps necessary for communication that must be carried out by the user are reduced or minimized. The user of the vehicle is thereby not unnecessarily distracted from the traffic situation, especially in certain driving situations. The user's attention to road traffic is generally improved. Furthermore, the user's attention can be increased in certain situations through the targeted interaction with the system.

Figure list

• 1 zeigt eine schematische Darstellung eines Instant Messaging Systems gemäß Ausführungsformen der vorliegenden Offenbarung zur Anwendung in einem Fahrzeug; und
• 2 zeigt ein Flussdiagramm eines Verfahrens gemäß Ausführungsformen der vorliegenden Offenbarung.

Exemplary embodiments of the disclosure are shown in the figures and are described in more detail below. Unless otherwise noted, the same reference symbols are used below for elements that are the same or have the same effect.

• 1 shows a schematic representation of an instant messaging system according to embodiments of the present disclosure for use in a vehicle; and
• 2 12 shows a flow diagram of a method according to embodiments of the present disclosure.

Embodiments of the disclosure

Unless otherwise noted, the same reference symbols are used below for elements that are the same and have the same effect.

When reference is made here to messages or reply messages, these messages not only include text messages (including emoticons), but can also contain other media or types of media in mixed form, for example emojis, images (including animated images, e.g. animated GIFs), photos, video, audio and the like and combinations of the above.

The term “vehicle” 80 includes cars, trucks, buses, mobile homes, motorcycles, etc. that are used to transport people, goods, etc. In particular, the term includes motor vehicles for passenger transport. vehicles 80 with electric drive (in particular hybrid or electric vehicles) are driven by an electric motor and include electrical energy storage devices (eg batteries, high-voltage storage devices) that operate via a vehicle charging device 80 can be connected to a charging station and charged. Various charging technologies can be used to charge the electrical storage of such hybrid or electric vehicles.

Preferably the vehicle comprises 80 an environment sensor system which is set up to record the environment data. The environmental sensor system preferably comprises at least one LiDAR system and / or at least one radar system and / or at least one camera and / or at least one ultrasound system. The environment sensor system can provide the environment data (also referred to as “environment data”), which depict an area around the vehicle.

The term “automated driving” can be understood in the context of the present disclosure as driving with automated longitudinal or lateral guidance or autonomous driving with automated longitudinal and lateral guidance. Automated driving can be, for example, driving on a motorway for a longer period of time and / or driving for a limited time as part of parking or maneuvering.

The term “automated driving” includes automated driving with any degree of automation. Exemplary degrees of automation are assisted, partially automated, highly automated or fully automated driving. These degrees of automation were defined by the Federal Highway Research Institute (BASt) (see BASt publication “Research compact”, edition 11/2012). With assisted driving, the driver continuously performs longitudinal or lateral guidance, while the system takes on the other function within certain limits.

With partially automated driving (TAF), the system takes over the longitudinal and lateral guidance for a certain period of time and / or in specific situations, whereby the driver has to continuously monitor the system as with assisted driving. With highly automated driving (HAF), the system takes over the longitudinal and lateral guidance for a certain period of time without the driver having to permanently monitor the system. However, the driver must be able to take control of the vehicle within a specified time. With fully automated driving (VAF), the system can automatically cope with driving in all situations for a specific application; a driver is no longer required for this application. The four degrees of automation mentioned above correspond to SAE levels 1 to 4 of the SAE J3016 standard (SAE - Society of Automotive Engineering). For example, highly automated driving (HAF) level 3 corresponds to SAE J3016 standard . In addition, SAE J3016 provides SAE level 5 as the highest level of automation, which is not included in the definition of BASt. SAE level 5 corresponds to driverless driving, in which the system can automatically handle all situations like a human driver during the entire journey. In this case, a driver is generally no longer required.

1 shows a schematic representation of an instant messaging system 100 according to embodiments of the present disclosure for use in a vehicle 80 . The system can essentially be based on a control unit 120 of the vehicle 80 are executed. The vehicle 80 includes besides the control unit 120 further a communication unit 130 used for data communication with to the vehicle 80 external components (e.g. mobile devices 70 and backend 150 ) is configured and a user interface 110 .

The user interface 110 includes one or more multimodal user interfaces, in particular user interfaces for operating the vehicle 80 configured (e.g. air conditioning, seat adjustment, infotainment, vehicle settings). The user interface 110 enables the multimodal recording of input from a user 60 (not in 1 shown), for example via a graphical user interface (e.g. touchscreen), via classic vehicle controls 80 (e.g. buttons, switches, iDrive controller), via voice control and the like. The user interface 110 further enables the multimodal output of information to a user 60 , for example via graphic display elements (e.g. touchscreen, head-up display, instrument cluster, central information display or CID), via tactile elements (e.g. vibration of the steering wheel or parts of the seat), by voice output via a loudspeaker system in the vehicle (e.g. infotainment system ) or acoustic signal generators (e.g. gong, beeper) and the like. The user interface can 110 Implement a graphical user interface based on corresponding configuration data, in which display elements and operating elements are shown that are provided by the user 60 for operating the vehicle 80 can be used.

Additionally or alternatively, the user interface can contain (further) display and operating elements, for example switches, buttons and displays.

Via the communication unit 130 can the control unit 120 enter into data communication with external components and services and thus communicate with instant messaging services (e.g. WhatsApp, Slack, Facebook Messenger, Google Chat, ICQ, iMessage) or protocols (e.g. IRC, XMPP). As an alternative or in addition, the control unit can use the communication interface 130 with instant messaging apps, for example on a mobile device 70 of a user 60 are installed, enter into data communication and thus use instant messaging services whose connection to the control device is not directly implemented or otherwise supported. A connection with mobile devices 70 can for example be done through proprietary interfaces (e.g. Apple Car Play, Google Android Car, Mirror Link).

The system can continue 100 one to the vehicle 80 external backend component 150 or have an infrastructure that provides one or more resources (eg server, services). The backend component 150 can temporarily or permanently with the control unit 120 of the vehicle 80 in data communication 140 stand. Resource-intensive processing steps (for example processing natural language and / or image or video data, speech recognition, processing large external data volumes) can preferably be sent to the external backend component 150 be swapped out by the control unit 120 in the vehicle 80 difficult or impossible to do. In this context, possible requirements with regard to computing power, storage capacity, available bandwidth, connection to external data sources and the like can also be taken into account.

One on the control unit 120 The executed software component establishes a data connection to one or more instant messaging services. Such a connection can preferably be established directly if a service or a platform (or the underlying protocol) is supported directly by the software component. Such a connection can alternatively or additionally via a by the user 60 with the vehicle 80 connected mobile device 70 respectively. All services, platforms and protocols that are supported by the mobile device can then be used by the software component.

The software component typically takes the place of the user as a participant in an instant messaging service and appears as the user in the chat. In some embodiments, the software component can alternatively or additionally occur as an independent participant in the chat, for example as a chatbot. The software component can also appear as a participant in several instant messaging services at the same time and make communication transparent for the user so that the user does not have to pay attention to which service is being used for communication.

The software component also has access to one or more context parameters. The context parameters characterize the context of an application of instant messaging services in a variety of ways. For example, context parameters contain general parameters such as location, time and / or date. Based on these parameters, for example, the user can be addressed individually, taking into account the respective time of day, holidays or days of the week and local conditions. General parameters can typically be found in the control unit anyway 120 Existing data are recorded because the location, time and / or date are recorded or made available by the navigation system and the control unit itself.

Context parameters can furthermore contain one or more weather parameters that relate to the vehicle 80 Describe local weather, for example based on temperature, precipitation, air pressure, air pressure profile, air humidity and / or visibility. Weather parameters can play a very important role in the context of an application in the vehicle, for example when bad weather conditions prevail and the user of the vehicle 80 thereby should pay increased attention to the traffic. A distraction for the user 60 or a reduction in their attention to the traffic is to be minimized or avoided in such situations. Weather parameters can at least partially be recorded by the control unit itself, which has access to sensor data (e.g. temperature sensor, rain sensor, camera of the driving safety systems) of the vehicle 80 Has. Other weather parameters can typically be determined via external data sources (e.g. weather services, connected drive).

Context parameters can furthermore contain one or more vehicle parameters that indicate a state of the vehicle 80 describe, for example, based on the speed of the vehicle, the status of driver assistance systems, the status of control systems, the vehicle's direction of travel, the vehicle's range, the number of vehicle occupants, the currently traveled route in the vehicle's navigation system, the type of road currently being driven based on the map material available in the navigation system and the like more. Vehicle parameters form a very important group of parameters for the context of the application. On the basis of the vehicle parameters, the control unit can draw the user's attention to the particular driving situation 60 conclude. For example, when driving at higher speeds on motorways or country roads, it can be assumed that the user 60 of the vehicle 80 should be very focused on traffic. The same applies to situations in which control systems in the vehicle tend to be active (e.g. slippery roads). A distraction for the user 60 or a reduction in their attention to the traffic is to be minimized or avoided in such situations. In the event that one or more driver assistance systems are active (for example with activated lateral and longitudinal guidance at moderate speed, with stop & go), it can be concluded that the user is 60 has to pay less attention to traffic. Messages that are received via instant messaging services or messages that have already been received at an earlier point in time can preferably be transmitted in such phases of low load on the user. Vehicle parameters can typically only be recorded by the control unit itself, which has access to sensor data (e.g. speed, steering angle, accelerator pedal position, wheel speeds, vehicle configuration) or data from the vehicle's control and driver assistance systems 80 (e.g. stability control, lane departure warning, automatic spacer / cruise control, stop & go assistant).

Context parameters can further include one or more user parameters that describe a state of the user of the vehicle, for example based on the state of mind of the user, the state of alertness of the user, the length of time the user has been in the vehicle, the user's response or reaction speed, the activity of the user Users and the like. User parameters form another very important group of parameters for the context of the application. On the basis of the user parameters, conclusions can be drawn more or less directly about a state of the user and its utilization. For example, if it is determined that the user 60 for a long time without a break at the wheel of the vehicle 80 is active and is already showing signs of fatigue (e.g. prolonged blinking of the eyelids, slowed reactions, restrained or overly comfortable driving style) can draw the user's attention 60 may be increased through interaction with instant messaging services if the user is involved in the conversation with one or more participants or the (vehicle) chatbot. For example, the control unit 120 initiate the transmission of messages that are received via instant messaging services or that have already been received at an earlier point in time, which require the user 60 is animated to interact. Alternatively or additionally, the control device 120 the user 60 Propose via instant messaging to temporarily interrupt the journey and take a break. A similar approach can be useful in other cases, for example if it is recognized that the user 60 is in a rather excited or aggressive state. Here, an interaction with instant messaging can create a distraction from stressors, so that the user 60 calms down. Individual user parameters can be recorded directly by the control unit, for example in the vehicle 80 existing sensors. Often used in modern vehicles 80 a camera is provided that points to the interior, in particular the driver or user 60 is aligned. The control device can be configured based on signals from the interior camera a head of the user 60 capture. In particular, a position of the head of the user can be obtained from the signals from the interior camera 60 , in particular whose eyes, line of sight, blinking frequency, etc., can be determined. The control unit generates from these signals 120 an assumed or recorded state of the user 60 . The signals from other sensors (e.g. pulse rate, blood flow, Temperature, movement pattern) can also be used.

Context parameters can further contain one or more environmental parameters that describe an environment of the vehicle, preferably based on the type and number of other road users in the vicinity of the vehicle, the type of a road being traveled on, the state of the road being used and the like. An even more precise determination of the specific driving situation can take place on the basis of environmental parameters. For example, a driving situation with low or medium dynamics that is not critical per se (for example, no control or driver assistance systems in use) can attract increased attention on the part of the user 60 if there is very heavy traffic and / or many other road users are in the immediate vicinity of the vehicle 80 located or move. In this respect, the values of a group of parameters can also be used to verify the values of another group of parameters, for example when the user 60 appears excited despite the not very dynamic driving style and this can be explained by a very high traffic density. Environmental parameters can be recorded or determined in different ways. For example, sensors that are present in the vehicle (for example, cameras, park distance control sensors, radar, night vision cameras) can provide corresponding signals that can be used to identify other road users. Corresponding data can also be obtained from external sources (eg reports or databases on the current traffic density), which are often already held in the local navigation system and continuously updated.

Based on one or more context parameters, it can therefore be determined whether a current driving situation is suitable for instant messaging activities of the user, for example receiving or sending messages.

In a first example, the user is 60 Participant in a chat through which individual messages are received sporadically. Usually every single incoming message would be sent to the user 60 be signaled and the user 60 would have to or would manually view and read each incoming message to understand whether a response is necessary. Depending purely on chance, an incoming message can be received at a time when the user is 80 is currently in a driving situation that requires full attention, for example in heavy traffic on a motorway. A distraction for the user 60 at this time would be unfavorable for the reasons mentioned above. Such a situation is created by the system 100 recognized based on appropriate context parameters (e.g. location: Autobahn A8 , Speed: 130 km / h, traffic density: high, user condition: tense, restless). Context parameters can be recognized, for example, on the basis of rules or learned using machine learning. In the present case, the system receives 100 (e.g. the one on the control unit 120 executed software component) the incoming message (s) and saves them. The system 100 can be configured to the user 60 initially not to signal the receipt of the message (s) in order to avoid additional demands on the user in this demanding situation 60 and thereby possibly reducing their attention to the traffic. After a short time the system will determine 100 that a traffic jam has formed and the user has activated the Stop & Go assistant. Based on the fact that the situation now has only a very low dynamic and the user 60 due to the active assistance system only has to perform a few or no driving tasks, the system can 100 realize that the user 60 now has sufficient free capacity to participate in instant messaging. That is why the system signals 100 the user 60 now the message (s) that have already been received and saved, without significantly or even partially impairing his attention to the traffic situation. The user 60 can then compose a reply message, preferably also with the help of the system 100 as described in detail below.

The system 100 can cause the message (s) to be signaled in various ways, for example by simple display on a graphical user interface and / or by voice output. In some situations it may be desirable that only the user 60 even gets the message (s) signaled or transmitted (e.g. in the case of chauffeur services or ride hailing or ride sharing apps). The system 100 can be configured for this purpose, the signaling or transmission via only the user 60 to accomplish accessible elements. The system 100 can be configured, for example, to signal incoming messages via vibrations on the steering wheel. The technology required for this is usually already available in the vehicle and is used, for example, in the course of the lane departure warning or similar. Alternatively, vibration can also take place via the driver's seat or parts thereof, which is imperceptible to other occupants. The system 100 can further be configured to show (short) messages in a display element that is only visible to the driver, for example in a head-up display that can typically only be viewed from the driver's position. Other examples include viewing angle dependent displays. In some embodiments, there can also be an interaction of the user 60 also take place unnoticed, for example by knocking lightly on the steering wheel, which can be detected by a corresponding sensor, or by pressing buttons on the steering wheel that can be assigned differently depending on the respective context, for example with different response messages.

Preferably the system can 100 also be configured to automatically generate a response message based on the context parameters and without any action on the part of the user 60 to send. Such a message could inform the sender (s) of the received message (s) that this message (s) can currently not be read and / or answered (e.g. "I am currently driving my car, will contact you later", or " I can't answer right now - please call in urgent cases. ”). A response can be generated, for example, by selecting and combining already specified text modules and / or adding parameters (eg template-based). The generation can alternatively or additionally take place or be expanded using deep learning, for example using LSTMs (cf. automatic generation of text translations, captions, etc.).

• • „Ich kann gerade nicht sprechen, bin im Auto, es ist viel Verkehr. Ich bin voraussichtlich in 17 Minuten wieder ansprechbar.“
• • „Ich kann mir gerade nur Deinen Text, aber nicht Dein Video dazu anschauen, weil [...]“
• • „Ich bin gerade nicht alleine im Auto. In 35 Minuten bin ich voraussichtlich wieder alleine im Auto und kann erst dann Deine vertrauliche Nachricht lesen.“

The one or more context parameters can identify a corresponding context on the basis of which incoming messages can be processed. For example, an appropriately identified context can be used to explain a prevention. Response messages generated accordingly can include, for example:

• • “I can't speak right now, I'm in the car, there is a lot of traffic. I will probably be available again in 17 minutes. "
• • "I can only look at your text, but not your video, because [...]"
• • “I'm not alone in the car right now. In 35 minutes I will probably be alone in the car again and only then can I read your confidential message. "

• • ein oder mehrere von Ort, Zeit und Datum;
• • ein oder mehrere Wetterparameter, die ein in Bezug auf das Fahrzeug lokales Wetter beschreiben, vorzugsweise beinhaltend ein oder mehrere von Temperatur, Niederschlag, Luftdruck, Luftdruckverlauf, Luftfeuchte und Sichtweite;
• • ein oder mehrere Fahrzeugparameter, die einen Zustand eines Fahrzeugs beschreiben, vorzugsweise beinhaltend ein oder mehrere von Geschwindigkeit des Fahrzeugs, Zustand von Fahrassistenzsystemen, Zustand von Regelsystemen, Fahrtrichtung des Fahrzeuges, Reichweite des Fahrzeugs, Anzahl der Fahrzeuginsassen, aktuell befahrene Route in einem Navigationssystem des Fahrzeugs, Art einer aktuell befahrenen Straße basierend auf im Navigationssystem vorhandenem Kartenmaterial, voraussichtliche Ankunftszeit, Verkehrsaufkommen,

One or more contexts can be used by the system 100 for example, can be determined based on the one or more context parameters. The context parameters can contain one or more of the following parameters:

• • one or more of the place, time and date;
• • one or more weather parameters that describe a local weather in relation to the vehicle, preferably including one or more of temperature, precipitation, air pressure, air pressure profile, air humidity and visibility;
• • one or more vehicle parameters that describe a state of a vehicle, preferably including one or more of the speed of the vehicle, state of driver assistance systems, state of control systems, direction of travel of the vehicle, range of the vehicle, number of vehicle occupants, currently traveled route in a navigation system of the Vehicle, type of road currently being driven on based on the maps available in the navigation system, estimated time of arrival, traffic volume,

Processing can also be based on a communication partner (e.g. conversation partner, chat partner), whereby different communication partners or different types of communication partners (e.g. family, friends, acquaintances, work colleagues) should only receive certain (context) information. For this purpose, a corresponding categorization can be carried out and corresponding parameters can be provided.

Preferably the system can 100 be configured to perform an analysis of the received message (s) in order to determine a semantics of the received message (s). Such an analysis can take place, for example, by means of natural language processing (NLP). This can provide several advantages.

For one thing, the system can 100 be configured to filter out outdated or irrelevant messages. Examples of this would be the intermediate status of conversations at the time of transmission to the user 60 would be out of date. For example, an appointment for a meeting, in which a named time or a named place has changed in the meantime, would no longer be for the user 60 current. In that case it wouldn't make sense to the user 60 to inform about a no longer current time or a no longer current location. A similar procedure can be used with messages that are not sent to the user 60 are directed (e.g. in a group chat "@other user: [...]").

On the other hand, the system can 100 be configured to combine a large number of messages, in particular in group chats, to shorten them and / or to reduce their length in a content-neutral manner, so that they can be transmitted to the user 60 is as efficient as possible and free of redundancies.

In addition, the system can 100 a communication style of the user using NLP 60 capture and imitate based on it. Through the system 100 generated (reply) messages from the user 60 can thus correspond to their personal individual writing and communication style, especially with regard to the vocabulary used, expressions, idioms, emojis and the like.

The system can be further configured, based on the analysis of the incoming or received message (s), to provide one or more response messages for selection by the user, which have generic responses (e.g. "Yes", "No", "Clear") , etc.) go out. In one example, the system 100 be configured, an appointment suggestion in terms of time and location with an appointment calendar of the user 60 and to generate a nearby alternative date in the event of a collision and to formulate this in a reply message (e.g. "Meeting tomorrow at 2:30 p.m.?", "Tomorrow I will only have time for the meeting from 4:00 p.m., up to and including 5:30 p.m. Clock. Can we move? "). If necessary, the system 100 Generate several alternative appointments and provide additional answers that take the context into account.

The user 60 can then by means of a corresponding input, for example via operating elements integrated in the vehicle (e.g. voice commands, iDrive controller, buttons on the steering wheel, touchscreen) one of the system's 100 Select and send the generated reply messages. Alternatively, the user can 60 request further answers or cancel the process.

2 shows a flow diagram of a method 200 in accordance with embodiments of the present disclosure. The instant messaging process 200 for use in a vehicle 80 of a user 60 starts at step 201 .

In step 202 one or more context parameters are determined which define the context in which instant messaging is used by the user 60 describe. As described above, the context parameters are important for the further handling of the one or more messages, in particular with regard to signaling (see step 206 ) and a transmission (see step 210 ) of messages to the user 60 . The type of context parameters determined and suitable sources are described above.

In step 204 one or more incoming messages are received. This can be a single chat message or a series of messages from one or more participants in the chat. The incoming messages, like outgoing messages, can generally contain various media (e.g. emojis, images, photos, audio, video, see above).

In step 210 it is determined, based on the one or more context parameters, whether the one or more input messages are to be transmitted to the user. The factors discussed above play a role here. In step 210 the current context, ie driving situation, state of the user, environmental parameters, environment and the like is evaluated in order to determine whether the situation allows the user 60 devotes part of its attention to instant messaging. As described above, a large number of factors can be taken into account here. In particular, it can also be determined whether the instant messaging draws the user's attention 60 could increase, for example, if the user is tired or stressed.

In step 212 are then, if the one or more incoming messages to the user 60 to be transmitted, the one or more input messages to the user 60 transmitted. As described above, different transmission modalities are available here that can be used depending on preference and / or context. The procedure ends at step 214 .

• • ein oder mehrere optisch wahrnehmbare Signale, vorzugsweise wobei die ein oder mehreren optisch wahrnehmbaren Signale ein oder mehrere der folgenden umfassen: Signale, die über ein Head-Up Display (HUD) ausgegeben werden; Signale, die über eine zentrale Instrumentenkombi angezeigt werden; und Signale, die über ein zentrales Informationsdisplay (CID) angezeigt werden;
• • ein oder mehrere akustisch wahrnehmbare Signale, vorzugsweise wobei die ein oder mehreren akustisch wahrnehmbaren Signale ein oder mehrere der folgenden umfassen: Signale, die über eine Lautsprecheranlage des Fahrzeugs ausgegeben werden; Signale, die über einen akustischen Signalgeber ausgegeben werden; und
• • ein oder mehrere taktil wahrnehmbare Signale, vorzugsweise wobei die ein oder mehreren taktil wahrnehmbaren Signale ein oder mehrere der folgenden umfassen: Vibrations- bzw. Klopfsignale, die über Bedienelemente des Fahrzeugs ausgegeben werden; Vibrations- bzw. Klopfsignale, die über einen Sitz des Fahrzeugs ausgegeben werden.

In a preferred embodiment of the method 200 can be done in one step 206 , preferably following step 204 is carried out, it is determined based on the one or more context parameters whether the one or more input messages are to be signaled to the user. If the one or more input messages are to be signaled to the user, this can also be done in an optional step 208 to the user 60 signaled that one or more incoming messages have been received. The signaling preferably comprises one or more of the following signals:

• • one or more optically perceptible signals, preferably wherein the one or more optically perceptible signals include one or more of the following: signals that are output via a head-up display (HUD); Signals displayed via a central instrument cluster; and signals that are displayed on a central information display (CID);
• One or more acoustically perceptible signals, preferably wherein the one or more acoustically perceptible signals comprise one or more of the following: signals that are output via a loudspeaker system of the vehicle; Signals that are output via an acoustic signal generator; and
• • one or more tactilely perceptible signals, preferably wherein the one or more tactilely perceptible signals are one or more of the include the following: vibration or knock signals that are output via controls of the vehicle; Vibration or knocking signals that are output via a seat in the vehicle.

In a further preferred embodiment, the method alternatively or additionally comprises processing the one or more input messages using natural language processing (NLP). The NLP can preferably comprise a content-neutral reduction of a number and / or length of the one or more input messages and / or a removal of outdated messages and / or for the user 60 irrelevant parts of the one or more incoming messages. A sequence, order and / or sorting that is present within the one or more input messages during reduction and / or removal is preferably retained.

In a further preferred embodiment, the method alternatively or additionally comprises generating a message from the user 60 based on the one or more input messages and the one or more context parameters. The message preferably includes a reply to one or more incoming messages. Additionally or alternatively, the message contains one or more of text, emoticon, image, audio data and video data. This preferred embodiment can also automatically send the message from the user 60 without his involvement.

Although the invention has been illustrated and explained in more detail by preferred exemplary embodiments, the invention is not restricted by the disclosed examples and other variations can be derived therefrom by the person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that there is a multitude of possible variations. It is also clear that embodiments cited by way of example really only represent examples that are not to be interpreted in any way as a limitation, for example, of the scope of protection, the possible applications or the configuration of the invention. Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without the To leave the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 2007/0162550 A1 [0006]
• US 9947319 [0007]

Non-patent literature cited

• Standard SAE J3016 [0037]

Claims (10)

Instant messaging method (200) for use in a vehicle (80) of a user (60), the method comprising: Determining (202) one or more context parameters that describe the context of a use of the instant messaging by the user; Receiving (204) one or more input messages; Determining (210), based on the one or more context parameters, whether the one or more input messages are to be transmitted to the user; and if the one or more input messages are to be transmitted to the user, transmitting (212) the one or more input messages to the user. The method (200) according to the preceding claim, further comprising: Determining (206), based on the one or more context parameters, whether the one or more input messages are to be signaled to the user; and if the one or more input messages are to be signaled to the user, signaling (208) to the user that one or more input messages have been received; preferably wherein the signaling comprises one or more of the following signals: one or more optically perceptible signals, preferably wherein the one or more optically perceptible signals comprise one or more of the following: signals that are output via a head-up display (HUD); Signals displayed via a central instrument cluster; and Signals that are displayed on a central information display (CID); one or more acoustically perceptible signals, preferably wherein the one or more acoustically perceptible signals comprise one or more of the following: signals that are output via a loudspeaker system of the vehicle; Signals that are output via an acoustic signal generator, and one or more tactilely perceptible signals, preferably wherein the one or more tactilely perceptible signals include one or more of the following: vibration or knocking signals that are output via control elements of the vehicle; Vibration or knocking signals that are output via a seat in the vehicle. The method (200) according to any one of the preceding claims, further comprising: Processing the one or more input messages using natural language processing (NLP); preferably comprising: content-neutral reduction of a number and / or length of the one or more input messages; and or Removing obsolete and / or for the user (60) irrelevant parts of the one or more input messages; in which a sequence, order and / or sorting present within the one or more input messages is preferably retained during reduction and / or removal. The method (200) according to any one of the preceding claims, further comprising: Generating a message from the user based on the one or more input messages and the one or more context parameters, the message preferably containing a response to one or more incoming messages and / or the message one or more of text, emoticon, image, audio data and video data includes; and preferably automatically sending the user's message without any action on the part of the user. Procedure according to Claim 4 , further comprising: determining, based on the one or more context parameters, whether the user's message should be sent automatically; and if the user's message is to be sent automatically, the user's message is automatically sent without any action on the part of the user. Method (200) according to Claim 4 , further comprising: generating a plurality of alternative messages from the user based on the one or more input messages and the one or more context parameters; Delivering the multiple alternative messages to the user; Receiving input from the user to select at least one of the plurality of alternative messages; and sending the selected at least one message. Method (200) according to Claim 6 wherein receiving an input from the user for selecting at least one of the plurality of alternative messages comprises: detecting a tactile input via a sensor in the steering wheel, preferably wherein the sensor is configured to detect vibrations or shocks; Detecting an acoustic input from the user; and detecting activation of a control element of the vehicle by the user. The method (200) according to any one of the preceding claims, wherein the context parameters include one or more of the following parameters: one or more of location, time and date; one or more weather parameters that describe a local weather in relation to the vehicle, preferably including one or more of temperature, precipitation, air pressure, air pressure profile, air humidity and visibility; one or more vehicle parameters that describe a state of a vehicle, preferably including one or more of the speed of the vehicle, state of driver assistance systems, state of control systems, direction of travel of the vehicle, range of the vehicle, number of vehicle occupants, currently traveled route in a navigation system of the vehicle , Type of a currently traveled road based on map material present in the navigation system, degree of attention of the user (60) available for use of instant messaging, and an expected arrival time of the vehicle (80); one or more user parameters that describe a state of the user of the vehicle, preferably including one or more of the state of mind of the user, state of alertness of the user, length of stay of the user in the vehicle, response or reaction speed of the user and activity of the user; and one or more environmental parameters that describe the surroundings of the vehicle, preferably including the type and number of other road users in the surroundings of the vehicle, the type of road being traveled on and the state of the road being traveled on. Instant messaging system (100) for use in a vehicle of a user, the system comprising a control unit (120) which is configured to carry out the method (200) according to one of the preceding claims. Vehicle (80) comprising an instant messaging system (100) according to the preceding claim.

Images