WO2020020464A1

WO2020020464A1 - Computer-implemented method and data processing system for predicting return of a user to a vehicle

Info

Publication number: WO2020020464A1
Application number: PCT/EP2018/070386
Authority: WO
Inventors: Alvin CHIN; Jilei Tian; Daria POPIV
Original assignee: Bayerische Motoren Werke Aktiengesellschaft
Priority date: 2018-07-27
Filing date: 2018-07-27
Publication date: 2020-01-30
Also published as: DE112018007858T5

Abstract

A computer-implemented method for predicting return of a user to a vehicle is provided. The method includes determining a first probability for return of the user to the vehicle based on a first trained model and at least data on a current location of the user and a current location of the vehicle. The first trained model is trained based on historic driving data of the user. The method further includes determining a second probability for return of the user to the vehicle based on a second trained model and contextual data from a mobile communication device of the user that indicates a current activity of the user. The second trained model is trained based on historic contextual data of the user. Additionally, the method includes combining the first probability and the second probability to a combined probability, and determining that the user returns to the vehicle if the combined probability exceeds a threshold value. If it is determined that the user returns to the vehicle, the method further includes controlling at least one sensor of the mobile communication device and/or the vehicle to collect sensor data capable of indicating a movement of the user. The method additionally includes sending a notification to the mobile communication device if the sensor data indicates that the user is approximately approaching the vehicle. The notification comprises a request to the user whether the user desires to have personalization of the vehicle.

Description

COMPUTER-IMPLEMENTED METHOD AND DATA PROCESSING SYSTEM FOR PREDICTING RETURN OF A USER TO A VEHICLE

Field

The present disclosure relates to driver return prediction for personalization in the vehicle. In particular, examples relate to a computer-implemented method and a data processing system for predicting return of a user to a vehicle.

Background

Personalization of settings in a vehicle is becoming an expected feature nowadays. In a pri vately owned vehicle, which is being driven by different members of the household, as well as in shared/fleet vehicles, customers expect to get into the car with their already pre-tuned favorite settings available from the very start. In order to have personalization in the vehicle, different approaches are being investigated and are partially already adopted in the serial pro duction. For example, the user’s favorite settings may be explicitly input via a key fob storing the settings, or the vehicles and customer smart phones may be connected to the manufac turer’s secured backend to automatically leam behavioral patterns in order to increase the benefit of personalized features.

To enable vehicle personalization, it is beneficial to recognize the driver returning to the ve hicle with the intention to drive away before the driver actually enters the vehicle. A predicted time of departure may help to leam when the driver may return back to the vehicle, but the driver does not always follow the predicted behavior. In conventional approaches, the pre dicted time may vary (for example up to 30 minutes) from the actual return and thus cause inaccurate false starts of electrical components in the vehicle. In addition, classic prediction may fail for non-frequent trips, i.e. deviations from the driver’ s usual behavior, or if the driver returns back to the vehicle but may be just moving things in/out of the vehicle and not intend ing to drive away.

Hence, there may be a demand for an improved technique for predicting return of a user to a vehicle. Summary

The demand may be satisfied by examples described herein.

An example relates to a computer-implemented method for predicting return of a user to a vehicle. The method comprises determining a first probability for return of the user to the vehicle based on a first trained model and at least data on a current location of the user and a current location of the vehicle. The first trained model is trained based on historic driving data of the user. The method further comprises determining a second probability for return of the user to the vehicle based on a second trained model and real time contextual data from a personal device, e.g. a mobile device, of the user that indicates a current activity of the user. The second trained model is trained based on historic contextual data of the user and used with real time context data, e.g. generated from multiple sensors. Additionally, the method comprises combining the first probability and the second probability to a combined probabil ity, and determining that the user returns to the vehicle if the combined probability exceeds a threshold value. If it is determined that the user returns to the vehicle, the method further comprises controlling at least one sensor of the mobile communication device and/or the ve hicle to collect sensor data capable of indicating a movement of the user. The method addi tionally comprises sending a notification to the mobile communication device if the sensor data indicates that the user is approximately approaching the vehicle. The notification com prises a request to the user whether the user desires to have personalization of the vehicle.

Another example relates to a non-transitory machine readable medium having stored thereon a program having a program code for performing the method as described herein, when the program is executed on a processor.

A further example relates to a data processing system for predicting return of a user to a vehicle. The data processing system comprises one or more processors configured to deter mine a first probability for return of the user to the vehicle based on a first trained model and at least data on a current location of the user and a current location of the vehicle. The first trained model is trained based on historic driving data of the user. The one or more processors are further configured to determine a second probability for return of the user to the vehicle based on a second trained model and contextual data from a mobile communication device of the user that indicates a current activity of the user. The second trained model is trained based on historic contextual data of the user. Additionally, the one or more processors are configured to combine the first probability and the second probability to a combined probability, and to determine that the user returns to the vehicle if the combined probability exceeds a threshold value. If it is determined that the user returns to the vehicle, the one or more processors are configured to control at least one sensor of the mobile communication device and/or the ve hicle to collect sensor data capable of indicating a movement of the user. Additionally, the one or more processors are configured to send a notification to the mobile communication device if the sensor data indicates that the user is approximately approaching the vehicle. The notification comprises a request to the user whether the user desires to have personalization of the vehicle.

Brief description of the Figures

Some examples of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

Fig. 1 illustrates a flowchart of an example of a computer-implemented method for predicting return of a user to a vehicle;

Fig. 2 illustrates a flowchart of an example of determining a combined probability;

Fig. 3 illustrates a flowchart of an example of determining a probability for return of a user to a vehicle based on contextual data;

Fig. 4 illustrates an example of user behavior; and

Fig. 5 illustrates a flowchart of an example of a driver return prediction.

Detailed Description

Various examples will now be described more fully with reference to the accompanying draw ings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and/or regions may be exaggerated for clarity. Accordingly, while further examples are capable of various modifications and alternative forms, some particular examples thereof are shown in the figures and will subsequently be described in detail. However, this detailed description does not limit further examples to the particular forms described. Further examples may cover all modifications, equivalents, and alternatives falling within the scope of the disclosure. Same or like numbers refer to like or similar elements throughout the description of the figures, which may be implemented iden tically or in modified form when compared to one another while providing for the same or a similar functionality.

If two elements A and B are combined using an“or”, this is to be understood to disclose all possible combinations, i.e. only A, only B as well as A and B, if not explicitly or implicitly defined otherwise. An alternative wording for the same combinations is“at least one of A and B” or“A and/or B”. The same applies, mutatis mutandis, for combinations of more than two elements.

The terminology used herein for the purpose of describing particular examples is not intended to be limiting for further examples. Whenever a singular form such as“a”,“an” and“the” is used and using only a single element is neither explicitly or implicitly defined as being man datory, further examples may also use plural elements to implement the same functionality. Likewise, when a functionality is subsequently described as being implemented using multi ple elements, further examples may implement the same functionality using a single element or processing entity. It will be further understood that the terms“comprises,”“comprising”, “includes” and/or“including”, when used, specify the presence of the stated features, integers, steps, operations, processes, acts, elements and/or components, but do not preclude the pres ence or addition of one or more other features, integers, steps, operations, processes, acts, elements, components and/or any group thereof.

Unless otherwise defined, all terms (including technical and scientific terms) are used herein in their ordinary meaning of the art to which the examples belong.

Fig. 1 illustrates a computer-implemented method 100 for predicting return of a user to a vehicle. For example, a data processing system comprising one or more processors may be configured to execute the method 100. The vehicle may be any apparatus that comprises wheels driven by an engine (and optionally a powertrain system). In some examples, the ve hicle may be a private vehicle or a commercial vehicle. In particular, the vehicle may be an automobile, a truck, a motorcycle, or a tractor.

The method 100 comprises determining (calculating) 102 a first probability for return of the user to the vehicle based on a first trained model and at least data on a current location of the user and a current location of the vehicle. The first trained model is trained based on historic driving data of the user.

The method 100 further comprises determining (calculating) 104 a second probability for re turn of the user to the vehicle based on a second trained model and contextual data from a mobile communication device of the user that indicates a current activity of the user. The second trained model is trained based on historic contextual data of the user.

Additionally, the method comprises combining 106 the first probability and the second prob ability to a combined probability.

The determination and the combination of the first and the second probability is further illus trated with more details in Fig. 2. Historic driving data 202 of the user is used for training the first model 210. As indicated in Fig. 2, the historic driving data 202 may comprise information about past trips / travel destinations of the user. Accordingly, the user’s departure/travel be havior may be learned by the first model 210. For example, machine learning may be used to train the first model 210 based on the historic driving data 202.

The first trained model 210 is subsequently used to predict the next departure of the user based on the data related to the past trips. In particular, data on the current location of the user and the current location of the vehicle are used for determining the probability for the next trip and, hence, the next return of the user to the vehicle. In some example, determining 102 the first probability for return of the user may further be based on data on a current date and data on a current time. In some examples, the first probability may be expressed as follows:

P( driver return datetime \ history, current datetime and location ) (1), with driver return datetime denoting the next point in time for departure/return of the user to the vehicle, and current datetime and location denoting the current date, the current time, the current location of the user and the current location of the vehicle. In other words, the first trained model 210 may further output a time for the return of the user to the vehicle.

The next point in time for departure/retum of the user to the vehicle may be the most likely point in time among a plurality of potential future (predicted) trips, i.e. driver return datetime =

argmax P( driver return datetime \ history, current datetime and location ) (2). model

In some examples, the first trained model 210 may further output a travel destination of the user associated to the return of the user to the vehicle. For example, the first trained model 210 may be used for determining the probability that a user will return to the vehicle for travelling to a destination he/she usually visits (e.g. Sunday mass at church, or a shopping mall on Saturday afternoon).

In parallel, a departure prediction based on the contextual data 204 is made using the second model 220. The contextual data 204 indicates a current activity of the user. For example, the contextual data 204 may be the user’s activity that is derived from sensor data of the mobile communication device (e.g. a smartphone, a laptop-computer or a tablet-computer). The user’s activity may, e.g., be determined by applications pre-installed on, or integrated into the operating system of the mobile communication device. For example, sensor data of an accel erometer of the mobile communication device may be used to determine/recognize the user’s activity as stationary, walking, running, biking, transportation, automotive, etc. Similarly, a barometer of the mobile communication device may be used to determine/recognize a floor the user is located or a change of the floor (e.g. when the user takes a lift in a parking building). Also a gyroscope and heading may be used to determine/recognize the user’s moving direc tion. Further, the user’s position may be used to determine a category of his/her current loca tion (e.g. a specific Point-of-Interest or POI, such as a shopping mall, church, home etc.).

In other words, the contextual data 204 indicating the current activity of the user may com prise sensor data from at least one sensor of the mobile communication device and/or a type of the user’s current location. Additionally, determining 104 the second probability for return of the user to the vehicle may be based on contextual data 204 from the vehicle that indicate the current activity of the user. For example, proximity sensors of the vehicle may be used to determine a movement of the user relative to the vehicle.

As indicated in Fig. 2, the contextual data 204 indicating the current activity of the user may be contextual data collected by the mobile communication device since the user most recently left the vehicle. This may become more evident from Fig. 3 illustrating a more detailed view of the determination of the second probability for return of the user to the vehicle.

Historic contextual data 206 of the user is used to train the second model 220. For example, the user’s contextual data may be collected for past trips. The historic contextual data 206 may, e.g., comprise contextual data of the user from a historic time span starting with parking the vehicle and coming back to the vehicle. For example, the historic contextual data 206 may comprise the activity pattern of the user at a destination location of one of the user’s past trips. In other words, the second model 220 may be trained based on the contextual data from the user’s everyday life in order to leam mobility patterns of the user (i.e. a kind of a user profile). Individual pieces of the contextual data 204 and 206 may be understood as features (e.g. a first movement direction of the user, a second movement direction of the user, change of floor by the user etc.).

In some examples, the second model 220 may be trained as supervised classification task. For example, a recurrent neural network may be used for the second model 220. Accordingly, a contextual activity sequence may be taken as input and labelled as output. In other examples, a convolutional neural network may be used by taking a feature as input and labelling it as output. For an activity sequence, all features may be embedded since the features are placed in a sequential order. Alternatively, a support vector machine or a decision tree may be used for the second model 220. It is evident from the above examples, that a variety of different approaches/structures/techniques of machine learning may be used for the second model 220. Hence, the proposed concept is not limited to one of the above examples.

As a result, the second trained model 220 may in some examples comprise at least one activity pattern of the user that indicates return of the user to the vehicle (e.g. for a specific location, POI and/or day and time), wherein the activity pattern is learned from the historic contextual data 206 of the user. The contextual data 204 indicating the current activity of the user is input to the second trained model 220 to identify a mobility pattern of the user. Additionally, determining 104 the second probability for return of the user to the vehicle may be based on data on a current location of the vehicle, data on a current date and data on a current time. For example, determining 104 the second probability may comprise comparing the current activity of the user as indicated by the contextual data 204 to the at least one activity pattern of the user learned by the second trained model 220.

In some examples, the second probability may be expressed as follows:

P( driver return datetime\ context ) (3) with context denoting the contextual data 204 indicating the current activity of the user (e.g. current datetime and location, user’s activity sequence, proximity to the vehicle, usage of Near-Field Communication (NFC), Bluetooth or WiFi).

Returning back to the process flow illustrated in Figs. 1 and 2, the first probability P( driver return datetime \ history, current datetime and location ) and the second probability P( driver return datetime \ context ) are combined to the combined proba bility P( driver return datetime ). For example, the first probability and the second prob ability may be added or multiplied. An example for mathematically linking both probabilities to a combined probability for return of the user to the vehicle with a certain date and time (i.e. a certain datetime) is given in the following expression (4):

P( driver return datetime \ history, context, current datetime and location )

oc log P( driver return datetime \ history, current datetime and location )

+ log P( driver return datetime \ context ) The first term is estimated using historic data for driver departure behavior as described above and may, hence, be understood as driver (user) return prediction based on behavior. The sec ond term is estimated using a supervised machine learning classifier as described above and may, hence, be understood as driver return prediction based on context. The logarithmic op eration may be introduced to optimize the complexity.

Combining 106 the first probability and the second probability to the combined probability may in some examples be done using a weighting factor for at least one of the first probability and the second probability. Referring to above expression (4), the combined probability may, e.g., be determined according to the following expression (5):

P( driver return datetime )

oc a * logP( driver return datetime \ history, current datetime and location ) + (1 — a) * logP( driver return datetime \ context ) with a denoting the weighting factor (0 < a < 1).

For example, the weighting factor may allow to balance the contribution of the user behavior patterns and the contextual inference to the combined balance. The weighting factor may, e.g., be based on a (explicit) feedback of the user and/or be based on a comparison of historic telemetry data of the vehicle and historic predictions of the first trained model 210 (i.e. be based on implicit feedback). For example, the historic telemetry data of the vehicle may be used as ground truth for the trips predicted by the first trained model 210.

The method 100 determines 108 that the user returns to the vehicle if the combined probability exceeds a threshold value T. Similar to the weighting factor, also the threshold value T may be based (optimized) on a (explicit) feedback of the user. Accordingly, the receiver operating characteristic of the method 100 may be improved.

If it is determined that the user returns to the vehicle, the method 100 further comprises con trolling 110 at least one sensor of the mobile communication device and/or the vehicle to collect sensor data capable of indicating a movement of the user. In other words, if the prob ability is high enough that the user returns to the vehicle with the intention to drive away, one or more sensors of at least one of the mobile communication device and the vehicle are used to verify whether the user is really returning to the vehicle.

For example, sensor data of the above described sensors of the mobile communication device may be collected and analyzed to determine the movement of the user. On the vehicle side, any sensor of the vehicle that allows to determine proximity of the user to the vehicle may be used. For example, sensors of the vehicle for object detection (such as a radar sensor, a video camera or a LIDAR sensor) may be used. Similarly, communication facilities of the vehicle may be used as proximity sensors. For example, Bluetooth or WiFi communication facilities of the vehicle may be used to detect presence and/or movement of the user (i.e. his/her mobile communication device) in the range of around 100 meters around the vehicle. Similarly, NFC communication facilities of the vehicle may be used to detect presence and/or movement of the user (i.e. his/her mobile communication device) in the range of a few meters around the vehicle. In other words, controlling 110 the at least one sensor of the mobile communication device and/or the vehicle may comprise controlling at least one sensor of the vehicle to collect sensor data capable of indicating movement of the user relative to the vehicle.

If it is determined that the user returns to the vehicle and if the vehicle is in a sleep-mode, the method 100 further comprises controlling the vehicle to wake up. The sleep-mode is an oper ation mode of the vehicle with reduced power consumption and reduced functionality com pared to a fully operational mode of the vehicle. For example, some or most circuitry, features or components of the vehicle may be deactivated in the sleep mode. After waking up the vehicle, the sensors of the vehicle may be used for collecting sensor data capable of indicating a movement of the user. By waking up the vehicle only if the probability is high enough that the user returns to the vehicle with the intention to drive away, unnecessary (false) wake up of the vehicle may be avoided. Accordingly, the vehicle may be operated more often and/or for longer times in the sleep-mode. A power consumption of the vehicle may, hence, be re duced by the method 100.

As described above, the first trained model may output a first (predicted) time for the return of the user to the vehicle. Similarly, the second trained model may output a second (predicted) time for the return of the user to the vehicle. In order to enable confirmation that the user really returns to the vehicle, method 100 may comprise controlling 110 the at least one sensor to start collecting the sensor data a predefined time instant prior to one of the first time and second time (e.g. one, two, three, four, five or more minutes before the first time or the second time). Accordingly, sensor data indicating return of the user to vehicle may be collected and analyzed well before the user reaches the vehicle.

If the sensor data indicates that the user is (approximately) approaching the vehicle, the method 100 additionally comprises sending 112 a notification to the mobile communication device. The notification comprises a request to the user whether the user desires to have per sonalization of the vehicle. Accordingly, the user is enabled to decide about the personaliza tion of the vehicle. Due to the combination of the two trained models and the confirmation via the sensor data of the mobile communication device and/or the vehicle, the probability that the user actually returns to the vehicle with the intention to drive away may be determined with high probability so that false notification of the user may be avoided.

If a notification indicating that personalization of the vehicle is desired by the user is received from the mobile communication device, the method 100 may further comprise controlling the vehicle to adjust at least one setting of the vehicle according to a stored or learned setting of the user. Accordingly, the vehicle is personalized according to the user’s desire when he/she arrives at the vehicle.

The method 100 may allow to personalize settings in the vehicle just before the user enters the vehicle to start driving. Hence, the user is not required to personalize the settings when he/she is in the vehicle. Accordingly, disruption of the user’s attention as well as cumbersome and difficult situations may be avoided. For example, the method 100 may allow to adjust the seat position of the driver seat according to a personalized setting stored in the vehicle for the user. The user is not required to sit in the vehicle just for adjusting the seat position.

The proposed concepts described above in connection with the method 100 leverages machine learning to learn the user’ s mobility patterns and to predict the next mobility state for starting to return back to the vehicle. Furthermore, the proposed concept leverages proximity sensors in the vehicle to confirm that the driver is indeed returning back and prevents false alarms for powering up the vehicle’s electronic components. Hence, battery power of the vehicle may be conserved. Further, the mobility patterns and the proximity sensors are fused together (combined) to a final decision for driver return prediction. The proposed concept may be un derstood as a context mobility framework for enabling personalization of vehicle settings that minimizes power consumption and optimizes the user experience.

In the following an exemplary relation between the driver return prediction and the user’s actual driving behavior is explained in connection with Fig. 4. Two scenarios will be ex plained below: the user makes a predicted trip or the user makes an ad-hoc trip.

A predicted trip may be understood as a regular trip that the user makes regularly. For exam ple, the user may drive from home to 11 AM mass every Sunday and drive from church to home after the mass at 12:30 PM. Regular trips may allow to predict rather simple and deter ministic behavior when the user will start the vehicle and combined with the out-of-vehicle activity, the time to return back to the vehicle may be predicted.

An ad-hoc trip may be understood as a trip that is not frequently driven. However, the type of destination (e.g. POI category) may be similar to others that the user is regularly driving to. For example, the user may drive to a new shopping mall in another state but have driven to other shopping malls. It is likely that use will follow the same mobility behavior (e.g. average stay in the shopping mall is about two hours). Another example might be going to a church in a new city the user hasn’t been to before. However, based on an average stay time at the user’ s home church, it may be predicted when the mass in the new church will finish and the result may be used for driver return prediction.

In the following examples, it is assumed that the activity of the user outside the vehicle (e.g. walking, running, stationary) can be gathered (e.g. via the user’s smartphone). Further, it is assumed that categories of POIs can be gathered (e.g. from the user’s smartphone).

First, a regular trip from home to church and back is explained in connection with Fig. 4. From previous trips, the usual departure location (i.e. home or church), the usual departure time (e.g. Sunday at 12:21 PM), the usual arrival location (i.e. church or home) and the usual arrival time (e.g. 12:34 PM) is learned. For example, from church to home the usual departure time may be 12:21 PM on a Sunday. In other words, the first model takes into account the user’s behavior as learned from historic driving data of the user. Further, the user’s activity at the location where he/she stayed and the vehicle are gathered for previous trips. For example, the raw data (e.g. from the smartphone of the user) may be smoothed (filtered) and a pattern for returning to the vehicle may be recognized. The activity pattern may, e.g., be Walking (W) -> Automotive (A). Further, the average walking duration from the location to the vehicle may be calculated. For example, the average walking time from church to the vehicle may be three minutes.

On a Sunday, the user is initially at his home in the morning as indicated in the very left area of the timeline illustrated in Fig. 4. From the first trained model, it is therefore known that the next trip is coming from home to church. After that, the current activity pattern of the user at home may be input to the second trained model. For example, after a stationary state S during the stay at home, a walking state W may be detected (indicating end of visit at home). The second trained model may predict the next activity state (e.g. using a Markov model) upon detection of the walking state. If the next state is automotive A, then the average for getting into the vehicle is calculated. For example, if the average walking time from home to the vehicle is learned to be three minutes from previous trips, and the user starts the walking activity at 11:45 AM, the approximate arrival time at the vehicle is 11:48 AM.

For confirming the driver’s return to the vehicle, the user’s smartphone may, e.g., check at 11:46 AM or at 11:47 AM whether the user is getting close to the vehicle. Similarly, sensor data from the vehicle’s proximity sensors (e.g. Bluetooth beacon, electromagnetic or ultra sonic sensor) may be used for determining approximate approach of the user to the vehicle. If approximate approach of the user to the vehicle is determined, this confirms with high probability that the user is returning to the vehicle. Accordingly, a notification whether to start personalization is sent to the user. For example, given a threshold for the driver return predic tion notification of one minute before entering the vehicle, the notification may be sent at 11:47 AM to the user’s smartphone. Accordingly, the user may select to personalize the ve hicle, so that the vehicle is personalized when the user arrives at the vehicle.

After arriving at the vehicle, the user’ s activity is again stationary S before it changes to au tomotive A as soon as the user starts driving with the personalized vehicle to church. As indicated in Fig. 4, at least one of the first trained model and the second trained model may further output a predicted departure time to church. Upon arrival at church, the user will park the vehicle so that the activity changes from auto motive A to stationary S. Again, at least one of the first trained model and the second trained model may further output a predicted arrival time at church. The time difference between the predicted arrival/departure time and the actual arrival/departure time may be small (e.g. a few minutes or even less).

After parking the vehicle, the user starts walking to church, i.e. changes his/her activity to walking W. When the user reaches church (i.e. starts the church visit), the activity changes to stationary S. After the mass has finished, the user walks back to the vehicle, i.e. changes his/her activity to walking W after finishing the visit at church.

Similarly to what is described before for going from home to church, the usual departure time for going from church back home is known to the first trained model so that the return of the user to the vehicle for going home may be predicted based on the known user behavior and his/her activity pattern in the same manner as described above.

Accordingly, the vehicle may be personalized before the user reaches the vehicle to drive home.

If the user does not drive to his home church on Sunday but to a church in another city (which may be understood as an ad-hoc trip), the proposed concept may also allow personalization before the user reaches the vehicle. It may be assumed that the user returns to the same loca tion where he/she parked the vehicle. Accordingly, the activity when he/she leaves or parks the vehicle for going to the destination may be recorded. For example, the time and the loca tion of parking the vehicle may be gathered, and the activities from the vehicle to the desti nation may be recorded (e.g. automotive A -> walking W -> stationary S). Again, the average walking time from the vehicle to the destination may be calculated. Referring to the example of Fig. 4, the vehicle is parked outside the new church at 10:52 AM and the user walks to the church so that he/she arrives at the church at 10:57 AM. Accordingly, an average walking time from the vehicle to the church may be recorded as five minutes.

Next, the stay duration at the new church may be predicted based on previous similar desti nations using the first trained model. For example, the average stay duration at the home church of the user may be used. If the average stay time at the home church is 80 minutes, the stay duration at the new church is assumed to be 80 minutes.

The above information may be used to predict the next time for leaving from the new church. When the user exits the church, i.e. changes its activity from stationary S to walking W, the second trained model may be used to predict if the next activity is automotive A. From the previous recording, the user’ s activity pattern for walking from the vehicle to the new church is known. Based on this pattern, the second trained model may predict the next activity state. Further, the average time for walking from the vehicle to the new church is known. Accord ingly, the time for walking from the new church to the parked vehicle may be predicted.

For example, the next time to leave may be calculated to be 85 minutes (average stay duration + average walking time from vehicle to the new church) after the arrival at the new church at 10:57 AM, i.e. the next time to leave would be 12:22 PM.

Again, the user’s smartphone and the proximity sensors of the vehicle may be used to check whether the user is getting close to the vehicle. For example, the smartphone and the proxim ity sensors may check at 12:20 PM or at 12:21 PM whether the user is getting close to the vehicle. If approximate approach of the user to the vehicle is determined, this confirms with high probability that the user is returning to the vehicle. A notification whether to start per sonalization is sent to the user so that the vehicle may be personalized when the user arrives at the vehicle.

A flowchart of another example for driver return prediction is illustrated in Fig. 5. In an initial step 505 locations of a car (as an example for a vehicle) and a driver’s phone are collected. The collected locations are used in a step 510 to train an algorithm for predicting trips. The trained algorithm outputs one or more predicted trips in a step 515. For the one or more pre dicted trips, the probability for return of the driver to the car is determined based on his/her behavior in step 520. Further, a time for return of the driver to the car is predicted in step 525. If it is determined in a step 530 that the current time corresponds substantially to the predicted time, the method proceeds to step 550 explained below. If not, it is waited and step 530 is repeated. In parallel contextual data is collected from the driver’s phone and proximity sensors of the car in a step 535. In step 540, a probability for return of the driver to the vehicle is calculated based on the contextual data. Further, a time for return of the driver to the car is predicted in step 545. Both probabilities are combined in step 550 to check whether it is likely that the driver returns to the vehicle. If it is determined in step 555 that return of the driver to the vehicle is likely, it is proceeded to step 560. If return of the driver to the car is not likely, the process goes back to start.

In step 560, it is checked whether the car is awake or in a sleep-mode. If the car is not awake, the car is woken in step 565. If the car is awake, the process directly goes to step 570. In step 570, the proximity sensors of the car are used to check whether the driver is really approaching the car. If the driver is approaching the car, a notification requesting whether the driver desires that the car is personalized is sent out to the phone of the driver in step 575.

Accordingly, also the process flow illustrated in Fig. 5 may allow prediction of return of the driver to the car in accordance with the proposed concept.

Given data on a driver’s driving behavior patterns, a driver’s mobility patterns and a vehicle’s proximity sensors, the proposed concept may allow to predict when the driver will return back to the vehicle based on the current and/or future context so that the vehicle can have appro priate personalized settings set up for the driver before starting the trip.

The proposed concept provides a system and framework that first collects context data from a driver’ s everyday life and second processes it to automatically learn the user profile, identify a driver’s mobility patterns, and extract a driver’s driving behavior patterns. Based on this, an algorithm is created to predict the next likely activity that the driver will take based on previ ous and current activities and the driver’s past mobility and driving behavior patterns. Com bined with the next predicted trip and the proximity of the driver to the vehicle, it may be confirmed with high probability that the driver is returning back to the vehicle to start driving.

The technical framework as proposed is a novel system and a framework for predicting when the driver will return to the vehicle that leverages driver’ s data from behavior learning in past data and contextual prediction with real time sensor data. It uses rich data and combines pre dicted time of departure for a trip (if known), the mobility patterns of the driver, the behavioral patterns of the driver through learning the user profile, the user and vehicle environments and proximity sensors in the vehicle to determine if the driver is near the vehicle. It optimizes its performance using user’s explicit feedback and implicit telemetry data as ground truth.

The proposed concept may enable predictive user experience and optimal vehicle manage ment. For example, it may be an enabler for smart intelligence for predictive personalization in the vehicle as well as optimal management of vehicle (e.g. improved power efficiency).

The aspects and features mentioned and described together with one or more of the previously detailed examples and figures, may as well be combined with one or more of the other exam ples in order to replace a like feature of the other example or in order to additionally introduce the feature to the other example.

Examples may further be or relate to a computer program having a program code for perform ing one or more of the above methods, when the computer program is executed on a computer or processor. Steps, operations or processes of various above-described methods may be per formed by programmed computers or processors. Examples may also cover non-transitory and machine readable program storage devices such as digital data storage media, which are machine, processor or computer readable and encode machine-executable, processor-execut able or computer-executable programs of instructions. The instructions perform or cause per forming some or all of the acts of the above-described methods. The program storage devices may comprise or be, for instance, digital memories, magnetic storage media such as magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media.

The description and drawings merely illustrate the principles of the disclosure. Furthermore, all examples recited herein are principally intended to be only for illustrative purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor(s) to furthering the art. All statements herein reciting principles, aspects, and examples of the disclosure, as well as specific examples thereof, are intended to encompass equivalents thereof.

A block diagram may, for instance, illustrate a high-level circuit diagram implementing the principles of the disclosure. Similarly, a flow chart, a flow diagram, a state transition diagram, a pseudo code, and the like may represent various processes, operations or steps, which may, for instance, be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such a computer or processor is explicitly shown. Methods disclosed in the specification or in the claims may be implemented by a device hav ing means for performing each of the respective acts of these methods.

It is to be understood that the disclosure of multiple acts, processes, operations, steps or func tions disclosed in the specification or claims may not be construed as to be within the specific order, unless explicitly or implicitly stated otherwise, for instance for technical reasons. Therefore, the disclosure of multiple acts or functions will not limit these to a particular order unless such acts or functions are not interchangeable for technical reasons. Furthermore, in some examples a single act, function, process, operation or step may include or may be broken into multiple sub-acts, -functions, -processes, -operations or -steps, respectively. Such sub acts may be included and be part of the disclosure of this single act unless explicitly excluded. Furthermore, the following claims are hereby incorporated into the detailed description, where each claim may stand on its own as a separate example. While each claim may stand on its own as a separate example, it is to be noted that - although a dependent claim may refer in the claims to a specific combination with one or more other claims - other examples may also include a combination of the dependent claim with the subject matter of each other de- pendent or independent claim. Such combinations are explicitly proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim.

Claims

Claims What is claimed is:

1. A computer-implemented method (100) for predicting return of a user to a vehicle, comprising: determining (102) a first probability for return of the user to the vehicle based on a first trained model and at least data on a current location of the user and a current location of the vehicle, wherein the first trained model is trained based on historic driving data of the user; determining (104) a second probability for return of the user to the vehicle based on a second trained model and contextual data from a mobile communication device of the user that indi cates a current activity of the user, wherein the second trained model is trained based on his- toric contextual data of the user; combining (106) the first probability and the second probability to a combined probability; determining (108) that the user returns to the vehicle if the combined probability exceeds a threshold value; controlling (110) at least one sensor of the mobile communication device and/or the vehicle to collect sensor data capable of indicating a movement of the user if it is determined that the user returns to the vehicle; and sending (112) a notification to the mobile communication device if the sensor data indicates that the user is approximately approaching the vehicle, wherein the notification comprises a request to the user whether the user desires to have personalization of the vehicle.

2. The method of claim 1, wherein combining (106) the first probability and the second probability to the combined probability comprises combining the first probability and the sec ond probability using a weighting factor for at least one of the first probability and the second probability.

3. The method of claim 2, wherein the weighting factor is based on a feedback of the user and/or based on a comparison of historic telemetry data of the vehicle and historic pre dictions of the first trained model.

4. The method of any of claims 1 to 3, wherein the first trained model further outputs a travel destination of the user associated to the return of the user to the vehicle.

5. The method of any of claims 1 to 4, wherein the first trained model further outputs a first time for the return of the user to the vehicle and the second trained model further outputs a second time for the return of the user to the vehicle, and wherein controlling (110) the at least one sensor of the mobile communication device and/or the vehicle comprises controlling the at least one sensor to start collecting the sensor data a predefined time instant prior to one of the first time and second time.

6. The method of any of claims 1 to 5, wherein the contextual data indicating the current activity of the user is contextual data collected by the mobile communication device since the user most recently left the vehicle.

7. The method of any of claim 1 to 6, wherein the contextual data indicating the current activity of the user comprises sensor data from at least one sensor of the mobile communica tion device and/or a type of the user’s current location.

8. The method of any of claim 1 to 7, wherein determining the second probability is further based on data on a current location of the vehicle, data on a current date and data on a current time.

9. The method of any of claim 1 to 8, wherein, if it is determined that the user returns to the vehicle and if the vehicle is in a sleep-mode, the method further comprises controlling the vehicle to wake up.

10. The method of any of claim 1 to 9, further comprising: controlling the vehicle to adjust at least one setting of the vehicle according to a stored or learned setting of the user, if a notification indicating that personalization of the vehicle is desired by the user is received from the mobile communication device.

11. The method of any of claims 1 to 10, wherein the second trained model comprises at least one activity pattern of the user that indicates return of the user to the vehicle, wherein the activity pattern is learned from the historic contextual data of the user.

12. The method of claim 11, wherein determining (104) the second probability comprises comparing the current activity of the user as indicated by the contextual data from the mobile communication device of the user to the at least one activity pattern of the user.

13. The method of any of claims 1 to 12, wherein controlling (110) the at least one sensor of the mobile communication device and/or the vehicle comprises controlling at least one sensor of the vehicle to collect sensor data capable of indicating movement of the user relative to the vehicle.

14. The method of any of claims 1 to 13, wherein determining (102) the first probability is further based on data on a current date and data on a current time.

15. The method of any of claims 1 to 14, wherein determining (104) the second probability is further based on contextual data from the vehicle indicating the current activity of the user.

16. A non-transitory machine readable medium having stored thereon a program having a program code for performing the method of any of claims 1 to 15, when the program is exe cuted on a processor.

17. A data processing system for predicting return of a user to a vehicle, comprising one or more processors configured to: determine a first probability for return of the user to the vehicle based on a first trained model and at least data on a current location of the user and a current location of the vehicle, wherein the first trained model is trained based on historic driving data of the user; determine a second probability for return of the user to the vehicle based on a second trained model and contextual data from a mobile communication device of the user that indicates a current activity of the user, wherein the second trained model is trained based on historic contextual data of the user; combine the first probability and the second probability to a combined probability; determine that the user returns to the vehicle if the combined probability exceeds a threshold value; control at least one sensor of the mobile communication device and/or the vehicle to collect sensor data capable of indicating a movement of the user if it is determined that the user returns to the vehicle; and send a notification to the mobile communication device if the sensor data indicates that the user is approximately approaching the vehicle, wherein the notification comprises a request to the user whether the user desires to have personalization of the vehicle.