DE102012212612A1 - Method for determining residence of user related to vehicle, involves determining state of user based on recognized movement of user and detection of stepping out process of user according to predetermined condition - Google Patents

Abstract

The method involves detecting the stepping out process of the user into the vehicle. The movement of the user is recognized. The state of the user is determined form a group of states that is configured to indicate the location of the user within or outside the vehicle. The state of the user is determined based on the recognized movement of the user and the detection of stepping out process of the user according to predetermined condition. The time duration of the recognized movement of the user is allocated. Independent claims are included for the following: (1) a device for determining residence of user related to vehicle; and (2) a computer program for determining residence of user related to vehicle.

Description

The invention relates to a method for determining a location of a user of a mobile terminal with respect to a vehicle and a device set up for this purpose. The location of the user may be inside or outside a vehicle, for example.

It is known that persons involved in a vehicle accident or affected by a vehicle breakdown by their own actions after the accident or the breakdown bring themselves and others into further dangerous situations or even cause further accidents. The reason for this are states of shock of the persons affected and thus thoughtless actions as well as ignorance about the necessary behavior.

Also helpers in accident or breakdown situations of others bring themselves and others occasionally due to shock states or ignorance about necessary actions in danger.

A typical hazard situation arises when the victim or helper leaves a vehicle that is still protective and no longer perceives the surrounding traffic as a danger due to a shock. By leaving the vehicle they are from relatively protected road users to vulnerable road users. It is therefore desirable to support victims and helpers in an accident or breakdown situation with warnings or hints. For this it is necessary, inter alia, to know how a road user behaves, for example, whether he leaves the vehicle.

Today, mobile devices such as smartphones are widespread and are carried by many users in the road, for example in the pocket or pocket, purse or hand. These terminals often have an electronic processing unit with memory, an optionally touch-sensitive display, transmitters and receivers for wireless and wired communication and sensors for measuring the position or movement of the smartphone, such as Global Positioning System (GPS) sensors, yaw rate and / or acceleration sensors. Smartphones can be set up via programs, often also called apps, to behave in accordance with the program sequence, ie to carry out, for example, methods described in the program.

From the DE 10 2010 003 429 A1 For example, a method is known in which a mobile terminal sends a signal to a second mobile terminal when it comes close to the second terminal. On the basis of the received signal, the driver of the vehicle can be informed that the second mobile terminal is nearby and z. B. is carried by a pedestrian.

The object of the invention is to provide a determination as to whether a road user is inside or outside a vehicle.

A method according to the invention for determining a location of a user, in particular of a user of the mobile terminal carrying a mobile terminal, with respect to a vehicle, comprises: detecting an entry / exit operation of the user in the vehicle from the vehicle; Recognizing the movement type of the user; Determining a state of the user from a group of states, the group including at least one state indicating that the user is in a vehicle and a state indicating that the user is outside of a vehicle, wherein the determination the state is based on the detected movement type and / or recognition of an entry / exit operation; wherein the determination of the state depends on the previously determined state, if a state has already been determined previously.

The method according to the invention at least determines whether the user, for example the driver, is inside or outside the vehicle. For this, the detected movement type and / or the recognized entry / exit process are taken into account. The method thus enables an output of a hint or a warning to the user, adapted to the respective situation of the user. If, for example, the user is about to leave the vehicle in a breakdown situation on a motorway, it is recognized that the user is outside the vehicle and a warning of dangerous traffic can be output on the mobile terminal.

When determining the state, the previously determined state is also taken into account. This makes it possible to increase the robustness of the determination of the state, because conditions for changing the state from one state to another can provide the actual occurrence of entry / exit while remaining in one state Actual occurrence corresponding conditions can be defined. In other words, other conditions may be provided for changing the state than for maintaining the state. A transition from one state to the other can be made to permanently affirmative recognitions, while maintaining state is provided for non-affirmative detections may be (for example in the case of unclear detection / error detection / noise). Occasionally, the transition from one state to another may be tied to the occurrence of a combination of conditions, such as detected movement type and vehicle speed. Furthermore, several alternative conditions may be provided, so that the occurrence of a condition already leads to a transition. In this way, various events can be taken into account and actual changes in the location of the user can be detected.

The optionally entrained terminal performs at least a part of the movement of the user. In this way, movement data of the mobile user carrying the mobile terminal can be collected by means of motion sensors of the mobile terminal. The movement data can be used to detect an entry / exit process and the movement types of the user. Sensor data of the vehicle can also be taken into account in the recognition, for example sensor data of seat mats in the seats of a vehicle. The detections can be carried out either in the mobile terminal or in the vehicle, whereby the vehicle also has the necessary electronic computing capacity. Since the mobile terminal and the vehicle can communicate (for example via wireless connections, Bluetooth), the necessary sensor data can be exchanged.

The recognition of the entry / exit operation need not distinguish between a boarding operation and a boarding procedure, but may refer both to the transition to a vehicle and to the transition out of a vehicle. The movement types of the user can be, for example, walking, running, cycling, driving with a motor vehicle or standstill. Walking and races can be grouped together. A vehicle may be, for example, a passenger car or a truck.

The group of states may additionally include a state indicating that the user is inactive and / or a state indicating that the user's state is unknown. The additional states may be states that must be assumed when leaving the state that the user is in a vehicle or that the user is outside of a vehicle. Furthermore, the states that the user is in a vehicle or that the user is outside of a vehicle can only be reached via one of the additional states. The additional states can therefore be transitional states and define a two-step process. It has been found that the determination of the user's state is best performed by this two-step process.

In a development, the method comprises assigning a time duration to the detected movement type; wherein determining a condition comprises determining whether the assigned time duration is greater than a threshold value. Assigning a time duration may include determining the amount of time since the detected movement type has been uninterruptedly detected. In this way the robustness of the determination of the condition is increased. Not every small or incorrect recognition of a movement type leads to the determination of another condition. Only if the occurrence is confirmed does the detected movement type be taken into account.

In a further development, determining a state comprises determining whether the speed of the vehicle is above or below a first threshold and / or determining a state determining whether the frequency of a change in the movement is above or below a second threshold. By considering the speed of the vehicle, the conditions for a state change or state maintenance can be further refined and the determination of the state made more robust. At the same time, the occurrence of events that actually occur does not go undetected.

In one variant, the recognition of a user's entry / exit process into a vehicle is based on data from motion sensors of the mobile terminal and / or data from sensors of the vehicle. Motion sensors of the mobile terminal may be acceleration and yaw rate sensors. The sensors of the vehicle may be seat mats, door opening sensors, steering wheel occupancy sensors or belt sensors. Depending on whether the mobile terminal or the vehicle is set up to recognize the entry / exit process, the respective other device transmits the data to the recognizing device, for example via a wireless connection.

In a preferred development of the method, the determination of a state is carried out in response to the detection of an entry / exit operation of the user. The recognition of the entry / exit process thus acts as a trigger for the determination of the state. In this way, the computing capacity is only used if a change of state could occur. Just when the determination of the state is carried out by the mobile terminal, this allows the saving of Energy resources and prolongs the time that the mobile device can live with the available energy and without reloading.

In an advantageous development, the method also includes determining the endangerment of the user based on the specific state of the user. Determining the hazard may be based on determining that the vehicle is on a lane that is normally driven at higher speeds (a highway) and the user is outside the vehicle. Further, the method may include issuing an indication to the user depending on the identified hazard. As soon as the hazard has been determined, the user can be informed accordingly, eg. B. "attention highway, behind the guardrail step", are issued.

In a further development, the method comprises determining an intention of the user to leave the vehicle, in particular due to an opening of the door, the safety belt. By detecting the user's exemplified activities by means of corresponding sensors of the vehicle, an intention of the user to leave the vehicle can be deduced. Accordingly, warnings or notes can be issued earlier.

BRIEF DESCRIPTION OF THE DRAWING

1 shows a schematic representation of a device according to an embodiment of the invention, a mobile terminal and a vehicle.

2 Figure 12 is a schematic state diagram as may be used in determining the state of the user according to the invention.

DETAILED DESCRIPTION OF THE EMBODIMENT

1 shows a schematic representation of a device according to an embodiment of the invention, a mobile terminal and a vehicle. Dashed line elements (vehicle, buffers, connections) are only optional elements. 1 shows a navigation system, which is included in the mobile terminal. Alternatively, the navigation system may be comprised of vehicle. The term "means" also includes general electronic processing means, which are set up by a program for the execution of the purpose, if necessary even only temporarily set up for it. The means and buffers shown are functional elements that may be included in both the mobile terminal and in the vehicle. The arrows indicate a data exchange, for example within the mobile terminal or within the vehicle or between the mobile terminal and the vehicle, depending on which device comprises the corresponding means. The data exchange takes place for example via a wireless connection such as Bluetooth or WiFi.

The mobile terminal comprises a motion sensor system, namely acceleration and rotation rate sensors. If appropriate, the mobile terminal comprises telecommunication means, for example for communication via mobile radio, WiFi or Bluetooth. Furthermore, in this exemplary embodiment, the mobile terminal comprises a navigation system which has a digital map and means for determining the position of the mobile terminal. The motion sensor system supplies data for movement of the mobile terminal, which carries out part of the movement of the accompanying user, to the means for detecting an entry / exit operation. An Entry / Exit Process The movement sensor system also supplies data for moving the mobile terminal to means for detecting the movement type of the accompanying user.

The means for detecting an entry / exit process and the means for detecting the movement type are adapted to recognize the entry / exit process or the movement type on the basis of statistical characteristics of the transaction data. For this purpose, movement data are considered over a time range, for example the last 3 s, 5 s or 10 s or a range from 3 s before the recognition time to 10 s before the detection time. The detection means are adapted to use the movement data by means of statistical positioning measures, e.g. Arithmetic mean, variance, standard deviation, correlation; statistical spreads: variance, standard deviation, interquartile range; Independent of the orientation of the mobile terminal relative to the user, independent of location, d. H. with regard to the carrying position of the mobile terminal: Signal Magnitude Area (arithmetic mean of the absolute values of all 3 axes), Total Acceleration and characteristics from the frequency domain in a manner known per se. It has also been found that decision trees such as "C4.5", linear models such as "Support Vector Machine" and instance-based classifiers such as the "k-Kearest Neighbor", "OneR" and statistical classifiers such as "NaiveBayes" are generally suitable ,

Optionally, the motion sensor data may be stored in a buffer that holds data received in a certain past period of time. In particular, the speed of the mobile terminal can be stored in the buffer. The detected movement type can also be stored in a buffer holding data.

The detection of an entry / exit process serves as a trigger, d. H. as a trigger, for means for determining a condition of the user to start the determination. Optionally, sensor data for outdoor or indoor monitoring can also serve as triggers. For example, a seat occupancy detection may provide the signal that a seat is leaving, or an outside camera and corresponding image processing may provide the information that people are leaving the vehicle.

Speed, acceleration of the vehicle, ignition active, communication signals, eg. B. Femtozelle, Bluetooth, WLAN, NFC, door / window status, seat occupancy detection, steering wheel occupancy detection, hands-on detection, gear selection, handbrake, accelerator / clutch pedal, current user input z. As or air conditioning, switch, steering wheel, voice input, interior camera / exterior camera, ultrasound, thermal imaging camera, laser scanner, position of the vehicle, classification, where the vehicle / driver is currently located, eg. At home, shopping, rain sensor, light sensor.

The means for determining a state of the user use the detection of the entry / exit process, the detected movement type and data of the motion sensor (possibly also for the past on the buffer) to determine the state. Optionally, data from the vehicle can also be used, such as NFC communication, door status, seat occupancy recognition, keyless entry, keyless access, interior camera / exterior camera, steering wheel occupancy sensor, handson detection, control (buttons, rotary pushbutton), belt contact, Microphones, speed, acceleration of the vehicle, ignition active, communication signals, eg. B. Femtozelle, Bluetooth, WLAN, NFC, door / window status, gear selection, handbrake, accelerator / clutch pedal, current user input z. As or air conditioning, switch, voice input, ultrasound, thermal imaging camera, laser scanner, position of the vehicle, classification, where the vehicle / driver is currently located, eg. At home, shopping, rain sensor, light sensor.

The means for determining a condition are arranged to proceed according to a state diagram as in 2 shown.

2 shows in a state diagram the states "vulnerable road users", "inactive", "unknown" and "vehicle". The "vulnerable road user" state represents the condition that the user is outside the vehicle. The state "vehicle" represents the condition that the user is in the vehicle. The arrows indicate the possible transitions and the conditions for it.

Starting, for example, from the "vulnerable road user" state, it is possible to move to the "unknown" state if the relative frequency of the change of the movement type, in other words of the movement pattern, is greater than the threshold value reIH_AeBM_1. You can use the movement types stored in the buffer to calculate the frequency. Alternatively, it is possible to proceed if an entry / exit movement was detected or if the movement type Driving was detected longer than the threshold t_fahr_3 and the speed is greater than v_fahr_3, for which purpose the values stored in the buffers can also be used.

From the "vulnerable road user" state, the system changes to the "inactive" state if the detected movement type is standstill. If none of the conditions for transition to another state applies, the vulnerable road user state remains.

The transitions from the "inactive" state to other states depend on the following conditions: The movement type Go / Race or the movement type Cycling has been detected longer than the threshold value t_FG-RD_1. The movement type Drive was detected longer than the threshold value t_fahr_1 and the speed is greater than the threshold value v_fahr_1. The movement type is different from inactive. For all other conditions, it remains in the Inactive state.

The status "Unknown" can be changed using the following conditions: The movement type Go / Race or the movement type Cycling was detected longer than the threshold value t_FG-RF_2. The movement type standstill was detected. The movement type Drive has been detected longer than the threshold value t_fahr_2 and the speed is greater than the threshold value v_fahr_2. If none of the mentioned conditions is fulfilled, the state "unknown" remains.

The transition from the "Vehicle" status can be made using the following conditions: The relative frequency of the movement type change is greater than the threshold value reIH_AeBM_1 or, alternatively, the recognition of an entry / exit process. The detected movement pattern is "standstill". If none of the mentioned conditions is fulfilled, the condition "vehicle" remains.

The means for determining a state are arranged to provide a state as output only when the state " Vulnerable road user "or" vehicle "was taken. Then, the condition that the driver is out of the vehicle or in the vehicle is output.

As in 1 Optionally, means may be provided for determining the endangerment of the user who receives from the navigation system the position and type of the roadway, for example, highway, country road, city street. If it is determined that the vehicle has come to a halt on a lane at which high speeds are driven, for example, on a highway or country road, a higher danger is detected than when the vehicle has come to a halt on a city street. Also, the means may be arranged to take into account the density of the surrounding traffic due to Car2X communication and corresponding data, as well as whether the vehicle has crashed or is broken down and the severity of the breakdown, using on-board sensors. The means for determining the endangerment of the user may also include data of the vehicle as in 1 shown received. For example, a radar can determine what speeds surrounding vehicles are traveling, which can be used to derive hazard potential to the user.

Further optionally, means may be provided for issuing an indication, including, for example, a display that alert the user to dangers and provide instructions for action or provide coaching. For example, spending can be "Attention Highway"; "Wear a safety vest"; "Electronic safety vest"; "Set up warning triangle at 200 m"; "Step Behind the Guardrail"; "Activate first responders app on smartphone"; "Left is the entry point for climbing spaces"; "Do not Forgot Umbrella"; "Baby Monitor or Mothers App Active"; "Welcome Home", "," Stop the engine when people stay in the vehicle ".

• – das Fahrzeug auf einer Fahrbahn zum Stehen gekommen ist, auf welcher typischerweise höhere Geschwindigkeiten gefahren werden, z. B. weil es in einen Unfall verwickelt ist, der Fahrer bzw. Benutzer gesundheitlich beeinträchtigt ist oder eine Panne hat oder Hilfe für andere Verkehrsteilnehmer erforderlich ist und ggf. weiterhin
• – Fahrzeuginsassen das Fahrzeug verlassen haben oder z. B. durch Lösen des Sicherheitsgurtes, Öffnen einer Fahrzeugtür die Absicht des Verlassens des Fahrzeugs zu erkennen geben.

Moreover, the invention is also based on the following considerations:
In order to increase the safety of vulnerable road users, the method may further include: detecting, with information from the vehicle interior and vehicle environment, that

• - The vehicle has come to a halt on a road on which typically higher speeds are driven, for. B. because it is involved in an accident, the driver or user is impaired health or has a breakdown or help for other road users is required and possibly continue
• - Vehicle occupants have left the vehicle or z. B. by loosening the seat belt, opening a vehicle door to disclose the intention of leaving the vehicle.

• – den Fahrzeuginsassen bzw. Benutzer das Verlassen des Fahrzeugs nicht empfohlen wird, z. B. durch einen Warnhinweis
• – die Fahrzeuginsassen bzw. Benutzer bei Verlassen des Fahrzeugs auf Beachtung des umliegenden Verkehrs hingewiesen werden sowie das Tragen der Warnweste, sonstiger Gegenstände oder Systeme, welche zu ihrer besseren Wahrnehmbarkeit beitragen,
• – oder bei Verlassen des Fahrzeugs dringend auf den umliegenden Verkehr zu achten ist, z. B. durch vorheriges Verriegeln der Türen des Fahrzeugs.

The support of the vehicle occupants is, for example, that:

• - The vehicle occupants or users leaving the vehicle is not recommended, eg. B. by a warning
• - the occupants of the vehicle or the user when leaving the vehicle are made aware of the surrounding traffic, as well as the wearing of the safety vest, other objects or systems which contribute to their better perceptibility,
• - or when leaving the vehicle is urgently to pay attention to the surrounding traffic, z. B. by previously locking the doors of the vehicle.

• – wenn die Insassen bzw. Benutzer des verunfallten Fahrzeugs oder Pannenfahrzeugs das Fahrzeug verlassen, werden umgehend entsprechende Warnhinweise an die umliegenden Fahrzeuge verteilt, beispielsweise durch
• – Erweiterung des TMC plus
• – Meldung an Backendserver, welcher diese Information beispielsweise durch Kommunikation an elektronische Systeme andere Fahrzeuge oder mobile Endgeräte von Fahrzeuginsassen, z. B. auch Beifahrer, zur Verfügung stellt
• – Broadcast einer Car2X-Nachricht

The support of vehicles of the surrounding traffic consists, for example, in that:

• - When the occupants or users of the accident vehicle or breakdown vehicle leave the vehicle, immediately appropriate warnings are distributed to the surrounding vehicles, for example by
• - Extension of the TMC plus
• Message to the backend server, which this information, for example, by communication to electronic systems other vehicles or mobile devices of vehicle occupants, eg. As well as passenger, provides
• - Broadcast a Car2X message

• – Art der gefahrenen Straße, z. B. Autobahn
• – Besonderheiten der Straße wie Breite und Anzahl der Fahrstreifen, Krümmungsverlauf, Einsehbarkeit
• – aktuelle Mindest-, Höchstgeschwindigkeiten, oder typischerweise gefahrene Geschwindigkeiten
• – Unfallschwerpunkte
• – Ort der gefahrenen Straße
• – aktueller Fahrzeugzustand, z. B. Reifenpanne, Unfall, Beleuchtungsstatus
• – Dynamikzustand des Fahrzeugs, z. B. Stillstand
• – lokale Wetter-, Sicht- und Lichtverhältnisse (aus Sensorik oder Internet)
• – Uhrzeit, Tag/Nacht- Belegung der Sitze im Innenraum, z. B. Anzahl der Fahrzeuginsassen
• – Status der Fahrzeugtüren, z. B. geöffnet. Diese Informationen werden einzeln oder kombiniert dazu genutzt, Informationen über die Situation zu generieren, wie z. B. das Vorliegen einer Gefährdungssituation, deren Grad und Zuverlässigkeit, und diese Information zu kommunizieren.

By the measures mentioned, it is possible for the occupants of an accident vehicle or breakdown vehicle and the occupants of another vehicle, which wants to come to the aid, to protect against consequential accidents. In this case, the fact is exploited that the vehicle via a series of data and information about the vehicle, eg. B. operating conditions, the situation in and around the vehicle has such. B .:

• - Type of road driven, z. B. highway
• - Special features of the road such as width and number of lanes, curvature, visibility
• - current minimum, maximum speeds, or typically driven speeds
• - Accident areas
• - Place of the driven road
• - current vehicle condition, eg. B. flat tire, accident, lighting status
• - Dynamic state of the vehicle, z. B. Standstill
• - local weather, visibility and lighting conditions (from sensors or the internet)
• - Time, day / night occupancy of the seats in the interior, z. B. Number of vehicle occupants
• - Status of the vehicle doors, z. B. opened. This information is used individually or in combination to generate information about the situation, such as: B. the existence of a hazardous situation, their degree and reliability, and to communicate this information.

On other systems, eg. B. in other vehicles, other road users or central processing units, eg. As a back-end server, incoming information such or similar information sources, eg. B. in the same accident entangled vehicles are optionally used in addition, z. B. to increase the information content about the situation, degree of risk and reliability. The linking of such information from multiple sources on a central, usually higher processing level allows a plausibility of this information.

These measures make this information accessible to the protection of passengers and persons outside the vehicle. This creates the basis for the driver, other inmates and helpers who are in the area of a lane, z. B. get out due to an accident or breakdown or first aid, consistently, d. H. outside the vehicle.

First responders also receive action information in the vehicle or on the smartphone. The aim is to prevent a subsequent accident or to support the accident consequences of the primary accident by supporting the driver / inmate / first responder.

Thus, for example, it is recognized when passengers leave the vehicle, which is suggested to point out a targeted warning to the surrounding traffic, the local situation and measures to improve their own protection.

In addition, the advantage is that the information to other systems, eg. B. other vehicles, infrastructure, other road users or institutions are communicated and there targeted an action is initiated, such as information, warning or control of an actuators, such. For speed reduction.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102010003429 A1 [0006]

Claims (15)

A method for determining a location of a user relative to a vehicle, wherein the user is in particular a user of a mobile terminal carried by the user, comprising: Detecting a user entering / leaving the vehicle in the vehicle; Recognizing the movement type of the user; Determining a state of the user from a group of states; wherein the group comprises at least one state indicating that the user is in a vehicle and a state indicating that the user is outside a vehicle; wherein the determination of the condition is based on the detected movement type and / or on the detection of an entry / exit operation; wherein the determination of the condition depends on the predetermined condition. The method of claim 1, further comprising: Assigning a time duration to the detected movement type; Wherein determining a condition comprises determining whether the assigned time duration is greater than a threshold value. The method of claim 2, wherein assigning a time duration comprises determining the length of time since the detected movement type has been uninterruptedly detected. The method of claim 1, wherein determining a state comprises determining whether the speed of the vehicle is above or below a first threshold and / or wherein determining a state comprises determining whether the frequency of a change of movement type is above or below a second threshold. The method of claim 1, further comprising: Assigning a time to the detected movement type since the movement type was continuously recognized; wherein the movement type of the user is assigned to driving the user in a vehicle, wherein determining the condition that the user is in a vehicle comprises determining whether the assigned time duration is greater than a first threshold value and the speed of the vehicle is greater than a second threshold value. Method according to one of the preceding claims, wherein the detection of an entry / exit operation of the user in a vehicle based on data from sensors of the mobile terminal and / or data from sensors of the vehicle. The method of any one of the preceding claims, wherein determining a state is performed in response to detecting a user entering / exiting. Method according to one of the preceding claims, further comprising: Determining the endangerment of the user also based on the specific state of the user. The method of claim 8, further comprising: Issuing a note to the user depending on the identified hazard. The method of any one of the preceding claims, further comprising: Determining an intention of the user to leave the vehicle, in particular due to opening the door or a safety belt. The method of claim 10 further comprising: detecting the situation in which the driver and the vehicle are located, in particular whether a vehicle is involved in an accident or comes to a stop as a first responder due to an accident. The method of claim 11, wherein detected for detecting the situation vehicle sensors, in particular crash sensors, rotation / acceleration sensors, communication, in particular a communicated accident scene, or driver inputs, especially in response to a prompt, such as accident - do you want to be first responders? ", Are used , Method according to one of claims 11 to 12, wherein the exiting person receives by the output of a computer program recommendations for action based on the particular situation, in particular recommendations for action to the first responder or persons of the accident vehicle. Apparatus comprising electronic processing means, the apparatus being adapted to carry out a method according to any one of claims 1 to 13. A computer program which, when executed by a computer, causes the computer to perform a method according to any one of claims 1 to 13.

Images