DE102019105490A1 - Method, device and computer program for operating a vehicle using a mobile user device - Google Patents

Abstract

A method (200) for operating a vehicle (110) that is in particular at least partially automated driving by means of a mobile user device (120) is described. The method (200) comprises detecting (201) that a mobile user device (120) of an occupant of the vehicle (110) is located in an interior of the vehicle (110). In addition, the method (200) includes determining (202) that at least one operating action is required for the condition of the vehicle (110). The method (200) further includes causing (203) an operating element to be provided for executing the at least one operating action on the mobile user device (120).

Description

The invention relates to a method, a device and a computer program for operating a vehicle, in particular a vehicle that can be driven at least partially in an automated manner, by means of a mobile user device. In particular, the invention relates to the provision of operating options for a vehicle, which are at least partially carried out using means of a mobile user device. The invention also relates to a correspondingly configured mobile user device.

A mobile user device, e.g. a smartphone, tablet PC or the like provides its user with a partly a strong distracting factor. A quick transfer of attention from the use or operation of the mobile user device to another activity, so to speak a transition from a virtual reality to a physical reality, is problematic or with a certain time delay or a possible irritation connected. Such effects are often underestimated or (mostly unconsciously) accepted by the users of mobile user devices. If a mobile user device in a vehicle is operated by a user of the vehicle, in particular by the driver of the vehicle, this can lead to a problematic situation.

The present document deals with the technical task of increasing the comfort and safety for a user of a vehicle, in particular of an (at least partially) automated driving vehicle, in relation to the interaction with the vehicle. Furthermore, it is an object of the invention to enable or improve the usability or use of a mobile user device in a vehicle. The usability or use of the mobile user device can be based on various purposes, e.g. Reading, communicating, playing games, editing documents, etc.

The problem is solved by each of the independent claims. Advantageous embodiments are i.a. described in the dependent claims. It is pointed out that additional features of a patent claim dependent on an independent patent claim without the features of the independent patent claim or only in combination with a subset of the features of the independent patent claim can form a separate invention independent of the combination of all features of the independent patent claim, which can be made the subject of an independent claim, a divisional application or a subsequent application. This applies equally to the technical teachings described in the description, which can form an invention that is independent of the features of the independent patent claims.

According to one aspect, a method for operating a vehicle (in particular a motor vehicle) is described. The method, in particular part of the method, can be carried out at least partially by a control unit and / or by a device of the vehicle. Alternatively or in addition, the method, in particular part of the method, can be carried out at least partially on a mobile user device. Exemplary user devices are a smartphone, a tablet PC, a smart watch, smart clothes, a laptop computer, etc. It can be a mobile user device of an occupant, in particular a driver, of the vehicle. In particular, the method is carried out as a function of the driving mode of the vehicle.

The method includes detecting that a mobile user device of an occupant of the vehicle is located in an interior of the vehicle. This can e.g. are determined on the basis of position data from a position sensor of the vehicle and a position sensor of the user device. For example, it can be recognized on the basis of the position data that the user device and the vehicle are in a matching position and / or are moving in a matching manner. Alternatively or in addition, it can be recognized that the user device is connected directly to the vehicle via a local communication connection (e.g. Bluetooth or WLAN). Alternatively or additionally, on the basis of the sensor data of the vehicle, by means of an interior sensor, e.g. an interior camera, it can be recognized that a user device is located in the interior of the vehicle and / or that a user device is being operated in the interior of the vehicle.

Furthermore, the method includes determining that at least one operating action is required to condition the vehicle. For this purpose, environment data can be evaluated by one or more environment sensors of the vehicle. For example, it can be recognized that there is a driving or traffic situation for which an operator action to operate the vehicle is required.

Furthermore, the method includes causing an operating element to be provided for performing the at least one operating action on the mobile user device. In particular, a control instruction can be sent from the vehicle to the user device for this purpose (via a wireless communication link). The user device can then in response to the control instruction via a user interface of the Providing the user device with an operating element (for example a button or a slider on a touch-sensitive screen) with which the required operating action can be initiated by a user of the user device. The provision of the operating element can include the appearance and / or activation of an operating element on the mobile user device. In particular, the operating element can be provided in such a way that the meaning of an operating element of the mobile user device that is already present, for example, is changed (only or mainly only for brief operation of the vehicle) so that the vehicle can be operated with this operating element.

For example, the provision can also take place in two steps and / or the operating element permit different types of operation or differently limited operating actions. In particular, an operating element can be provided (only or predominantly) for performing the specific (required or maximum required) operating action for operating the vehicle. In this case, depending on the specific (required or maximum required) operating action, an operating element (which is suitable for the operating action, equipped for a correspondingly defined or limited possibility of intervention) can be provided.

The operation of a vehicle relates in particular to the control of at least one functionality of the vehicle or of the vehicle as a whole and / or to an interaction with the vehicle. The interaction here is a mutual, in particular rapid successive, influence of the user on the vehicle and the vehicle on the user or vice versa. Accordingly, the operating element on the mobile user device can be an operating element for interacting with the vehicle.

The operation of the vehicle (or the interaction with the vehicle) can in particular include an operation of the vehicle while driving, in particular a control of the longitudinal dynamics, the transverse dynamics, and / or the execution of the maneuver. The operation of the vehicle can also be (only) participation in controlling the vehicle or in at least one function of the vehicle. The operation of the vehicle can also be an explicit, implicit or (exceptionally) tacit permission (of the user, the operator) that a certain action, e.g. an intervention in the chassis of the vehicle, a certain driving maneuver, etc. is automated and in particular carried out in a certain way.

In general, the vehicle can also be a vehicle that can be driven or driven predominantly manually, at least during the execution of the method.

If the vehicle is operated by the user, in particular the driver, at least temporarily, in particular for a short time, e.g. should be required due to a rapidly changing situation, a required operation, in particular a cooperation (permission or refusal to carry out an action, a correction of an at least partially executable action and the like) of the user, in particular the driver of the vehicle, can be faster and / or safer (with reduced risks). A disruption or a resulting stress level for the user can also be reduced (due to unfavorable experiences or an otherwise intensive operating request from the vehicle). Furthermore, a legal situation in connection with the use of a user device in the vehicle can be at least partially clarified or improved. Furthermore, there is an advantage with regard to better use of time in an at least partially automated vehicle.

For example, it is made possible for an occupant, in particular a driver, of a vehicle to deal with a mobile user device in a safe and comfortable manner while driving, and nevertheless to initiate an operator action to operate the vehicle relatively quickly. The risk of possible instinctive actions or actions by the user caused by shock or surprise can be reduced. In particular, an action stimulated by a virtual reality (for example by an interactive application running on the mobile user device, e.g. by a game being played) cannot (almost inadvertently and / or instinctively) be transferred to an operation of the vehicle.

As a result of the fact that an operating element is provided for executing the at least one operating action on the mobile user device, in particular when the operating element on the mobile user device is actually operated by the user, a (otherwise typically occurring) output of additional driver information, in particular warning information, changed, e.g. be reduced or prevented. For example, a (typically) loud or intense request to return to a driving task, e.g. a so-called takeover request, a “hands-on request”, etc., do not take place, or take place at a later point in time, with a changed intensity, etc.

The method can include determining a degree of attentiveness of a user (in particular the occupant) of the mobile user device in relation to the mobile user device. In particular, it can be determined how deeply the user is immersed in the use of the user device. For this purpose, a degree of activity of the user device can be determined. Furthermore, a sensor (for example a camera) of the user device can be set up to record sensor data relating to the user. Alternatively or additionally, a degree of attentiveness of a user of the mobile user device with regard to a driving task of the vehicle can be determined. For this purpose, sensor data from an interior camera of the vehicle can be evaluated.

It can then be arranged that the mobile user device is operated as a function of the determined degree of attentiveness with regard to the mobile user device and / or as a function of the determined degree of attention with regard to the driving task. In particular, the way in which the user is informed of the required operation of the vehicle can be adapted as a function of the level of attention. In this way, the comfort and safety of the operation of a vehicle can be further increased.

The method can include determining that an operating action has been initiated by the mobile user device, in particular by actuating the operating element provided on the user device. In this case, the operating action can be geared towards executing an automated function and / or a control-related task with a means of the vehicle and / or executing a driving task and / or a driving maneuver of the vehicle. For example, a control instruction can be sent from the user device to the vehicle via the wireless communication link and received by the vehicle. The vehicle can then be operated as a function of the operator action initiated on the mobile user device. In particular, the required operation of the vehicle can take place in accordance with the recorded operator action. This enables a comfortable and safe operation of a vehicle.

The method can include determining whether an actuation of an operating element of the vehicle confirming the operating action has taken place and / or whether no actuation of an operating element of the vehicle has taken place that contradicts the operating action. In other words, it can be checked whether an operating action initiated on the user device was confirmed by actuating an operating element in the vehicle or whether it was canceled again.

The operation of the vehicle as a function of the operator action initiated on the mobile user device can possibly be prevented if it has been determined that an operator control element of the vehicle has not been actuated confirming the operator control action and / or that an operator control element of the vehicle has been operated in contradiction to the operator control action is done. Alternatively or additionally, the vehicle can only be operated depending on the operating action initiated on the mobile user device if it has been determined that an operating element of the vehicle has been actuated confirming the operating action and / or that no actuation contradicting the operating action has taken place a control element of the vehicle has taken place. In this way, the safety of the operation of a vehicle can be further increased.

In particular, within the scope of the method it can be determined whether within a predetermined time interval (e.g. 5 seconds or less, 3 seconds or less, 1 second or less, etc.) before and / or after the initiation of the operator control action and / or none contradicting actuation of a control element of the vehicle has taken place. A temporal correlation between the operating action on the user device and the actuation of an operating element in the vehicle can thus be checked.

The operation of the vehicle as a function of the operator action initiated on the mobile user device can possibly be prevented if it is determined that within the predetermined time interval before and / or after the initiation of the operator action, no confirming actuation and / or contradicting actuation of an operating element of the Vehicle is done. As an alternative or in addition, the vehicle can only be operated depending on the operator action initiated on the mobile user device if it is determined that within the predetermined time interval before and / or after initiating the operator action, an affirmative actuation and / or no contradicting one can be used Operation of a control element of the vehicle has taken place. In this way, the safety of the operation of a vehicle can be further increased.

The control element of the vehicle to be actuated for one control action can be different for different control actions. As an alternative or in addition, the control element of the vehicle can have a control means for the longitudinal and / or lateral guidance of the vehicle for an operating action that affects and / or influences the longitudinal and / or lateral guidance of the vehicle. and / or lateral guidance of the vehicle (for example a steering means, an accelerator pedal and / or a brake pedal). The safety of the operation of a vehicle can be further increased through the use of operating action-specific operating elements of a vehicle.

The operating element of the vehicle can thus comprise a control means for longitudinal and / or lateral guidance of the vehicle. The vehicle can be set up to temporarily decouple the control means for longitudinal and / or lateral guidance of the vehicle from the longitudinal and / or lateral guidance of the vehicle (e.g. by using drive-by-wire technology).

A confirming and / or contradicting actuation of the control means for longitudinal and / or lateral guidance of the vehicle can preferably be detected while the control means is decoupled from the longitudinal and / or lateral guidance of the vehicle (so that the actuation of the control means has no effect on the Longitudinal and / or lateral guidance of the vehicle is effected). The decoupling can e.g. take place while the vehicle is at least partially automated. After the control means have been actuated, the control means can then be coupled to the longitudinal and / or lateral guidance of the vehicle (e.g. to enable manual driving intervention by the vehicle driver). By temporarily decoupling the control means, comfortable and safe operation of a vehicle can be made possible.

The required operating action or operation of the vehicle can be aimed at setting and / or tracking a quantitative variable of the vehicle. The quantitative variable can possibly be adjustable in a continuous range of values (e.g. by deflecting the accelerator pedal). The method can include causing a plurality of (e.g. 10 or fewer) selectable options for each different discrete values for the quantitative variable to be output on the mobile user device. By specifying discrete values, an operator action can be carried out comfortably and safely.

The method can include the detection of a driving situation and / or the determination of a need for action during operation of the at least partially automated vehicle. This can e.g. can be determined on the basis of environment data from one or more environment sensors of the vehicle. In addition, the method can include the transmission of information on the recorded driving situation and / or on the determined need for action to the mobile user device. The information can be sent from the vehicle to the user device, and user device can be received. In addition, the method can include determining at least one operating action in relation to the driving situation and / or in relation to the need for action on the basis of the transmitted information. In this way, a more comfortable, efficient and safe operation of a vehicle can be made possible. In particular, it can thus be made possible to use the resources of a mobile user device at least partially for the determination of a required operating action in a vehicle.

The method can include the determination of authentication information in relation to an authentication of a user on the mobile user device. For this purpose, the user device can comprise one or more sensors (e.g. a fingerprint sensor, a camera for face recognition, etc.). The validity of an operating action initiated by the mobile user device can then be checked on the basis of the authentication information. It can thus be guaranteed that an operator action can be initiated by an authenticated user. Safe operation of a vehicle is thus made possible.

The method can include checking whether a user of the mobile user device has caused a decision on an operator action to be made by an automatic system. This can be done, for example, by setting a parameter on the user device. The automatic system can e.g. Be part of the user device, an external (backend) server and / or the vehicle. It can then be caused (e.g. by the user device and / or by the vehicle) that the automatic system makes a decision with regard to the operating action for the condition of the vehicle. Comfortable and safe operation of a vehicle can thus be made possible.

According to a further aspect, a device for an at least partially automated vehicle is described. The device is set up to detect that a mobile user device of an occupant of the vehicle is located in an interior of the vehicle. In addition, the device is set up to determine that at least one operating action is required for the condition of the vehicle. Furthermore, the device is set up to cause an operating element to be provided for executing the at least one operating action on the mobile user device.

According to a further aspect, a mobile user device is described which is set up at least partially by a device automatically driving vehicle to receive information (eg a control instruction) in relation to a required operator action for the condition of the vehicle. In addition, the user device is set up to cause an operating element to be provided for performing the at least one operating action on the mobile user device. In particular, the mobile user device is set up to request, establish and / or operate an operative connection with a vehicle. The mobile user device can be set up to recognize that it is being carried or operated in the vehicle and be set up to request, establish or operate an operative connection with the vehicle depending on the detected information, with at least one (further) step of the method is executable.

According to a further aspect, a (road) (motor) vehicle (in particular a passenger car or a truck or a bus) is described which comprises the control unit or device described in this document. Furthermore, the vehicle can be a (in particular at least partially automatically operated) watercraft or (in particular at least partially automated flying) aircraft. As an exception, the vehicle can be almost any machine that can be operated at least partially automatically, e.g. be an agricultural machine, manufacturing machine, etc., wherein the method can be applied in a correspondingly adapted form. In particular, the vehicle is set up to request, set up or operate an operative connection with a mobile user device. The vehicle (in particular at least one corresponding device of the vehicle) can be set up to recognize that a mobile user device is being carried and / or operated in the vehicle. In particular, the vehicle is also set up to request, set up or operate an operative connection with the mobile user device as a function of the detected information, with at least one step of the method being carried out.

According to a further aspect, a computer program, in particular a computer program product comprising a computer program, is described, the computer program being designed to carry out the method described in this document when it is executed on at least one computing unit.

The computer program product can be designed as an update of a previous computer program which, for example, includes the parts of the computer program or the corresponding program code as part of a function expansion, for example as part of a so-called “remote software update”. The computer program product comprises, in particular, a medium that can be read by the data processing device and on which the program code is stored, or at least an encrypted file. According to a further aspect, a program product is described which includes an authorized access right to stored data of the computer program product.

In the context of the document, the term “automated driving” can be understood to mean driving with automated longitudinal or lateral guidance or autonomous driving with automated longitudinal and lateral guidance. The automated driving can be, for example, driving on the motorway for a longer period of time or driving for a limited time as part of parking or maneuvering. The term “automated driving” includes automated driving with any degree of automation. Exemplary degrees of automation are assisted, partially automated, highly automated or fully automated driving. These degrees of automation were defined by the Federal Highway Research Institute (BASt) (see BASt publication “Research compact”, edition 11/2012). With assisted driving, the driver continuously performs longitudinal or lateral guidance, while the system takes on the other function within certain limits. With partially automated driving (TAF), the system takes over the longitudinal and lateral guidance for a certain period of time and / or in specific situations, whereby the driver has to continuously monitor the system as with assisted driving. With highly automated driving (HAF), the system takes over the longitudinal and lateral guidance for a certain period of time without the driver having to monitor the system permanently; however, the driver must be able to take over driving the vehicle within a certain period of time. With fully automated driving (VAF), the system can automatically cope with driving in all situations for a specific application; a driver is no longer required for this application. The four degrees of automation mentioned above correspond to SAE levels 1 to 4 of the SAE J3016 standard (SAE - Society of Automotive Engineering). For example, highly automated driving (HAF) Level 3 corresponds to the SAE J3016 standard. In addition, SAE J3016 provides SAE level 5 as the highest level of automation, which is not included in the definition of BASt. SAE level 5 corresponds to driverless driving in which the system can automatically handle all situations like a human driver during the entire journey; a driver is generally no longer required. The aspects described in this document relate in particular to a vehicle according to SAE level 3, 4 or 5.

It should be noted that the procedures, devices and Systems can be used both alone and in combination with other methods, devices and systems described in this document. Furthermore, any aspects of the methods, devices and systems described in this document can be combined with one another in diverse ways. In particular, the features of the claims can be combined with one another in diverse ways.

• 1 ein beispielhaftes System zur Interaktion mit einem, insbesondere zumindest teilweise automatisiert fahrenden, Fahrzeug; und
• 2 ein Ablaufdiagramm eines beispielhaften Verfahrens zur Interaktion mit einem, insbesondere zumindest teilweise automatisiert fahrenden, Fahrzeug.

The invention is described in more detail below using exemplary embodiments, without restricting the generality. The features of the examples can be combined with any of the other features identified in this document. Show:

• 1 an exemplary system for interacting with a vehicle that is in particular at least partially automated driving; and
• 2 a flowchart of an exemplary method for interacting with a vehicle that is in particular at least partially automated driving.

1 shows an exemplary system 100 to interact with a vehicle 110 , especially for operating a vehicle 110 . The vehicle 110 is set up to drive at least partially automatically. For example, the vehicle is designed with driver assistance systems or other means for performing an at least partially automated journey. The vehicle can 110 be set up to implement different degrees of automation, e.g. 1, 2, 3, 4, 5 according to BASt. For this purpose the vehicle can 110 one or more environmental sensors 112 which are set up, environment data in relation to an environment of the vehicle 110 capture. Exemplary environment sensors 112 are an image camera, a radar sensor, a lidar sensor, etc. A control unit 111 of the vehicle 110 can be set up, one or more longitudinal and / or lateral guidance actuators 113 (eg a drive motor, a braking device and / or a steering device) of the vehicle 110 to operate the vehicle depending on the environmental data 110 (if necessary, partially, vertically or fully) automatically feed lengthways and / or crossways. In general, it can be a vehicle that is only or predominantly manually drivable or manually driven 110 act.

The system also includes 100 a mobile user device 120 (e.g. a smartphone or a tablet PC) that is set up with the vehicle 110 (especially with a communication unit 114 of the vehicle 110 ) via a (wireless) communication link 101 to communicate. The communication can take place directly between the user device 120 and the communication unit 114 of the vehicle 110 respectively. For example, the communication unit 114 of the vehicle 110 set up a WLAN access point for the user device 120 provide, and / or a Bluetooth connection with the user device 120 build up.

The user device 120 can be a user interface 121 , 122 for one user. In particular, the user device 120 an output unit 121 (eg a screen) for outputting information and / or an input unit 122 (for example a keyboard, a control element, a touch-sensitive screen, etc.) for entering data (especially for controlling the vehicle 110 ) include.

Communication between the vehicle 110 and the user device 120 can if necessary via a central unit 130 enabled and / or controlled. Through the central unit 130 In particular, it can be checked whether the user device 120 and / or a user of the user device 120 are entitled to use the vehicle 110 to communicate.

Furthermore, the vehicle 110 and / or the user device 120 each position sensors 115 , 125 which are set up, position data (eg GPS coordinates) in relation to the position of the vehicle 110 or the user device 120 to investigate. Furthermore, the vehicle 110 and / or the user device 120 each user sensors 116 , 126 (eg a camera) which are set up include sensor data relating to the user of the vehicle 110 or the user device 120 capture.

The user of the vehicle is in particular the driver of the vehicle, in particular a person at a driver's seat of the vehicle or a person who at least partially carries out a driving task or who is responsible or authorized to carry out a driving task if necessary understand. Alternatively or in addition, the user can be a passenger of the vehicle, in particular a person who is not involved in the driving task or is normally not responsible or authorized for performing the driving task.

In one application scenario, the user of a vehicle 110 in the driver's seat of the vehicle 110 sitting and (during an automated journey) operating a mobile user device 120 (e.g. a smartphone, data glasses, a smart watch, etc.). The user can take this deep into the operation of the user device 120 Be in-depth that the user would need up to 5 seconds, for example, to get fully back to driving the vehicle as the driver 110 to be able to return. Also can it can be inconvenient for a user from preoccupation with his user device 120 to be torn.

In a further application scenario, the user of a vehicle 110 be a passenger, for example a person not or normally not, or not involved in the driving task in the current driving operation. This person can be in a passenger seat or back seat of the vehicle 110 sit and (while another person is sitting in the driver's seat and steering the vehicle, or in a highly automated driving mode of the vehicle) in operating a mobile user device 120 (e.g. a smartphone, data glasses, a smart watch, etc.).

In this application scenario, the (actual, active and / or responsible) driver of the vehicle may possibly be inadequate, permanently or temporarily reduced, driving readiness 110 recognized and taken into account. For example, if it is recognized that a responsiveness, readiness to drive, driving performance and / or a state of the driver falls below a certain level, the method for the user device can 120 one, in particular the most suitable, further user of the vehicle 110 are executed.

In another application scenario, the vehicle can 110 at a standstill and / or in a stopped state, e.g. at a traffic light, pedestrian crossing, barrier, stop sign, etc.

In one example of the method, a change in the status of a traffic light or a barrier can be determined, in particular predicted, and taken into account in the method. For example, a state, in particular a change in a state of a traffic light or barrier, can be determined, in particular predicted, and in response to this it can be caused that an operating element for performing the at least one operating action on the mobile user device 120 provided.

For example, a transition of the traffic light to a red phase can be determined, in particular predicted, and it can then be arranged that an operating element is provided for carrying out an (for example at least partially automated) stop at the traffic light (for example at the stop line or behind a vehicle in front) becomes. The stopping can alternatively or additionally be dependent on a (slight, normally insufficient to stop) actuation of a brake pedal of the vehicle 110 are executed.

For example, a transition of the traffic light to a green phase can be determined, in particular predicted, and it can then be arranged that an operating element is provided for carrying out an (for example at least partially automated) approach to the traffic light (for example with or without a vehicle in front). The start-up can alternatively or additionally be dependent on a (slight, normally insufficient for starting) actuation of an accelerator pedal of the vehicle 110 are executed.

In particular, the provision, in particular effectiveness or operability, of the operating element on the mobile user device 120 take place in the method depending on a determined, predetermined condition, which is indicative of the fact that an at least partially automated action (in the above example, stopping or starting at the traffic light) can or will be performed with sufficient reliability. For example, with the predetermined operation it can be sufficiently ensured that there are no obstacles in the way of the vehicle 110 and / or there are no risks (in particular that exceed a certain limit value or that are specific) in the case of an at least partially automated action being carried out.

The vehicle 110 , especially the control unit 111 of the vehicle 110 , can be set up to enable the user to select an operating action for the vehicle 110 from the user device 120 to make out. For example, it can be effected on the screen of the user device 120 an operating request appears that enables the user to control the vehicle 110 at least partially from the user device 120 out to use. For this purpose, on the screen of the user device 120 a control element can be provided.

• • zusätzlichen (insbesondere einer bestimmten ersten) Betätigung eines ersten Bedienelements des Fahrzeugs 110, und/oder
• • bei einer nicht erfolgten (insbesondere einer bestimmten zweiten) Betätigung eines ersten Bedienelements und/oder bei einer nicht erfolgten Betätigung eines zweiten (von dem ersten Bedienelement unterschiedlichen) Bedienelements des Fahrzeugs 110,

gültig (und in Reaktion darauf ausgeführt) wird.The reliability, safety and / or quality of operating the vehicle 110 , especially an interaction with the vehicle 110 , can be increased, for example, by a safety-critical operating action for operating the vehicle 110 only with one:

• • additional (in particular a specific first) actuation of a first control element of the vehicle 110 , and or
• If a first operating element is not actuated (in particular a specific second) and / or a second operating element of the vehicle (different from the first operating element) is not actuated 110 ,

is valid (and executed in response).

Furthermore, a safety-critical operating action for operating the vehicle 110 in the event of a detection of a worsened (further worsening) driving situation and / or an inability to drive or inability of the user of the vehicle to make decisions (anyway).

For example, if an urgent decision about a maneuver is necessary, which vehicle 110 alone cannot perform sufficiently well, or if there are several decision options, then the user can be enabled to make a decision from the mobile user device 120 out and then (briefly) a control element of the vehicle 110 (e.g. the accelerator pedal or the steering wheel) to confirm the on the user device 120 to confirm the decision made. A (subsequent or alternative) manual option for taking over the vehicle control by the user (ie the driver) can be retained.

There is thus a method, a device comprising a mobile user device 120 , and a computer product for operating an at least partially automated vehicle 110 described.

2 shows a flow diagram of an exemplary method 200 for operating an at least partially automated vehicle 110 . The procedure 200 includes detecting 201 that is a mobile user device 120 of an occupant of the vehicle 110 in an interior of the vehicle 110 is located. The procedure also includes 200 determining 202 that at least one operator action to condition the vehicle 110 is required. The method also includes 200 causing 203 that an operating element for performing the at least one operating action on the mobile user device 120 provided.

It becomes a process 200 to operate a vehicle 110 by means of one with the vehicle 110 carried mobile user device 120 described. The procedure 200 may include recognizing that the mobile user device is 120 inside the vehicle 110 is located. In addition, the procedure 200 comprise determining that at least one operator action for operating the vehicle 110 is required. Furthermore, the procedure 200 controlling or operating the mobile user device 120 depending on the required operator action to operate the vehicle 110 include. The control or operation can take place in such a way that an operating element for executing the at least one operating action of the vehicle 110 on the mobile user device 120 is activated.

For example, using a local wireless connection (e.g. Bluetooth, WLAN) in the vehicle 110 be recognized that the mobile user device 120 with the vehicle 110 is carried along, and in particular that the user device 120 inside the vehicle 110 is located (even when the vehicle is in motion 110 ). Alternatively or in addition, the mobile user device 120 , especially explicitly as the mobile user device 120 of the driver, on the vehicle 110 or can be registered and / or automatically recognized in a vehicle's own system, for example by means of "pairing". Alternatively or in addition, the position data of a position sensor can be used 115 , 125 of the vehicle 110 and / or the user device 120 be recognized that the user device 120 with the vehicle 110 is carried.

The transmission of a control signal for executing a via the mobile user device 120 initiated operating command can possibly by means of such a wireless connection 101 , preferably using the same wireless connection, within the vehicle 110 respectively.

The determination of at least one operating action necessary for operating the vehicle 110 necessary or recommended, can be done by means of the vehicle 110 respectively. As an alternative or in addition, such an operator control action can be carried out using the mobile user device 120 (e.g. based on collected data, based on that from the vehicle 110 read out data, using further means of the mobile user device 120 , etc.) can be determined.

An operating element for executing the at least one operating action can be on the mobile user device 120 for example as a touchscreen area to be operated, as a recognizable gesture, in particular a 3D gesture, for example a gesture to be performed with fingers, or as another type of virtual operating element. Such a control element can preferably be activated as a selection option, a slider, a coordinate field, etc. Such an operator control element is preferred as a function of the determination of at least one operator control action required to operate the vehicle 110 is necessary or recommended. A certain operating action can be suggested to the user.

Alternatively or additionally, the operating element of the user device 120 are already displayed in advance, but not necessarily activated and / or not displayed as activated. For example, in connected mode (if the user device 120 with the vehicle 110 via the (local) wireless communication link 101 connected) the one or more control elements of the user device 120 (to the Causing an operator action) to be reduced in size and / or represented as inactive.

The control element can then be activated on the user device as required 120 are displayed. When it is determined that the vehicle 110 an operator control action, in particular a specific one, is required or is recommended, the operator control element on the mobile user device can 120 marked as active. For example, this can be represented by an enlargement, coloring, flashing, vibrating, etc.

The procedure 200 has advantages compared to a possible mirroring of a vehicle instrument cluster on a user device 120 (e.g. on a tablet). According to the procedure described 200 not all information from the vehicle is constantly available 110 on the user device 120 only or primarily only the at least recommended or necessary operator actions are shown on the user device 120 displayed. An operating element can thus be selectively provided for a specific operating action (while no operating element is otherwise provided). In this way, convenient use of the user device can thus be made possible.

The (selectively) recommended or necessary operating actions can be carried out by means of a comparison with one another or with possible further actions or operating actions that are carried out with the mobile user device 120 are executable, selected, in particular prioritized. The prioritization can take place, for example, according to importance and / or urgency.

This enables the mobile user device 120 can be used largely undisturbed (and most of the time on the full display area). Furthermore, unauthorized control of the vehicle 110 excluded from the outside.

An operating action can preferably also be or include a (specific) sequence of operating actions. For example, an operating action can operate, in particular the control, of different devices or systems of the vehicle 110 , for example to achieve a goal. Furthermore, an operating action can include the execution of a, for example, multi-part driving maneuver or parking maneuver, turning maneuver, etc.

The procedure 200 may determine a measure of the user's (ie, driver's) attentiveness with respect to the mobile user device 120 include. In particular, it can be determined whether or that a mobile user device 120 is currently being used by the driver. Alternatively or in addition, the procedure 200 include identifying a level of driver alertness with respect to a driving task.

The mobile user device 120 can then be dependent on the determined level of attention of the driver in relation to the mobile user device 120 and / or controlled and / or operated depending on the determined level of attention of the driver with regard to a driving task.

In the context of this document, the term “driver” can be used in particular as a user of an at least partially automated vehicle 110 be understood. In particular, the driver can use one in the vehicle 110 be the person present who is at least partially responsible or responsible for the performance of driving tasks. In other words, the driver may not necessarily be responsible for carrying out a substantial part of the driving tasks, but in particular is not a pure passenger of the vehicle 110 . The driver can, for example, at the driver's position (in particular in the driver's seat) of a vehicle 110 be arranged

As part of the procedure 200 can be a frequency and / or a continuity of the operation of the mobile user device 120 determined and / or taken into account. The frequency and / or the continuity of the operation can be used as the degree of attention and / or as a further criterion for controlling or operating the mobile user device 120 be taken into account.

The mobile user device is particularly preferably controlled 120 , in particular the activation of (certain) operating elements, depending on a distribution of the driver's attention between the mobile user device 120 and the driving task. As the distribution of attention, a mathematical relationship (for example, in a simple example, a proportion) of the said measures of attention can be taken into account. Furthermore, it can be taken into account that the ability to act and / or reaction time, etc. due to the (specific, determined) distribution of the driver's attention between the mobile user device 120 and the driving task (also overall) decreases. It can be taken into account that in the event of an unfavorable, for example rapidly alternating, distribution of attention, both of the aforementioned activities of the driver can be significantly impaired. In other words, it can be ascertained and taken into account that an ascertained divided attention leads to an overall deteriorated ability to act and / or reaction time, etc.

For example, if it is determined that the driver's attention is predominantly on the mobile user device 120 is, more operator actions on the mobile user device 120 activated (and thus offered for operation). Thus, the extent of the operator actions on the mobile user device 120 with increasing attention to the mobile user device 120 climb.

If, on the other hand, it is determined that the driver's attention is predominantly or at least sufficiently during the driving task, an operating action may possibly not be possible on the mobile user device 120 activated (and thus offered for operation). Such an operating action can then preferably be performed by a device (for example, by an operating element) of the vehicle 110 offered or prevented.

The level of attention of the driver in relation to a driving task can be recognized with means known per se and / or on the basis of the driving activity of the driver, for example operating the steering wheel and / or a pedal. Furthermore, one can use means of the vehicle 110 determined driving quality of the driving activity of the driver (eg as correctness, precision and / or meaningfulness of the driver's actions) can be estimated and taken into account.

As part of the procedure 200 can possibly be a specific one on the mobile user device 120 Operator action initiated only or mainly only on the vehicle 110 be executed when it is detected that a certain control element of the vehicle 110 , especially suitable for the operation of the control element of the mobile user device 120 , is operated and / or none for the operator action on the mobile user device 120 Negative or contradicting operation of a control element of the vehicle 110 is recognized.

For example, via the mobile user device 120 one, especially complex, operation of the vehicle 110 (for example to carry out a maneuver, eg a lane change maneuver after or in front of a certain vehicle, an overtaking maneuver of a certain vehicle, a parking maneuver in a certain parking space, etc.). The specification of the operator action can be specified in the procedure 200 for example only then or only then (completely) carried out if a correspondingly appropriate confirmation via a control element of the vehicle is additionally provided 110 is recognized.

For example, an at least brief, “symbolic” actuation of the steering wheel and / or a pedal of the vehicle can be used 110 are assumed. The operating action can possibly only be carried out with the detection of such an actuation. During or in response to the detection of the confirmation, the on the mobile user device 120 entered operator action on the vehicle 110 are executed.

For example, the user can (also with the aid of various input and output devices of the mobile user device 120 ) an operator control for a specific sequence of actions (the execution of which the user can control from the vehicle 110 wishes) into the mobile user device 120 enter. Optionally, the mobile user device 120 generate an output indicating that the sequence of actions is at least partially executable or not executable and / or recommended or not recommended.

Optionally, information can be displayed to the user that an input, for example a specific control action, via a specific (safe) control element of the vehicle to implement the control action 110 must be done. Alternatively or in addition, another option of the procedure 200 the operating action (for example the sequence of actions) is carried out if no, in particular no contradicting or contrary, operation of an operating element of the vehicle is performed in the same or a subsequent time interval 110 is recognized.

Thereupon, after the detection of the actuation and / or the non-detection of a negative or contradicting actuation of an operating element of the vehicle 110 , the operating action, for example a sequence of actions and / or maneuvers, etc. can be performed by the vehicle 110 are executed.

The at least one operating action, in particular relating to setting and / or tracking a quantitative variable of the vehicle 110 , can possibly be assigned to a limited number of selection options or options for action. The selection options or options for action can be used when activating the mobile user device 120 offered for selection (and by the user of the mobile user device 120 can be selected selectively).

For example, an operating action can be a continuous setting and / or tracking of a quantitative variable, for example a pedal angle, steering wheel angle, etc. of the vehicle 110 include. This continuous adjustment can be part of the procedure 200 , in particular instead of a pedal angle or steering wheel angle on the mobile user device 120 must be entered explicitly, converted or broken down into a relatively small number of discrete operating variants. This Operating variants can then be selected error-free and free of misunderstandings by the driver.

For example, when it comes to continuous control or tracking of an actuator of the vehicle 110 goes, and the driver this control or tracking via his mobile user device 120 wants to operate (or has to operate in favor of a quick reaction in a critical situation), the operating action can be represented as a limited number (eg 2-5) simplified operating options (which can be selected with a relatively short operating action). The via the mobile user device 120 The operation offered is thus prepared for the driver in a limited number of selectable options. The driver can then, with at least one operating action on the mobile user device 120 select one of the options for action relatively quickly.

An operator action on the mobile user device 120 can possibly an execution of one, with means of the vehicle 110 automated, action, in particular a control task with a specific goal, in particular a maneuver.

As an alternative or in addition, an operating action for operating the vehicle can be used 110 relate to a driving task, in particular to the execution of a certain maneuver or a maneuver of a certain type. In particular, the operating action can affect the guidance of the vehicle 110 , especially on the path of the vehicle 110 , eg the longitudinal guidance and / or the lateral guidance of the vehicle 110 , Respectively. The driving task can relate to the control, in particular the regulation in relation to at least one object in the vicinity of the vehicle 110 , Respectively. For example, the user can (possibly with a touch input on a touch-sensitive screen 121 of the user device 120 ) select which other object from the vehicle 110 should be followed automatically, overtaken, etc.

An operator action that controls the vehicle 110 affects or influences, can only be carried out if it is recognized that a control means of the vehicle is at the same time or concomitantly or in a predetermined time interval 110 to control the longitudinal guidance and / or the lateral guidance of the vehicle 110 particularly in the predetermined manner.

This particularly preferably relates to a selection of a maneuver option. For example, it may be necessary that the vehicle 110 due to a traffic situation that changes relatively quickly, recognizes the decision of a user relatively quickly, for example that the vehicle 100 overtake, slow down, or cut in. Alternatively or additionally, it may be necessary to abort a (specific) maneuver or to change it (in a specific way).

It can be used in the (possibly in his mobile user device 120 intellectually deepened) users on the user device 120 offered to execute a certain maneuver, to modify it, for example to execute it to the right or to the left, or to cancel it. The user can do this by simply clicking on a relevant option via the control element of the mobile user device 120 to do. Furthermore, it may be necessary that at the same time or at least shortly after the user make his decision by operating a control element of the vehicle 110 (For example, the steering wheel, at least by gripping the steering wheel and / or an accelerator pedal or brake pedal). So can a safer operation of the vehicle 110 are made possible.

It can also be provided that the user can use the user device 120 initiated operation of the vehicle 110 , for example a maneuver, with a control element of the vehicle 110 correct, specify, take over and / or continue.

These measures can increase the security of the procedure 200 especially with regard to IT security. Furthermore, input errors by the user can be avoided. This also ensures that the driver's will is recorded and communicated safely and unambiguously.

Particularly preferably, a selected option is only carried out if this is done at the same time by at least one control means of the vehicle 110 , in particular by a control means for controlling the longitudinal guidance and / or the transverse guidance of the vehicle 110 particularly in a predetermined manner. For example, it may be necessary for the driver, for example depending on the information provided by the mobile user device 120 initiated maneuvers, including the accelerator pedal and / or the steering wheel and / or the brake pedal of the vehicle 110 pressed so that the proposed action becomes valid and then carried out. For example, when an overtaking maneuver is made via the mobile user device 120 is displayed, suggested or recommended as an option for action (and selected by the user), an actuation of the accelerator pedal of the vehicle can possibly 110 , in particular in a predetermined manner, may be required to carry out the overtaking maneuver.

In this case, the option of action can be confirmed by a control means of the vehicle 110 a certain time interval before and / or after the operation of a corresponding one Control element of the mobile user device 120 be held up. In other words, the user can, for example, a maneuver, a maneuver option, a maneuver parameter through the mobile user device 120 pretend. A corresponding confirmation of his operating action can then also take place after the beginning of the execution of his operating action, in particular within a predetermined time interval. Furthermore, an operating action can possibly be ended or corrected by manual control up to a point in time at which the operating action can still be reversed (without significant consequences).

As part of the procedure 200 can possibly be an affirmative action on a control element of the vehicle 110 , in particular on a handlebar and / or a pedal of the vehicle 110 , be done while the control of the vehicle 110 is in an at least partially decoupled state. A confirmation of a by the mobile user device can be particularly preferred 120 initiated operator action in relation to the lateral guidance of the vehicle 110 by operating the at least partially from the steering line of the vehicle during this time 110 decoupled steering wheel. Alternatively or in addition, a confirmation of a by the mobile user device 120 initiated operator action in relation to the longitudinal guidance of the vehicle 110 by operating the during this time at least partially from the control of the drive and / or the brake system of the vehicle 110 decoupled accelerator or brake pedal.

The procedure 200 can transmit information on a detected traffic situation and / or on a specific need for action by the occupant of the vehicle 110 to the mobile user device 120 of the occupant of the vehicle 110 include. At least one operating action can then be used to operate the vehicle 110 is necessary or recommended by means of the mobile user device 120 be determined.

In other words, the user device 120 can be used to evaluate vehicle environment data 110 be integrated. Furthermore, the user device can 120 be set up, for example with the aid of digital map material or data on which the user device 120 accesses and with its own computing power at least co-determined to determine when an operator action of the vehicle 110 and / or an at least partial takeover of vehicle control by the driver of the vehicle 110 is necessary or recommended.

As part of the procedure 200 can at least one means of authentication of the mobile user device 120 , in particular a biometric authentication means of the mobile user device 120 to be read out. It can be checked, for example, whether or not the user is authorized to perform the specific operating action, for example in relation to driving the vehicle. Depending on this, there can then be a confirmation of the validity of the by the mobile user device 120 on the vehicle 110 Operator action initiated, for example with regard to driving the vehicle.

A fingerprint sensor and / or an iris scanner of the mobile user device can be used as an authentication means 120 or a specific swiping movement typical for the authorized user (eg as a type of “signature”) can be recognized.

For example, the method can be designed in such a way that each, in particular one that is safety-critical and / or associated with driving the vehicle, is operated with a separate finger pressure on a fingerprint sensor and / or with a look into the camera of the mobile user device 120 must be confirmed. So can the safety of operating a vehicle 110 be increased further.

The control of the mobile user device 120 may include a decision option with which the driver transfers the decision or operating action, in particular also a power of attorney to carry out the decision or operating action, to an automatic system. In other words, the user or driver can transfer or leave the decision about the operating situation to an automatic system for a specific case. The automatic system can be in the vehicle 110 , in a backend server 130 , or in the mobile user device 120 be arranged. The user can pass on the decision, for example, if the user cannot determine a sufficiently good decision or operating action in a given situation, and / or if the user is of the opinion that the automatic system can determine a better decision or operating action in the present situation. This can be particularly advantageous in time-critical situations.

The procedure 200 the detection of a traffic situation and / or the determination of a specific need for action of at least one occupant of the at least partially moving vehicle 110 include. Furthermore, the procedure 200 the transmission of information on the recorded traffic situation and / or on the at least one specific need for action of the vehicle occupant 110 to the mobile user device 120 of the occupant of the vehicle 110 include. Furthermore, the procedure 200 then include that Controlling an actuator of the mobile user device 120 depending on the information transmitted.

For example, while the occupant is still at the driver's seat, for example, in his mobile user device 120 looks and his attention essentially on the user device 120 is concentrated, the occupant is enabled to identify the situation and / or a required action required by the mobile user device relatively quickly 120 capture. Then the user can act in the real world with increased speed and meaningfulness.

This is particularly preferably done in such a way that the display and / or an actuator of the user device 120 a specific (concrete) information, in particular a specific feature on the traffic situation and / or on the specific (concrete) need for action of the driver, represents.

The evaluation of the traffic situation is preferably at least partially using the vehicle 110 executed. The (appropriately designed and / or equipped) sensors of the vehicle 110 and / or means of at least one driver assistance system or a system for executing the at least partially automated driving are used.

Thus, the driver or the occupant can do something related to driving the vehicle 110 should do and focus his attention on the user device 120 focused, prepared for the real situation. As a result, it takes longer to find your way around and / or orientate yourself in the real situation (after you have turned your gaze away from the user device 120 ) be avoided.

• • eine Veränderung einer aktuellen Anzeige des mobilen Anwendergeräts 120, z.B. Minimierung einer ablenkenden Anzeige, und/oder ein Erzeugen einer weiteren Anzeige an dem mobilen Anwendergerät 120 in Zusammenhang mit der erkannten Verkehrssituation bzw. mit dem Handlungsbedarf des Insassen des Fahrzeugs 110; und/oder
• • eine Veränderung einer Audioausgabe des mobilen Anwendergeräts 120, z.B. Reduktion der Lautstärke oder Pausieren einer auditiven Kommunikation, und/oder eine Ausgabe eines Sounds in Zusammenhang mit der erkannten Verkehrssituation bzw. mit dem Handlungsbedarf des Insassen des Fahrzeugs 120 mittels des mobilen Anwendergeräts 120; und/oder
• • eine Ansteuerung eines weiteren Aktors des mobilen Anwendergeräts 120, insbesondere eines Unwuchtaktors, Vibrationsaktors, und/oder Beschleunigungsaktors, in Zusammenhang mit der erkannten Verkehrssituation bzw. mit dem Handlungsbedarf des Insassen des Fahrzeugs 120.

Particularly preferably, depending on the information transmitted in connection with the recognized traffic situation or with the need for action by the occupant of the vehicle 110 one or more of the following measures are taken:

• A change in a current display of the mobile user device 120 , for example minimizing a distracting display, and / or generating a further display on the mobile user device 120 in connection with the recognized traffic situation or with the need for action by the occupant of the vehicle 110 ; and or
• • a change in an audio output of the mobile user device 120 , eg reducing the volume or pausing an auditory communication, and / or outputting a sound in connection with the recognized traffic situation or with the need for action by the occupant of the vehicle 120 by means of the mobile user device 120 ; and or
• • a control of a further actuator of the mobile user device 120 , in particular an unbalance actuator, vibration actuator, and / or acceleration actuator, in connection with the recognized traffic situation or with the need for action by the occupant of the vehicle 120 .

• • eine Zuwendung der Fahreraufmerksamkeit zu der Fahrzeugumgebung;
• • eine Zuwendung der Fahreraufmerksamkeit zu einer Information des Fahrzeugs 110, insbesondere zu einer Anzeige des Fahrzeugs 110, insbesondere betreffend die Fahrzeugführung des Fahrzeugs 110;
• • eine Aufforderung zum Greifen des Lenkrads des Fahrzeugs 110, z.B. für einen Hands-On-Request;
• • eine Aufforderung zum Antippen eines Pedals des Fahrzeugs 110, insbesondere zur Überprüfung der Bedienbereitschaft;
• • eine Aufforderung zur Veränderung der Fahrzeuggeschwindigkeit;
• • eine Aufforderung zum Einhalten eines, insbesondere seitlichen Abstands;
• • eine Aufforderung zum Mitwirken des Fahrers bei einem bestimmten Manöver; und/oder
• • eine Übernahmeaufforderung der Fahrzeugführung durch den Fahrer.

As part of the procedure 200 can control a display and / or an actuator of the mobile user device 120 respectively. This can be representative for:

• • a turning of the driver's attention to the vehicle environment;
• • A turning of the driver's attention to information about the vehicle 110 , in particular a display of the vehicle 110 , in particular with regard to driving the vehicle 110 ;
• • a request to take hold of the vehicle's steering wheel 110 , e.g. for a hands-on request;
• • a request to tap a pedal on the vehicle 110 , especially to check the operational readiness;
• • a request to change the vehicle speed;
• • a request to keep a distance, in particular a lateral distance;
• • a request for the driver to participate in a certain maneuver; and or
• • a request for the driver to take over the vehicle.

The procedure 200 can transmit information on the recorded traffic situation and / or on the need for action and / or on the control of a display and / or an actuator of the mobile user device 120 include, depending on one or more predetermined criteria in relation to the determined traffic situation and / or in relation to the determined need for action at least one occupant of the at least partially moving vehicle 110 .

• • eine Einschätzung der Wichtigkeit und/oder der Eiligkeit einer Information zu einer erkannten Verkehrs situation und/oder eines Handlungsbedarfs des Fahrers;
• • ein Erkennen einer Verkehrs situation als eine Verkehrs situation einer bestimmten Art, z.B. mit einer relativ hohen Kollisionswahrscheinlichkeit;
• • ein Erkennen des Handlungsbedarfs als einen Handlungsbedarf hinsichtlich: der Längsführung des Fahrzeugs 110 und/oder der Querführung des Fahrzeugs 110; und/oder
• • ein Überschreiten eines vorausbestimmten Schwellwertes. Dabei können Schwellenwerte abhängig vom Fahrkontext (Stadtfahrt, Landstraßen Schnellstraße, Parken, Fahrer im dichten Verkehr, Nachtfahrt, etc.) gewählt werden.

A predetermined condition can be, for example:

• • an assessment of the importance and / or the urgency of information about a recognized traffic situation and / or a need for action by the driver;
• • recognition of a traffic situation as a traffic situation of a certain type, for example with a relatively high probability of a collision;
• • A recognition of the need for action as a need for action with regard to: the longitudinal guidance of the vehicle 110 and / or the lateral guidance of the vehicle 110 ; and or
• • exceeding a predetermined threshold value. Threshold values can be selected depending on the driving context (city driving, country roads, expressway, parking, driver in heavy traffic, night driving, etc.).

In other words, only selected prioritized information (according to predetermined criteria) can be transmitted and / or (according to predetermined criteria) prioritized displays and / or control of the actuator of the mobile user device 120 respectively. As a result, the occupant, or driver, will only then be out of their occupation with the user device 120 taken out if this is the situation, in particular the "overall situation" of the vehicle 110 requires.

As part of the procedure 200 can be an active display content of the mobile user device 120 can be reduced or transformed, and / or a further display can be displayed that shows a relevant traffic situation and / or a specific need for action by the occupant of the vehicle associated with the traffic situation 110 represents.

A previous display can preferably be reduced, in particular minimized, with regard to its display area, display sharpness, virtually perceived display proximity, etc. A complete shutdown, hiding or shifting to the background can also be carried out as the reduction in this context.

As part of the procedure 200 Different threshold values can be differentiated with regard to the importance of the traffic situation and / or the importance and urgency of the occupant's need for action, and then different changes in the display of the mobile user device 120 and / or different, for example differently strong, actuations of the actuator of the mobile user device 120 are executed.

For example, in a relatively uncritical traffic situation on the mobile user device 120 a relatively unobtrusive information display, for example also as a picture-in-picture display together or within a further application. If the traffic situation requires an immediate reaction or action by the driver, the display can be emphasized more strongly.

The occupant's need for action can relate to the specific traffic situation. Furthermore, a specific action requested by the occupant or recommended to the occupant can be shown in the display of the mobile user device 120 be represented.

A display device 121 of the mobile user device 120 can be set up to represent a graphic image, in particular an image sequence, using the means of the vehicle 110 recognized traffic situation and / or an action of the occupant requested or recommended in this context. For example, if an acute collision is imminent while a user is using applications on the mobile user device 120 concerned, a video can be output that shows the focus of the traffic situation in question. A required action can also be displayed virtually in the video, which shows what the user must do to solve or defuse the traffic situation.

A representation of the traffic situation on the mobile user device that is interpreted by machine, in particular a simplified or focused on the essentials, is preferred 120 shown. This can help to grasp the traffic situation immediately.

• • zumindest einen Ausschnitt einer mit einer Kamera des Fahrzeugs 110 erfassten Abbildung der Umgebung des Fahrzeugs 110 sein;
• • eine mit einer Wärmebildkamera des Fahrzeug 110 erfasste Information repräsentieren, beispielsweise Fußgänger, andere Fahrzeuge, etc., umfassen;
• • zumindest teiltransparent in die Abbildung eingeblendete grafische Information umfassen, die eine betreffende Verkehrs situation kennzeichnet, insbesondere eine Abweichung von einer vorausbestimmten Norm. Beispielsweise kann ein zu geringer Abstand des Fahrzeugs (seitlich, hinten, und/oder vorne) zu einem weiteren Verkehrsteilnehmer angezeigt werden.

For example, the graphic illustration can:

• • At least a section of one with a camera of the vehicle 110 captured image of the surroundings of the vehicle 110 be;
• • one with a thermal imaging camera of the vehicle 110 represent captured information such as pedestrians, other vehicles, etc .;
• • include graphic information that is at least partially transparent into the image and characterizes a relevant traffic situation, in particular a deviation from a predetermined norm. For example, too little distance between the vehicle (to the side, rear and / or front) from another road user can be displayed.

A need for action, e.g. be designed as an instruction by means of a sequence of images, in particular a causally related sequence of images (in an animated form), in particular superimposed on the image of the environment.

The graphic image, in particular an image sequence representing the traffic situation and / or a need for action by the user related to the traffic situation, can possibly represent a reality predicted in the near future.

It can be a reality predicted in the near future, for example 1-15 seconds. Such a reality can be determined, for example, using a simulation. This can also be determined as a probability-based prediction. For example, at least one or two probable (most likely) developments in the traffic situation can be determined and on the mobile user device 120 are displayed.

The user can be shown what can happen and what he can or should do (only if this is the case). This saves time in order to explain a required action to the user by other means if it is already due or overdue. Instead, the required action (initially optional, for a variant of the development of the real situation) is explained. When the user turns his gaze to the real environment, he knows what to do when the traffic situation develops in the predicted way.

Depending on the information from one or more databases within the mobile user device 120 and / or outside of the mobile user device 120 , e.g. from the backend 130 , a cloud or from the vehicle 110 , read out data are determined and taken into account. The computing power of the mobile user device can also be used 120 can be used to make a corresponding decision to output a display and / or to operate an actuator of the user device 120 to calculate.

Furthermore, at least partially within the mobile user device 120 a need for action by the user in a relevant traffic situation can be determined.

You can also use the mobile user device 120 recorded data on the attention of the user and / or on the viewing direction of the user and / or on the behavior of the user and / or on the physical condition of the user (alertness, pulse, and / or reaction parameters) are taken into account.

As part of the procedure 200 can possibly be recognized that the occupant an audible content with the mobile user device 120 hears and / or telephones. Auditory information can then be output by means of the mobile user device 120 representing the information about the specific recognized traffic situation and / or about the requested or recommended action in connection with a recognized traffic situation. A vibration can particularly preferably be output as an alternative or in addition.

As part of the procedure 200 can by means of the mobile user device 120 a direction and / or an object in the vicinity of the vehicle 110 to which or to which the driver's attention should be directed.

When the driver uses his mobile user device 120 on his steering wheel and operated it, and if by means of the vehicle 110 it is recognized that the driver's attention is required in a certain direction, then the mobile user device can then use the means 120 represent a specific direction and / or direct the driver's attention in a specific direction.

In a simplified case e.g. an arrow, a running light and / or an image sequence can be displayed, which represents a direction that is related to the recognized traffic situation and / or in which the attention of the user is to be directed, and / or represents a direction of an action taken by is requested or recommended to the occupant (ie the user).

In doing so, the procedure 200 the determination of the determined direction by means of the vehicle 110 include. Furthermore, the procedure 200 determining the position (orientation) of the mobile user device 120 relative to the vehicle 110 and / or a specific direction relative to the mobile user device 120 includes. Furthermore, the procedure 200 the output of information relating to the determined direction by means of the mobile user device 120 include.

As part of the procedure 200 can be a mobile user device 120 inside the vehicle 110 than the user device used by the occupant in the driver's seat 120 be recognized. Furthermore, at least one mobile user device can if necessary 120 as a mobile user device 120 a further specific occupant or an occupant at a further seat can be recognized. The output of the information, representing the recognized traffic situation and an action by the occupant in the driver's seat 110 can then selectively on the mobile user device 120 of the occupant in the driver's seat. As an alternative or in addition, information representing the recognized traffic situation and an action by the occupant in the front passenger seat can be output using the mobile user device 120 of the occupant at another seat of the vehicle 110 respectively. Different occupants can thus possibly different information on the user device used 120 be transmitted.

Both pieces of information (for the two different user devices 120 ) preferably be different. For example, the information can differ with regard to the predetermined thresholds and / or with regard to the role of the occupants.

An occupant can, for example, sit in the front passenger seat while the vehicle is highly automated. The occupant in the driver's seat can sleep if necessary. In the case of one from the vehicle 110 recognized (urgent) need for action, the occupant in the front passenger seat can also use his user device 120 be notified. The occupant in the front passenger seat can then be requested to wake the driver or to give the driver a hint to pay more attention if there is no urgent need for action. Possibly. The occupant in the front passenger seat can also operate the vehicle 110 be taken over.

As part of the procedure 200 can therefore use several different user devices 120 be detected in the interior of the vehicle 110 are located. Furthermore, the different user devices 120 be operated in different ways to an operating action for an operation of the vehicle 110 to enable (e.g. depending on whether the user device 120 a driver or a passenger).

In particular, a specific occupant can be a specific person, for example the holder of the specific mobile user device 120 , or an occupant at a specific seat, such as the passenger seat of the vehicle 110 , be. The on the respective user device 120 Information to be output can be generated appropriately and / or appropriately addressed.

The procedure 200 may include recognizing that the driver's attention is essentially at a mobile user device 129 is, in particular, that a mobile user device 120 operated by the driver. Furthermore, the procedure 200 include the detection of a traffic situation and / or the determination of a need for action for the driver of the vehicle 110 . Furthermore, the procedure 200 include controlling a display and / or an actuator of the mobile user device 120 , depending on the recorded traffic information and / or depending on the determined need for action by the driver of the vehicle 110 .

The recognition that the driver's attention is essentially with a mobile user device 120 is, in particular, that a mobile user device 120 is operated by the driver, can with means of the mobile user device 120 be recognized by yourself. It can receive data from sensors 126 , for example the camera, the finger sensor and / or the operating elements, for example the buttons or the touchscreen, of the mobile user device 120 be evaluated.

It can be determined that the user device 120 demands a certain amount of excessive attention from the driver and / or that the user device 120 has an operating activity that exceeds a certain level.

As an alternative or in addition, a certain amount of excessive attention by the driver on a mobile user device can be used 120 by means of the vehicle 110 e.g. through a wireless connection to the vehicle 110 , be determined. Furthermore, this can be done using at least one sensor 116 of the vehicle 110 , for example an indoor camera, can be determined.

The traffic situation and / or the driver's need for action can be determined using the mobile user device 120 and / or by means of the vehicle 110 and / or by means of another with the vehicle 110 carried device. Furthermore, the traffic situation and / or the need for action can take place as a function of data from a computing unit arranged at a distance.

The traffic situation and / or the driver's need for action can, for example, be determined permanently and, for example, regularly taken into account. The traffic situation and / or the driver's need for action can then preferably be recorded or used to carry out the method 200 can be interpreted when it is recognized that the driver's attention is essentially in the case of a mobile user device 120 is, in particular, that a mobile user device 120 operated by the driver. The resources for determining the required data and for the data transmission can thus be reduced.

As part of the procedure 200 it can be recognized that the driver's attention to a driving task and / or driving activity of the driver falls below an appropriate level. A display and / or an actuator of the mobile user device 120 can then be controlled depending on the detected attention of the driver in relation to a driving task and / or driving activity of the driver.

In this context, the driver's attention with regard to a driving task can be drawn using known means, for example by detecting his head position, viewing direction and / or recognizing the driver's state, in particular using means in the vehicle 110 respectively. The driving activity of the driver is also to be understood as a (sufficient, in particular necessary for a sufficiently good driving style) perception of a (current or upcoming) driving task by the driver.

The driving activity of the driver can be determined by evaluating the actuation of the control elements of the vehicle 110 , especially for the longitudinal guidance and / or lateral guidance of the vehicle 110 to be executed. For example, the driving activity of the driver can be dependent on the haptic contact with a pedal of the vehicle 110 and / or a driver of the vehicle 110 be determined. The driving activity of the driver can particularly preferably be determined, in particular estimated, as a function of the degree of haptic contact. The degree of haptic contact can in turn with a hands-on sensor, a hands-off sensor, using a pressure-sensitive mat in the steering wheel of the vehicle 110 or with an indoor camera 116 of the vehicle 110 be determined.

That is particularly preferred, in particular with means of the vehicle 110 recorded driving activity is evaluated in relation to the current driving task or the current driving situation. The “appropriate level” can be determined as a variable related to the driving context and / or the driving situation.

As part of the procedure 200 can be the current application of the mobile user device 120 be recognized. As an alternative or in addition, it can be recognized which perception channel of the driver is due to a present application of the user device 120 is claimed. It can then be an actuator of the mobile user device 120 depending on the identified application of the mobile user device 120 be controlled.

For example, the mobile user device 120 held in the hand and thus telephoning, a first control, for example a vibration and / or a sound output, in particular including an announcement, can take place. For example, the mobile user device 120 operated on the screen, for example playing a game or looking at a map, a second control, for example a change and / or a display on the mobile user device, can be generated 120 , respectively.

The use case of the mobile user device 120 can be classified into at least two or more different classes. These classes can differ according to the required attention, the required look, the required listening and / or the required haptic operation, etc.

The procedure 200 may include the recognition of at least one particular perception channel of the user that is triggered by the mobile user device 120 and, if applicable, to what extent it is used. An actuator of the mobile user device can then be activated 120 depending on the recognized perception channel and possibly depending on the extent to which the perception channel is used.

Preferably, depending on the input and / or output of the mobile user device 120 , in particular also depending on (at least rudimentarily) evaluated content, it can be recognized which of the user's perception channels is currently the content of the mobile user device 120 perceives.

• • eine Vibration und/oder Soundausgabe, insbesondere umfassend eine Ansage, erfolgen; und/oder
• • eine Veränderung und/oder ein Erzeugen einer Anzeige am mobilen Anwendergerät 120 erfolgt.

Furthermore, it can be determined how intensively this perception channel is used or overloaded. It can then be controlled by one or more actuators of the mobile user device 120 to be decided. For example, it can be decided that

• • a vibration and / or sound output, in particular including an announcement, take place; and or
• • a change and / or generation of a display on the mobile user device 120 he follows.

The procedure 200 may include determining, in particular predicting, a variable representing a degree of automation and / or a degree of remote control of the vehicle 110 . Furthermore, the procedure 200 include controlling an actuator of the mobile user device 120 depending on the recognized variable, representing a degree of automation and / or a degree of remote control of the vehicle 110 .

This can be done by the mobile user device 120 Information to be output depending on the determined size, representing the degree of automation, selected differently and / or different in content. With a relatively low degree of automation, for example, relatively brief attention or operation of the mobile user device can also be required 120 lead to control of the actuator. This may possibly have little or no dependence on the driving situation and / or driving context.

In the case of a relatively high degree of automation, only relatively critical information can possibly be provided, for example on traffic situations from the vehicle 110 are not adequately mastered. These may have little or no dependence on the distraction time and a relatively high dependence on the determined (automatically evaluated, interpreted) driving situation and / or driving context.

The invention further comprises a device, possibly including at least one mobile user device, which is used to carry out the method 200 is set up, as well as a corresponding computer product, in particular an app.

The mobile user device includes 120 an appropriately set up communication interface with the vehicle 110 to at least part of the process 200 execute.

The measures described in this document can bring about numerous advantages: In particular, the traffic safety of an at least partially automated vehicle can be achieved 110 increase. Furthermore, it can be made possible for an occupant to communicate with a mobile user device in a comfortable manner 120 to employ, and yet promptly for interventions on a vehicle 110 to be available. In doing so, (to relieve the vehicle's resources 110 ) Computing resources of the mobile user device 120 be used.

The present invention is not restricted to the exemplary embodiments shown. In particular, it should be noted that the description and the figures are only intended to illustrate the principle of the proposed methods, devices and systems.

Claims (15)

Method (200) for operating a vehicle (110) by means of a mobile user device (120); wherein the method (200) comprises, - Detecting (201) that a mobile user device (120) of an occupant of the vehicle (110) is located in an interior of the vehicle (110); - Determining (202) that at least one operating action is required for the condition of the vehicle (110); and - causing (203) that an operating element for executing the at least one operating action is provided on the mobile user device (120). Method (200) according to Claim 1 wherein the method (200) comprises - determining a level of attentiveness of a user of the mobile user device (120) in relation to the mobile user device (120); and / or - determining a degree of attentiveness of a user of the mobile user device (120) in relation to a driving task of the vehicle (110); and - causing the mobile user device (120) to be operated as a function of the determined level of attentiveness in relation to the mobile user device (120) and / or in relation to the determined level of attention in relation to the driving task. The method (200) according to any one of the preceding claims, wherein the method (200) comprises, - Determining that the mobile user device (120), in particular by actuating the control element provided on the user device (120), has initiated an operator action; whereby the operator action is aimed in particular at - to carry out an automated function and / or a control-technical task with a means of the vehicle (110); and or - perform a driving task and / or a driving maneuver; and - Operating the vehicle (110) as a function of the operating action initiated on the mobile user device (120). Method (200) according to Claim 3 , wherein the method (200) comprises, determining whether an actuation of an operating element of the vehicle (110) confirming the operating action has taken place and / or whether no actuation of an operating element of the vehicle (110) contradicting the operating action has taken place; and - operation of the vehicle (110) as a function of the operating action initiated on the mobile user device (120) is prevented if it has been determined that no actuation of an operating element of the vehicle (110) confirming the operating action has taken place, and / or that one of the operating actions contradicting actuation of an operating element of the vehicle (110) has taken place; and / or - the vehicle (110) is only operated as a function of the operating action initiated on the mobile user device (120) if it has been determined that an operating element of the vehicle (110) has been actuated confirming the operating action, and / or or that no actuation of an operating element of the vehicle (110) that contradicts the operating action has taken place. Method (200) according to Claim 4 wherein - it is determined whether a confirming actuation and / or no contradicting actuation of an operating element of the vehicle (110) has taken place within a predetermined time interval before and / or after the initiation of the operating action; and - an operation of the vehicle (110) as a function of the operator action initiated on the mobile user device (120) is prevented if it is determined that within the predetermined time interval before and / or after the initiation of the operator action no confirming operation and / or a contradicting actuation of an operating element of the vehicle (110) has taken place; and / or - the vehicle (110) is only operated as a function of the operating action initiated on the mobile user device (120) if it is determined that an affirmative actuation and / or confirmation within the predetermined time interval before and / or after the operating action is initiated / or no repeated actuation of an operating element of the vehicle (110) has taken place. Method (200) according to one of the Claims 4 to 5 wherein - the operating element of the vehicle (110) to be actuated for an operating action is different for different operating actions; and / or - the operating element of the vehicle (110) for an operating action that relates to and / or influences longitudinal and / or lateral guidance of the vehicle (110) is a control means for longitudinal and / or lateral guidance of the vehicle (110). Method (200) according to one of the Claims 4 to 6th wherein - the control element of the vehicle (110) comprises a control means for longitudinal and / or lateral guidance of the vehicle (110); - The vehicle (110) is set up to decouple the control means for longitudinal and / or lateral guidance of the vehicle (110) from a longitudinal and / or lateral guidance of the vehicle (110); and - a confirming and / or contradicting actuation of the control means for longitudinal and / or lateral guidance of the vehicle (110) is detected while the control means is decoupled from the longitudinal and / or lateral guidance of the vehicle (110). Method (200) according to one of the preceding claims, wherein - The required operating action is directed towards setting and / or tracking a quantitative variable of the vehicle (110); and - The method (200) comprises causing a plurality of selectable options for respectively different discrete values for the quantitative variable to be output on the mobile user device (120). The method (200) according to any one of the preceding claims, wherein the method (200) comprises, - Detecting a driving situation and / or determining a need for action during operation of the at least partially automated vehicle (110); - Transmission of information on the detected driving situation and / or on the determined need for action to the mobile user device (120); and Determination of at least one operating action in relation to the driving situation and / or in relation to the need for action on the basis of the transmitted information. The method (200) according to any one of the preceding claims, wherein the method (200) comprises, - Determination of authentication information in relation to an authentication of a user on the mobile user device (120); and - Checking the validity of an operating action initiated by the mobile user device (120) on the basis of the authentication information. The method (200) according to any one of the preceding claims, wherein the method (200) comprises, - Checking whether a user of the mobile user device (120) has caused a decision on an operator action to be made by an automatic system; and - causing the automatic system to make a decision regarding the operator action on the condition of the vehicle (110). Method (200) according to one of the preceding claims, wherein - the vehicle (110) is an at least partially automated drivable or moving vehicle (110); and or - The vehicle (110) is operated in an at least partially automated driving mode and / or with a certain degree of automation. Computer program, in particular a computer program product, comprising a computer program, wherein the computer program is designed, when it is executed on at least one computing unit, one or more steps of the method (200) according to at least one of the Claims 1 to 12 execute. Device (111) for a vehicle (110), the device (111) being set up - to detect that a mobile user device (120) of an occupant of the vehicle (110) is located in an interior of the vehicle (110); - to determine that at least one operating action is required for the condition of the vehicle (110); and - to cause an operating element to be provided for executing the at least one operating action on the mobile user device (120). Mobile user device (120) which is set up to receive from a device (111) of a vehicle (110) information relating to a required operating action for the condition of the vehicle (110); to cause an operating element to be provided for performing the at least one operating action on the mobile user device (120); and - depending on an actuation of the control element, to send a control signal relating to a required control action to the vehicle (110).

Images