DE102022106094A1 - Method, device and computer program for assisting an occupant of a vehicle in operating the vehicle - Google Patents

Abstract

Method, device and computer program for supporting an occupant of a vehicle (F10, F11, F12) when operating the vehicle (F10, F11, F12). The method (100) comprises: - determining (101) the first data characterizing one or more first operating processes of a first mobile user device (AG10, AG11, AG12); - detecting (102) a second operating process of a vehicle (F10, F11, F12 ); wherein the second operation has at least some degree of similarity to the one or more first operations; and then: - Executing (103) a measure in relation to the detected second operating operation of a vehicle (F10, F11, F12).

Description

The invention relates to a method, device and a computer program for supporting an occupant of a vehicle when operating the vehicle. The invention further relates to a corresponding system consisting of several mobile user devices and at least one vehicle.

Occupants (e.g. drivers or passengers) of a vehicle often tend to make certain, sometimes person-specific errors that occur particularly in certain contexts under certain conditions. Some errors may therefore be repeated several times or frequently. Most of the time, these cannot be derived or predicted through analytical connections because the underlying perceptual, cognitive and psychological connections can be complex.

The present document deals with the technical task of supporting an occupant of a vehicle in operating a vehicle, in particular an information system and/or communication system, and/or the display operating system of a vehicle. In particular, the technical task includes a technical solution with which an improved operation of a vehicle system, in particular an information system and/or communication system, and/or the display operating system of a vehicle is made possible or the occupant of the vehicle is supported in doing so.

The task is solved by each of the independent claims. Advantageous embodiments are described, among other things, in the dependent claims. It should be noted that additional features of a patent claim dependent on an independent patent claim can form a separate invention independent of the combination of all the features of the independent patent claim, without the features of the independent patent claim or only in combination with a subset of the features of the independent patent claim can be made the subject of an independent claim, a division application or a subsequent application. This applies equally to technical teachings described in the description, which may constitute an invention independent of the features of the independent patent claims.

According to a first aspect, a method for supporting an occupant of a vehicle when operating the vehicle, in particular a vehicle system. The method includes: determining the first data characterizing one or more first operations of a first mobile user device; and: detecting a second operation of a vehicle, the second operation having at least some degree of similarity to the one or more first operations; and then (e.g. in response thereto, immediately or at a later time interval): carrying out a measure in relation to the detected second operating operation of a vehicle.

The term “operating process” refers in particular to an operation that deviates from the (classic) driving activity, in particular an operation of the vehicle, in particular of a vehicle system (e.g. an information system, communication system,) that is superimposed on the driving activity or is carried out at least partially simultaneously with the driving activity. as well as the display and operating system of a vehicle).

• - Daten eines oder mehrerer Bedienelemente (z.B. rohe oder zumindest teilweite verarbeitete Sensordaten) im Zusammenhang mit dem Bedienvorgang;
• - Daten zum Zustand des j eweiligen Nutzers;
• - Daten zu einer oder mehreren Handlungen des Nutzers, wobei die Handlungen der Vorbereitung oder Ausführung des Bedienvorgangs dienen;
• - Daten einer oder mehreren Tätigkeiten, die der Nutzer neben dem Bedienvorgang, in einem zeitlichen und/oder kausalen Zusammenhang mit dem Bedienvorgang ausgeführt hat.

The first data are, for example, in a temporal and/or causal connection with one or more operating processes, in particular with (eg certain user errors or user errors of a certain type) in one or more first operating processes. For example, it is the first data from a time interval (e.g. of approximately 10, 20, 30 seconds) before the user error, and/or from a time interval in which the user error occurs (e.g. during the user error), and/or in a Time interval (e.g. of approx. 10, 20, 30 seconds) after the user error. The first data may include or be:

• - Data from one or more operating elements (e.g. raw or at least partially processed sensor data) in connection with the operating process;
• - Data on the status of the respective user;
• - Data on one or more actions by the user, whereby the actions serve to prepare or carry out the operating process;
• - Data from one or more activities that the user has carried out in addition to the operating process, in a temporal and/or causal connection to the operating process.

The first data can in particular (also) be data that is determined dependently, in particular on the basis of the first data.

For example, the user device can be a device that can be worn (by the first person or second person), for example a smartphone, tablet, an electronic personal assistant, one that is attached to the body of the (respective) user or that is portable or at least partially connected to the body of the respective user The user device can be connected to the user, for example a so-called smart watch or smart close. In a portable embodiment, (the first and/or the second) user device may have a weight of less than 1 or 0.5 kg.

For example, the user device can be a substantially stationary user device, e.g. a flat screen device or TV device, a further development or a combination of one or more such devices. The user device can, for example, be a development of a stationary or portable flat screen device, in particular with a sensor, in particular a camera, aimed at the user.

Alternatively or additionally, the user device may be or include (or be understood as a user account) as a user account (also: account, user account, user account or personalized access) of the respective user. Alternatively or additionally, the user device can be a virtual user device or a (e.g. web-based) application. The method or the effect of the method can be at least partially independent of the user device (as hardware) that is (currently) used by the respective user.

The user device can be a user device that can be operated or operated (e.g. in a vehicle that differs from the vehicle). However, it can be a user device operated under any other circumstances or context. The method can be carried out particularly effectively because a measure of similarity between the respective operating processes in the vehicle and on the user device is taken into account, in particular at least the existence of a certain measure of similarity is recognized.

For example, it can be determined and taken into account whether or not the one or more operations of the (respective) mobile user device took place when the user device was operated in a vehicle.

The term “operating error” by an occupant of the (respective) vehicle is, in particular, an incorrect information input into a vehicle system that does not lead to the desired result, an information output and/or action or interaction with a display operating system (in particular also to be understood as the human-machine interface) of a vehicle.

In particular, the one or more operating errors are those that are caused by the occupant of the vehicle, in particular by perceptual, cognitive and/or psychological effects.

Determining a user error during one or more first operating processes of a first mobile user device can include determining various, for example qualitative and/or quantitative, parameters of the user error. These can be taken into account in the process (e.g. when selecting the measure, determining or adjusting the parameters of the measure, etc.). For example, if the user of the respective mobile user device made a typing error when entering information or operated a different control element instead of or next to the desired control element, one or more parameters can be the first data that is in a (e.g. temporal and/or causal) context stand, determined and taken into account.

In particular, the first operating process relates to a system or functionality of the mobile user device that provides information output (for perception by the occupant), information input (e.g. offer to enter the information, e.g. typing, selection of options, by the occupant). For example, the corresponding functionality of the mobile user device can be a navigation system, an infotainment or a communication system (e.g. a corresponding app). Such processes are also subsumed under the term “operation of the mobile user device”.

In particular, the second operating process of the at least one vehicle relates to a vehicle system or functionality of the vehicle. This can be a similar or different functionality than the first operation of the mobile user device.

In the context of this document, the term “recognizing” is also understood to mean, in particular, predication. Determination can in particular mean that a (second) operating procedure is predicted with respect to the future (e.g. with a certain degree of probability and/or for a certain time interval). Such recognition or prediction can relate to an operating action that has only been partially implemented (e.g. only to a small extent) or has not yet been started. For example, a measure can be carried out before or at an early stage of executing the operating process (and can therefore be particularly effective).

The determination, in particular prediction, can be based on, for example, a determined intention of the occupant to carry out a (certain) operating procedure and/or various other signs, for example para meter patterns that often occur shortly before a (certain) operation or an operation (of a certain type).

For example, only or primarily the first operating processes that have been recognized, in particular predicted, with at least a certain degree of probability are taken into account.

The (second) occupant error can be an occupant error that has not yet been recognized or is recognizable and/or is taking place or will possibly take place in a second operating process that has not yet taken place (but, for example, has only been predicted) or is not sufficiently advanced.

The second operation may differ from the first operation. For example, it can be a related and/or at least 30, 50, 70% identical (but not exactly the same) operating process as one or more first operating processes of one (or more) mobile user devices. For example, the determined or expected second operating process can be similar or the same as the one or more first operating processes with regard to at least 30, 50, 70% of the features. This increases the number of second operating procedures in which the occupants can be supported as part of the procedure.

Recognizing that the second (recognized, in particular predicted) second operating operation of a vehicle has at least a certain degree of similarity to one or more first operating operations plays a very important role in the method.

For example, by taking into account the similarity, in particular a finely graded similarity measure, it can be achieved that no or no inappropriate (e.g. counterproductive or error-causing) measure is carried out in the vehicle due to a different, dissimilar first operating process on a mobile user device.

Executing the at least one measure can include or be a selection of one (of several possible) measures and/or determining or adjusting the parameters of the measure and/or initiating the measure. A measure can include or be a (e.g. appropriate) change in the operating mode of a vehicle system, in particular an information system, communication system and display operating system of a vehicle.

Depending on the similarity of a second operating process with different first operating processes (e.g. variants of the operating actions), a selection of the measure, determining or adjusting a parameter of the measure, and / or reducing a measure, in particular with regard to its intensity or deactivating a measure (e.g. such , which is normally active or should be activated according to the occupant's settings). This can, for example, ensure that no disadvantages arise from less effective or at least partially counterproductive measures.

Depending on the degree of similarity between the second operating process and one or more different first operating processes, for example, measures of different types and/or measures with different or differently adapted parameters can be carried out.

The measure may be a proactive measure. The measure can be a measure specific to the specific occupant, occupants with a specific personal profile, and/or to a specific type or variant of the second operating process and/or to the specific (respective, specific operating process). Different (each appropriate, expedient) measures can therefore be carried out for operating operations of the same type, which are carried out or (initially only) intended by a second driver and a second passenger.

Several measures, in particular an appropriately selected combination and/or sequence of measures, can be carried out.

The execution of the at least one measure can be carried out in a predetermined manner, in particular according to a predetermined mathematical relationship, from the information determined in the method. For example, the selection of the measure, determining or adjusting a parameter of the measure, a reduction or deactivation of a measure can take place in a dependence on one or more pieces of information determined in the method, for example characterized by the mathematical context.

Such a mathematical connection can characterize a mathematical function, law, rule, regulation, formula and/or relation. This can, for example, be characterized, in particular represented, by parameters and/or coefficients of a predetermined (computer-implemented) function. For example, the mathematical connection, ins in particular depending on a large number of the determined (also to be understood as: collected) first data, for example determined on the basis of a large number of mobile user devices (e.g. in a backend) and then transmitted to the respective vehicle (directly or indirectly) and used in it.

The measure can be, for example, a measure to avoid a possible, in particular expected (second) occupant error and/or to reduce a consequence of a, in particular expected (second) occupant error in the at least one vehicle.

Particularly preferably, the measure can relate to a specific second operating process or a second operating process of a specific type. The measure can also relate to the operating process of a specific occupant (also referred to as the second occupant) or an occupant from a specific group of people.

For example, for certain or specific second operating processes, one (specific) measure is carried out. Particularly preferably, it is a specific measure determined for the specific operating process, e.g. determined or adapted for the specific operating process. There can be measures tailored to the identified or predicted operating processes. The process can be carried out dynamically. The effect of the process may therefore differ from or go significantly beyond a general improvement of the vehicles.

A user error (e.g. during one or more first operations) can be a common error and/or one that is typical for the specific person. A user error can be recognized, for example, by the fact that the user of the user device has pressed the undo button one or more times in a menu item (e.g. for an operation in a menu) without the user having previously pressed a button (e.g. in the operating menu). . (recognized as meaningful) caused or carried out an action. The user error can possibly also be recognized by a subsequent action (which is obviously unsuccessful or does not correspond to the occupant's expectations).

A user error during a (first) operating process can, for example, be a specific operating pattern or a sequence pattern of actions by the user of the user device (as part of an operating process) that is subsequently assessed as “undesirable”. Such patterns can, if necessary, be recognized and taken into account in relation to a specific user or in relation to a group of people to which the user belongs (also understood as assigned).

In particular, the user of the vehicle and the user of the user device can be the same person. One or more first operations may be (or may have been) operations of the user device at different (e.g. multiple and/or earlier) time intervals than the at least one second operation of the vehicle. The information about whether it is the same person can be provided, for example, by identifying the person and/or using the same user account or at least two linked accounts (e.g. on the user device and in the vehicle). This information can be taken into account in the process (e.g. in the mathematical context).

In particular, the user of the user device is a person other than the occupant of the at least one vehicle. For example, the first data can be determined and used by a (large) number of users of the user devices. According to the invention, these can be aggregated (e.g. in a backend). This allows the amount of usable data to be significantly increased and the precision of the measure to be increased. In addition, the data on operating processes from user devices (at the latest in the desired quantity) is much easier to determine than from vehicles, for example. The occupant of the vehicle can preferably be a person with a similar personal profile or a person who belongs to the same group of people. The procedure can also be carried out without identifying people (which is undesirable for many vehicle owners).

As the occupant (also and equally the occupant), in particular as a driver, passenger or user of a (possibly specific) vehicle, e.g. a user who regularly uses the vehicle or is recognized as such and/or (e.g. by a Vehicle access system, key, etc.) is logged in, can be understood.

Preferably, the at least one user error can be a typically human error, for example a careless error, an attention error, a memory error, a skill error, perception error, etc. The user error can be unintentional, erroneous, different than desired and/or in the expectation of another The result is the action carried out by the user of the user device. In the context of this document, an “incorrect operating procedure” is collectively referred to as an operating procedure that whose result does not correspond to the underlying motivation or expectation. This can be a technically correct or flawlessly recorded or executed process. The second (eg initially only potential or expected) occupant error can be of the same type as the first user error of the one or more users of the one or more user devices.

The method preferably includes recognizing the one or more user errors (in the one or more operating processes). These can be recognized as specific operating errors and/or operating errors of a specific type and/or operating errors corresponding to a specific pattern. In particular, the method at least primarily includes the detection of human or human-specific errors. These may be recognizable as such and taken into account (in particular in deviation from any other effects or defects). These can be (operating) errors that are specific or typical for human nature or weaknesses (e.g. in contrast or comparison to technical errors, defects or weaknesses).

Preferably, for a specific (already taking place or predicted) second operating process or a second operating process of a specific type, a specific measure can be determined and, in particular, implemented in a forward-looking manner. The measure can support the at least one (second) occupant of the at least second vehicle in that the, in particular certain (second) occupant errors and/or occupant errors of a specific type can be at least partially avoided or their consequences reduced.

Preferably, the method can be carried out entirely or predominantly automatically, dynamically, and/or in a person-specific or group-specific manner.

Executing a measure can include selecting a measure, determining or adjusting one or more parameters of the measure, initiating and/or directly or indirectly effecting the measure, e.g. with appropriately equipped means of the respective vehicle. Means of the vehicle set up in this way form a further aspect of the invention.

The method can be made executable in such a way that a (e.g. appropriate) measure is determined and initiated even before the occupant of the second vehicle makes or can possibly make a (certain) passenger error. A measure can already be determined and, in particular, initiated before a (certain) second operating operation by the occupant begins or a (certain) occupant error can occur.

In particular, it can be recognized (in particular predicted) that a second operating operation is being carried out (or will be carried out) on the vehicle, which is sufficiently similar to at least a previous first operating operation, at least partially of one or more, in particular mutually similar, subject to user errors , in particular has at least a certain degree of similarity to this one or these.

The method particularly preferably includes a comparison of a large number of the first operating processes and/or first user errors with one another. The degree of similarity between these, including one another, can be determined and taken into account. An accumulation of (mutually) similar groups of operating processes and (mutually) similar occupant errors (e.g. compared to an average value or a statistical value) can be recognized and the corresponding frequency information can be taken into account.

If (particularly) occupant errors of the occupants of the at least one vehicle have at least a certain (possibly high) degree of similarity with the user errors of the (respective) users of the (respective) user device, and in particular are recognized during (particularly) similar operating processes, a , in particular to the group of the first similar errors in the first similar operating procedures, for this specific or adapted measure (i.e. also for this specific and/or tailored to this) can be effected.

Each of the similarity measures described in this document (also to be understood as “degree of similarity and/or affinity”) can preferably be an objective one, assigned according to certain, predetermined or adaptively selected criteria (e.g. by means of an appropriately set up pattern recognition), in particular quantifiable, measure of action. Qualitative and/or quantitative parameters or patterns of operating processes or occupant errors can also be compared or evaluated according to a specific criterion.

A similarity within the meaning of this document can also take into account, include or be a dissimilarity. For example, said similarity measure can alternatively or additionally take into account a dissimilarity measure.

For example, the presence of at least a certain level of similarity (e.g. a minimum level of similarity) can also be understood as a dissimilarity lying below a (certain) threshold value (e.g. as a maximum level of dissimilarity).

For example, “at least a certain level of similarity” is to be understood as a similarity that exceeds a certain threshold and/or as a dissimilarity that falls below a certain threshold.

In an exemplary case, a dissimilarity measure can be determined (e.g. using a negative classifier) or recognized by applying a dissimilarity criterion if the dissimilarity is below a certain threshold value. This can also be taken into account as a reciprocal variable of similarity, for example by forming a reciprocal variable. For example, a measure, a specific parameter of the measure or a proactive measure can be implemented primarily or only if no excessive dissimilarity in the operating processes is detected.

A similarity measure or a dissimilarity measure can (each) be evaluated on the basis of several (including different, multi-dimensional) criteria. For example, this can be done by comparing amplitudes, patterns (e.g. the operations to be compared), and/or the time functions according to which different parameters change.

In particular, a similarity measure and/or a dissimilarity measure can be used, for example in relation to operating processes (various parameters or patterns of operating processes), in relation to occupant errors (various parameters or patterns of occupant errors), and/or in relation to occupants (various parameters or patterns relating to the person of the occupant) are identified and taken into account.

The similarity measure and dissimilarity measure can also be determined or taken into account using different procedures (which can be viewed as part of the procedure). For example, using any method or a predetermined criterion, e.g. similarity criterion and/or dissimilarity criterion, it can be recognized or a conclusion drawn that at least a certain (sufficient) degree of similarity and/or at most a certain (e.g. maximum permissible) degree of dissimilarity is present. This can also be done without determining a (e.g. explicit) similarity measure or dissimilarity measure.

For different similarity measures (e.g. when detecting different similarity measures between operating processes, occupant errors, occupant personal profiles, etc.), different measures or measures with different parameters are preferred, e.g. measures of different types, measures at different times and/or with different intensities, effects.

For example, a second operation can be recognized that has a particularly high level of similarity (or a particularly low level of dissimilarity) with a large number of first operations, each with a particularly high level of similarity (or a particularly low level of dissimilarity) with occupant errors recognized, especially predicted. In response to this, in particular depending on the information determined, when executing the measure it can be assumed with particular certainty that a measure is necessary and/or a particular measure (most likely, actually) will be the (at least comparatively well, particularly well) appropriate measure becomes. The risk of unnecessary, disruptive or even dangerous measures can be reduced. This can also be done in advance. The selection of the measure and/or the determination or adjustment of the parameters of the measure can be carried out particularly precisely.

The corresponding data from a (large) number of the first operating processes on one or (in each case) several user devices are preferably used. The device (also to be understood as the system) can therefore include many, for example thousands, millions of user devices or data from them.

Alternatively or additionally, one or more parameters of the second operating process can be determined. One or more similarity measures between the parameters can (also) be determined and taken into account. For example, one or more patterns of the parameters of the second operating process can be determined and at least one measure of similarity of the one or more determined patterns with at least one or more parameters of a first operating process can be determined and taken into account. For example, a combination and/or a pattern of parameters can be determined and taken into account.

The (e.g. initially only possible) occupant error can be an occupant error when using and/or operating the vehicle and/or when interacting with the vehicle (also ver stand with a vehicle functionality, for example a navigation system).

An operating process can include or be the actuation of a control element (such as a button, rocker, a multi-dimensionally movable control or input element, a rotary knob, a slider, etc.).

Preferably, the operating process can include or be an input using a sensitive screen (e.g. a touchscreen or a non-contact interactive screen), using gestures, e.g. swiping gestures, finger gestures or gesture recognition, however configured.

An operating process can include or be the selection or operation of a menu item in a user menu of the vehicle, e.g. typing, setting or fine-tuning values, operating a virtual controller.

An operating process can also include an, in particular non-contact, action or interaction of the occupant with the operating interface of the vehicle.

The (first and/or second) operating process preferably includes one or more fine motor actions and/or actions of the occupant that can be carried out with individual fingers. Such operating procedures can typically involve multiple or different user errors or occupant errors, for example in comparison to pressing a button.

For the at least one mobile user device, the first operating process can correspond to reading in or recognizing a specific operating process or an input and/or a result desired by the user.

The method may include determining one or more first parameters related to the first operating process, and determining one or more second parameters related to the second operating process. These can be taken into account in the process. The one or more parameters can describe the respective operating process (e.g. the control element that is activated; the menu item that was selected as part of the operating process, etc.).

Particularly preferably, one or more parameters that are characteristic of a sequence of the operating process are determined and taken into account. One or more such parameters, in particular a change characteristic of one or more parameter values, e.g. characterizing an interaction with the operating interface, can be determined and taken into account over time.

A similarity measure (also to be understood as a value of a similarity indicator) for the similarity of the first and second operating processes can then be determined based on the one or more first parameters and based on the one or more second parameters. Furthermore, based on the determined similarity measure, it can be determined whether the second operating process has a (certain, sufficient) similarity, in particular a similarity measure, to the first operating process or not (and if necessary or the second operating process is similar to the first operating process, for example in terms of its type and/or or parameter). In particular, a second operating process with a possible occupant error can be recognized (in particular predicted) in a precise and reliable manner. The method may further include, in response to the detected second operation (which is similar to or corresponds to the first operation according to the similarity measure), executing at least one action with respect to a possible occupant error in the second operation. In particular, bringing about the measure can include determining, in particular selecting, determining or adapting the measure (e.g. several variants, characteristics, methods, intensities, etc.). In other words, it can be a special and/or dynamically determined measure (tailored to a second occupant, so to speak).

The measure can be aimed at changing the operating logic and/or a parameter of the user interface of the vehicle at least temporarily or only temporarily. Alternatively or additionally, the measure can be aimed at avoiding the possible occupant error during the second operating process and/or an effect of the possible occupant error on the vehicle. Alternatively or additionally, the measure can include an output to the occupant of the vehicle, the output being, for example, a suggestion for changing the behavior of the vehicle (in particular also to be understood as the behavior of a system or a functionality of the vehicle), in particular the behavior of the information input, Information output, user interface of the vehicle (where the change can be made or confirmed, for example, by the occupant of the vehicle).

The method thus makes it possible to reliably detect (in particular to predict) a possible occupant error when operating a vehicle and to avoid and/or at least partially (at least in relation to a possible consequence).

The method can be executable (also: parameterized, optimized) with the objective or can be carried out in such a way that (at least statistically) the probability of operating errors and/or the consequences thereof decreases. In doing so or as a result, the comfort, effectiveness, reliability and/or success in operating, in particular in operating, the one or more second vehicles can be increased.

In particular, the method and/or the (corresponding) device can be aligned and/or optimized and/or controlled or regulated so that comfort, effectiveness, reliability and/or success in operating a vehicle, at least in a statistical view, one or more objective ones or to increase objective measures or change them towards a target value, in particular towards a maximum. Success is to be understood in particular as an improved correspondence between a result of the operating process and the motivation and/or expectations of the occupant. In particular, the method can be carried out in such a way that a qualitative and/or quantitative key figure (simplified: number and/or severity) of further occupant errors is reduced. For example, in the method for this purpose, appropriate processing of the data can be provided or the device can be set up accordingly.

According to a further embodiment, the first data is indicative of: a user error when operating the mobile user device; and/or for one or more information about the settings and/or operating logic and/or technical parameters of the respective mobile user device.

According to a further embodiment, the first data is characteristic of: one or more first measures carried out to prevent and/or correct one or more user errors during one or more first operating processes of one or more mobile user devices; and/or for: a result, in particular a measure of the success, prevention and/or correction of one or more user errors in one or more first operating processes of one or more mobile user devices.

According to a further embodiment, depending on, in particular on the basis of, the first data, determination of: training data for training a neural network to carry out a measure in relation to the recognized or predicted second operating operation of a vehicle; and/or learning data of a neural network for executing a measure in relation to the recognized or predicted second operating operation of a vehicle;

According to a further embodiment, the method includes determining first user information relating to a user of the user device, in particular characterizing a personality profile of the user and/or a group of people to which the user of the user device belongs; and/or determining second occupant information relating to an occupant of the vehicle, in particular characterizing a personality profile of the occupant of the vehicle and/or a group of people to which the occupant of the vehicle (F10, F11, F12) belongs; and: taking the first user information and/or the second occupant information into account in the method.

According to a further embodiment, the method comprises determining a similarity measure between the personality profile of the user of the user device and the personality profile of the occupant of the vehicle; and/or determining a similarity measure between the group of people to which the user of the user device belongs and the group of people to which the occupant of the vehicle belongs; and: taking at least one similarity measure into account in the process.

According to a further embodiment, the method includes recognizing the second operating process; and/or executing the measure in relation to the detected second operating operation of a vehicle. This can be carried out: depending on the first user information of the user of the user device; and or
depending on the second occupant information relating to an occupant of the vehicle; and or
depending on the degree of similarity between the personality profile of the
user of the user device and the personality profile of the occupant of the vehicle;
and or
depending on the degree of similarity between the group of people to which the user of the user device belongs and the group of people to which the occupant of the vehicle belongs.

• abhängig von der ersten Nutzerinformation des Nutzers des Anwendergeräts; und/oder
• abhängig von der zweiten Insasseninformation in Bezug auf einen Insassen des Fahrzeugs; und/oder
• abhängig von dem Ähnlichkeitsmaß zwischen dem Persönlichkeitsprofil des Nutzers des Anwendergeräts und dem Persönlichkeitsprofils des Insassen des Fahrzeugs;
• und/oder
• abhängig von dem Ähnlichkeitsmaß zwischen der Personengruppe zu der der Nutzer des Anwendergeräts angehört und der Personengruppe zu der der Insassen des Fahrzeugs angehört.

According to a further embodiment, the method includes determining training data for training a neural network, wherein the neural network is used to carry out a measure in relation to the recognized or predicted second operating operation of a vehicle. This can be done:

• depending on the first user information of the user of the user device; and or
• depending on the second occupant information relating to an occupant of the vehicle; and or
• depending on the degree of similarity between the personality profile of the user of the user device and the personality profile of the occupant of the vehicle;
• and or
• depending on the degree of similarity between the group of people to which the user of the user device belongs and the group of people to which the occupant of the vehicle belongs.

According to a further embodiment, the method includes determining an input from an occupant of the vehicle regarding a measure to be carried out, in particular an association between a specific operating error or an operating error of a specific type and the measure then to be carried out or a type of measure to be carried out; and taking an action dependent on the occupant's input.

According to a further embodiment, the measure is aimed at changing an operating logic and/or a parameter of a user interface of the vehicle, in particular temporarily and/or in a passenger-specific manner. Alternatively or additionally, the measure includes a, in particular temporary and/or occupant-specific, change to an operating logic and/or a parameter of the operating interface of the vehicle.

According to a further embodiment, the method includes generating a suggestion for changing a behavior of the vehicle, in particular the behavior of an information output device, an information input device and/or a user interface of the vehicle. This can be done by changing the operating mode of one or more vehicle systems or the corresponding functionalities.

According to a further embodiment, the one or more user devices are viewed as the one or more vehicles; and: the one or more vehicles are considered one or more user devices. The method can include or be supporting the user of the one or more user devices by carrying out the at least one measure in relation to the operating process of the one or more user devices. In other words, the process can be carried out (only, depending on a predetermined condition or alternately) in the other direction, so to speak. One or more of the functional principles described in this document can be applied accordingly. For example, the first data can (also) be provided by the vehicles.

The at least one measure may include: changing an information output channel in which information is output to an occupant of a vehicle. The information channel can be changed (e.g. only temporarily and/or occupant-specific or person-specific) (e.g. depending on the determined probability and/or consequence of an operating error during the second operating process). The change can take place, for example, from or towards a visual, auditory, haptic and/or kinesthetic information channel.

Alternatively or additionally, the at least one measure can include: changing an information input channel that is activated to an occupant of the second vehicle for operating input or information input, in particular is offered for inputting information; and/or changing the information input channel via which an input from the occupant of the vehicle is at least primarily processed, in particular towards a voice input, gesture control, touchscreen and/or physically movable control element.

Alternatively or additionally, the at least one measure can include: changing a parameter of an output and/or an input option in connection with the respective operating process.

There can be a scope of the output; and/or a display type and/or level of detail of the output; and/or a position of the output that can be perceived by the occupant can be changed. For example, a contrast, symbol, and/or font and/or size, and/or position and/or displacement of virtual controls can be changed.

The method may include determining one or more context parameters in relation to a boundary condition and/or in relation to a context of the one or more user errors in the one or more first operations. In a corresponding manner, one or more context parameters can be determined and taken into account in relation to a boundary condition and/or in relation to a context of the second operating process of the at least one vehicle.

In particular, the at least one (eg first) context parameter identifies a further context parameter for operating the mobile user device indicate a different activity by the user (e.g. an activity carried out by the user at approximately the same time) and/or a disruption and/or distraction of the user.

Alternatively or additionally, a (e.g. second) context parameter can be determined and taken into account in relation to the occupant of the vehicle. This can be a further activity by the occupant that deviates from operating the vehicle system (e.g. an activity carried out by the occupant at approximately the same time, e.g. driving activity, actions or actions to control the vehicle), and/or a disruption and/or distraction of the occupant of the vehicle Mark the vehicle. Furthermore, processing of the first context parameter and the second context parameter, in particular a comparison of the corresponding context parameters and/or a determination of a similarity measure between the context parameters, can be carried out and the result of the processing can be taken into account in the method.

The second operating procedure (in which a further occupant error will possibly occur) can then be recognized (in particular predicted) with increased reliability depending on the one or more driving situation parameters and/or context parameters. Alternatively or additionally, the measure can be determined and/or effected in relation to the possible occupant error during the second operating process depending on the one or more context parameters. The current and/or previous and/or predicted driving situation for the near future can be taken into account.

In the context of this document, the term “driving situation” can be understood, for example, as a specific situation characterized by an arrangement, action, or interaction of road users or by certain driving parameters of road users.

In particular, the meaning of the term “situation” or “driving situation” differs from the meaning of the term “traffic situation” that is often used colloquially, which rather summarizes general categories such as “free traffic”, “dense traffic”, “slow-moving traffic”, “ Traffic jam”, “end of traffic jam” etc. corresponds.

In particular, a driving situation is (rather) characterized by a specific pattern (also to be understood as a data pattern), e.g. a pattern characterizing an arrangement and/or speed of the objects and/or a pattern of the parameters of the driving situation. The driving situation can also be characterized by a spatial pattern of so-called open spaces in the area surrounding the vehicle. For example, a trigger condition can relate to such features or corresponding parameters of the driving situation.

• - eine (bestimmte) räumliche Verteilung der Verkehrsteilnehmer und/oder der Bewegungsparameter der Verkehrsteilnehmer, insbesondere ein Anordnungsmuster der Verkehrsteilnehmer in der Umgebung des (realen) Fahrzeugs,
• - eine (bestimmte) räumliche Verteilung unbeweglicher Objekte in der Umgebung des Fahrzeugs,
• - eine relative Position und/oder Bewegungsparameter zu bestimmten Arten von Spurmarkierungen, Verkehrszeichen, Ampeln (nicht notwendigerweise zu bestimmten Ampeln, etc.),
• - eine Information über die Vorfahrt des Fahrzeugs, insbesondere gegenüber bestimmten Verkehrsteilnehmern und/oder Verkehrsteilnehmern, die tatsächlich oder zumindest potentiell aus bestimmten Richtungen, z.B. einer querenden Straße von rechts oder von links kommen oder kommen können,
• - eine Information zu einer, z.B. einen Grenzwert überschreitenden, Handlung eines Verkehrsteilnehmers in der Umgebung des Fahrzeugs, wie z.B. ein Hupen, Lichthupen, Drängeln, Überholen des Fahrzeugs, ein Überholversuch und dergleichen.

The at least one driving situation can preferably be characterized by one or more features listed below:

• - a (certain) spatial distribution of the road users and/or the movement parameters of the road users, in particular an arrangement pattern of the road users in the area surrounding the (real) vehicle,
• - a (certain) spatial distribution of immovable objects in the area surrounding the vehicle,
• - a relative position and/or movement parameter to certain types of lane markings, traffic signs, traffic lights (not necessarily to specific traffic lights, etc.),
• - information about the right of way of the vehicle, in particular towards certain road users and/or road users who are actually or at least potentially coming from certain directions, e.g. a crossing street from the right or from the left,
• - Information about an action by a road user in the vicinity of the vehicle, for example exceeding a limit value, such as honking, flashing lights, jostling, overtaking the vehicle, attempting to overtake and the like.

Furthermore, the driving situation can be characterized by one or more parameters in connection with relevant traffic rules, traffic signs, right-of-way, traffic lights and/or traffic light phases.

The at least one driving situation can preferably be a driving situation that exceeds certain limit values or a driving situation that is characterized by parameters that exceed certain limit values. For example, the at least one driving situation can be a driving situation with an undesirable or dangerous approach to an object or a road user, an acceleration value that exceeds a limit value, an undesirable arrangement with other road users, etc.

The driving situation can be a special situation (e.g. that occurs comparatively rarely) or a dangerous driving situation dles, e.g. a driving situation for which an increased risk is determined or assumed.

The at least one context parameter can include or be a context for the perception (or perception context) of the occupant. For example, a context parameter can be characteristic, in particular representative, of lighting conditions, in particular a strength and/or direction of the lighting, changes in the lighting over time (e.g. driving along avenues, bridges, winding streets, etc.) and/or noise conditions, in particular noise other vehicles, noise from one or more other occupants (e.g. certain voices), radio, etc.

Thus (e.g. an effective and targeted) measure can be carried out to prevent a second operating error that can arise due to a temporarily disturbed or slightly changed perception of the information and/or a consequence of such a second operating error.

The method may include determining at least one consequence parameter in relation to a consequence and/or an effect of the possible occupant error in the second operating procedure. Depending on the consequence parameter, a measure can then be determined or effected in relation to the possible occupant error during the second operating process. In this way, the reliability of the operation of the vehicle can be further increased.

As part of the method, data relating to a plurality (e.g. hundreds, thousands, millions) of first operating processes may have been determined (in advance or in advance). These can be determined using a backend server, stored and/or processed by it. The first operations can be carried out on one or more different (possibly identical) mobile user devices or apps. Based on the data, frequency information relating to the frequency of user errors in the plurality of first operations can be determined or has been determined. In particular, it can be determined or has already been determined (at the time interval at which a second occupant error can occur on the second vehicle) how frequent the user error, in particular a specific user error or a user error of a specific type, is during the first operating processes. The measure in relation to the possible occupant error during the second operation of the vehicle can then be effected depending on the determined frequency information.

For example, the selection, determination or adjustment of a parameter of the measure takes place depending on the frequency information. For example, no action can be taken if the frequency of the occupant error is less than a certain frequency threshold. On the other hand, a measure (e.g. a measure of a certain type and/or a measure with a certain parameter value range) can possibly only be implemented or primarily if the frequency of the occupant error is greater than the specific frequency threshold value. Especially by taking frequency information into account, the comfort and reliability of operating the vehicle can be further increased.

According to a further aspect of the invention, a neural network for supporting an occupant of a vehicle in operating the vehicle is proposed. The neural network is trained depending on the first data characterizing one or more first operating processes of a mobile user device. In this case, the neural network (e.g. controlled or supported by the neural network and/or dependent on the output layer of the neural network) can be used to carry out a measure in relation to a second operating process of a vehicle that has at least a certain degree of similarity to the one or more first ones Operations of a mobile user device.

According to a further aspect, a system for supporting an occupant of a vehicle when operating, in particular when operating, the vehicle is described. The system is set up: to determine the first data characterizing one or more first operations of a mobile user device; and set up: to recognize a second operation of a vehicle; wherein the second operation has at least some degree of similarity to the one or more first operations; and is further set up: to then carry out a measure in relation to the detected second operating operation of the vehicle.

Alternatively or additionally, the system can be set up to carry out the method according to one or more features described in this document. In particular, the system can (also) be set up to support the user of the mobile user device (as already described).

The system preferably includes one or more user devices, at least one vehicle and/or a backend (also to be understood as a cloud, a server, a portal). One or more user devices are preferred for immediate use ren or indirect active connection for carrying out the method with the at least one vehicle and / or the backend. The one or more first vehicles can be designed to determine parameters characterizing one or more (first) user errors, for example type of user errors, type of corresponding operating processes, etc.

Furthermore, the one or more user devices (in particular the corresponding software or apps) can be designed to transmit the corresponding data (e.g. when executing the active connection) to the backend (BE). The backend can be designed to process the data from a plurality of user devices, in particular to determine one or more frequency information (e.g. for several different groups of operations and user errors). One or more measures can be implemented (at least partially in the backend and/or at least partially in the vehicle) based on the processed data, in particular based on the frequency information.

In particular, the system can also include an operating system of the vehicles and/or data interfaces set up to carry out the method, for example hardware-related interfaces. The system, e.g. the majority of mobile user devices (e.g. the app), vehicles and/or (common or networked) backend can be operated by the same provider, operator, service provider and/or a standard or convention designed in accordance with the invention. The one or more apps and the one or more vehicles may be offered directly or indirectly by the same manufacturer or operator.

According to a further aspect, a computer program product comprising a computer program and/or data for executing the computer program is described, wherein the computer program product is designed to carry out the method described in this document when it is executed on at least one computing unit.

The computer program product can be designed as an update of a previous computer program, which includes the parts of the computer program or the corresponding program code, for example as part of a functional expansion, for example as part of a so-called “remote software update”. The computer program product includes in particular a medium readable by the data processing device, on which the program code is stored, or at least an encrypted file. According to a further aspect, a program product is described that includes an authorized access right to stored data of the computer program product.

It should be noted that the methods, devices and systems described in this document can be used both alone and in combination with other methods, devices and systems described in this document. Furthermore, any aspects of the methods, devices and systems described in this document can be combined with one another in a variety of ways. In particular, the features of the claims can be combined with one another in a variety of ways.

• 1 zeigt ein Beispiel für das System. Dieses umfasst mehrere Anwendergeräte AG10, AG11, AG12, ein Backend (BE) sowie zumindest ein zweites Fahrzeug F10, F11, F12.
• 2 zeigt ein Beispiel für das System. Dieses umfasst mehrere Anwendergeräte AG10, AG11, AG12, ein Backend (BE) sowie zumindest ein zweites Fahrzeug F10, F11, F12, wobei das Verfahren derart ausgeführt wird, dass ein Nutzer des Anwendergeräts beim Betreiben (Nutzen) des Anwendergeräts unterstütz wird. Bei dieser Variante des Systems bzw. des Verfahrens haben die Anwendergeräte und die Fahrzeuge jeweils sinngemäß vertauschte Rollen.
• 3 zeigt ein Ablaufdiagramm eines beispielhaften Verfahrens 100 zum Betreiben eines oder mehrerer Fahrzeuge F10, F11, F12, insbesondere eines System umfassend das ein oder mehrere Fahrzeuge F10, F11, F12.

In the following, the invention is described in more detail using exemplary embodiments without restricting the generality. All features of the exemplary embodiments can be combined with one another and/or with the features of the claims. Show:

• 1 shows an example of the system. This includes several user devices AG10, AG11, AG12, a backend (BE) and at least one second vehicle F10, F11, F12.
• 2 shows an example of the system. This includes several user devices AG10, AG11, AG12, a backend (BE) and at least one second vehicle F10, F11, F12, the method being carried out in such a way that a user of the user device is supported in operating (using) the user device. In this variant of the system or method, the user devices and the vehicles each have reversed roles.
• 3 shows a flowchart of an exemplary method 100 for operating one or more vehicles F10, F11, F12, in particular a system comprising the one or more vehicles F10, F11, F12.

The one or more user devices AG11, AG11, AG12 and/or vehicles F10, F11, F12 can (each) include a user interface or HMI (=human-machine interface). These can (each) be designed to carry out the method 100. Alternatively or additionally, the operating interface can be understood as an information input device, information output device.

Furthermore, the vehicle F10, F11, F12 comprises a (control) device which is set up to operate the vehicle F10, F11, F12 depending on the recognized operating process (e.g. the input or output of information, the playing of an audio signal, Etc.).

It can be recognized that a specific user of the user device and / or an occupant of the vehicle F10, F11, F12 who belongs to or is assigned to a group of people, in particular to a particular extent that overlaps a statistical extent, tends to make certain occupant errors, in particular in Referring to operating procedures of a certain type, in a certain context, etc. making (certain) operating errors, in particular repeating them.

For example, an operating error can consist of one or more users who are part of a group of people (often) forgetting to carry out a certain operating action on their vehicle (in a timely manner), not activating the intended option, or achieving an incorrect or undesirable selection or effect , or “click” or a certain operating action is carried out incorrectly (e.g. in the wrong direction).

The system described in this document can (also) be set up to recognize a user error as such. This can be recognized, for example, by the fact that the user himself revokes the (incorrect) operating process, corrects it or carries it out too late (much later than a corresponding causal event).

To detect an incorrect or unsuccessful operating process, data relating to the user device (also to be understood from the user device AG10, AG11, AG12, F10, F11, F12 (e.g. in relation to one or more operating processes of an application of the user device AG10, AG11, AG12, F 10, F11, F12). In particular, a (machine-learned) artificial intelligence can be trained to recognize occupant errors as such.

For example, it can be recorded that users (statistically) and/or a specific user selects a specific menu item on the user interface of an application (app) of the user device AG10, AG11, AG12, F 10, F11, F12 each time and then several times each time one after the other, with several operations, or sufficiently frequently, the selection of the menu item is canceled again.

Alternatively or additionally, consequences (of any kind or of one or more specific types) of the user's actions, e.g. in different operations, situations, contexts, etc., can be determined. There may be consequences that (particularly in hindsight or according to a subsequent evaluation of the data) lead to an unexpected result, with none of one or more predetermined criteria (defined as “correct”, “positive”, or “not incorrect”) and/or patterns have led to sufficiently consistent or similar, so to speak strange or obviously undesirable results.

Alternatively or additionally, a deviation of the operating process of a specific user from a statistically typical operating process and/or deviation of the user, e.g. a typical user and/or an occupant who belongs to the same group of people, can be determined and taken into account. By analyzing data relating to the operation of user devices AG10, AG11, AG12, it can thus be made possible to identify an occupant error when operating a vehicle F10, F11, F12 (as such or as an operating error of a certain type or with a certain consequence) to recognize and/or predict.

If an occupant error is detected, in particular predicted, when operating a vehicle F10, F11, F12, at least one (e.g. as expedient or appropriate as possible), in particular preventive measure, can be initiated. The measure can be determined depending on one or more patterns (e.g. depending on a pattern of actions that does not lead to an expected or satisfactory result).

If necessary, several alternative or additional measures can be selected. As a measure, occupant information (suitable, e.g. to be issued preventively) can be generated.

Particularly preferably, preventative occupant information (e.g. dependent on a recognized pattern) can be generated, with which a new occupant error or a corresponding consequence can be prevented. Furthermore, a measure can be a change or a targeted adaptation of one or more properties of the operating interface and/or the functionality of the vehicle F10, F11, F12 and/or another similar vehicle (in particular also to be understood as a vehicle of a similar type, equipment , age, technologies used, etc.) include F10, F11, F12.

For example, a measure can be generated (partially, predominantly or completely automated), in particular in the form of information for the occupant as a suggestion, to change the corresponding meaning or the functioning of the operating interface and/or a parameter of the corresponding functionality.

The measure can be for the same specific person as an occupant of the vehicle F10, F11, F12 and/or for another person, in particular an occupant who belongs to the same group of people and/or with a similar personal profile as the one or more users of the user devices AG10, AG11, AG12 is assigned.

Alternatively or additionally, a measure can be aimed at avoiding or reducing a consequence of the occupant error. If necessary, a repetition of the occupant error can be permitted, but at least one measure to avoid a consequence associated with the occupant error (on the vehicle F 10, F11, F12, ...) can be initiated or prepared.

As part of the process described in this document, it can be recorded (also to be understood as a type of “observation”) that certain occupant errors and/or certain frequency distributions of (certain) occupant errors depend on a characteristic of the occupant's personality, e.g. the group of people, to which an inmate can be assigned. This can be recognized, for example, by the frequency of (similar) operating errors on the user device and/or determined in a quantitative manner. For example, the probability or risk and/or consequence of certain operating errors in the vehicle can be determined, in particular predicted, depending on the information determined in the method.

For example, the occupant's membership of one or more groups of people can be viewed as a personal profile or parameters of a personal profile of the occupant.

The detection or interpretation of an occupant error and/or a measure to avoid the occupant error or to avoid the repetition of the occupant error and/or a measure to reduce a consequence of an (expected) occupant error can therefore be dependent on the recognition of a group of people who are the Inmate is heard or assigned automatically, e.g. based on his information, input and/or behavior. Thus (at least statistically) the operation of a vehicle F10, F11, F12, ... can be further improved.

3 shows a flowchart of an exemplary method 100 (possibly computer-implemented) for supporting an occupant of a vehicle F10, F11, F12.

The method 100 includes determining 101 the first data that is characteristic of one or more first operations when operating a mobile user device AG10, AG111, AG12.

• - Daten eines oder mehrerer Bedienelemente (z.B. rohe oder zumindest teilweite verarbeitete Sensordaten) im Zusammenhang mit dem Bedienvorgang;
• - Daten zum Zustand des jeweiligen Nutzers;
• - Daten zu einer oder mehreren Handlungen des Nutzers, wobei die Handlungen der Vorbereitung oder Ausführung des Bedienvorgangs dienen;
• - Daten einer oder mehreren Tätigkeiten, die der Nutzer neben dem Bedienvorgang, in einem zeitlichen und/oder kausalen Zusammenhang mit dem Bedienvorgang ausgeführt hat.

The first data are, for example, in a temporal and/or causal connection with one or more operating processes, in particular with (eg certain user errors or user errors of a certain type) in one or more first operating processes. For example, it is the first data from a time interval (e.g. of approximately 10, 20, 30 seconds) before the user error, and/or from a time interval in which the user error occurs (e.g. during the user error), and/or in a Time interval (e.g. of approx. 10, 20, 30 seconds) after the user error. The first data may include or be:

• - Data from one or more operating elements (e.g. raw or at least partially processed sensor data) in connection with the operating process;
• - Data on the status of the respective user;
• - Data on one or more actions by the user, whereby the actions serve to prepare or carry out the operating process;
• - Data from one or more activities that the user has carried out in addition to the operating process, in a temporal and/or causal connection to the operating process.

The first data can in particular (also) be data that is determined dependently, in particular on the basis of the first data.

Optionally, the method 100 includes determining, in particular recognizing, a user error when operating one or more first user devices AG10, AG11, AG12, which is caused in particular by one or more users who are (at the same time or later) passengers of a vehicle. The user errors can be recognized as such, especially retrospectively or post-factum (e.g. through data processing in the backend).

The user errors relate in particular to operating processes or are made in connection with the operating processes (generally: usage processes) of the user devices AG10, AG11, AG12, ... by their respective, in particular current, users. These can be recognized as such, assigned to the specific operations and/or the first data can be processed or taken into account in connection with the first operations. The user errors and/or occupant errors may be caused by other, For example, technical errors can be distinguished, for example by recognizing that there was no technical error or that the influence of a technical error was comparatively small.

Furthermore, the method 100 includes the recognition 102, in particular the prediction, of a second operating process of a second vehicle F10, F11, F12, wherein the second operating process has a (sufficient) similarity, in particular at least a certain degree of similarity, with the one or more first operating processes, in particular those in which one or more (preferably an accumulation) of the first operating errors has been detected. A quantitative or finely graded similarity measure can be determined and taken into account. For example, the selection of a measure (from several possible ones), and/or determining or adjusting the parameters of the measure and/or initiating the measure can (also) be a function of the similarity measure. For example, if the similarity measure is relatively low, a less intensive measure can be initiated.

For example, a second operating process can be recognized for which a similarity (e.g. estimated by means of pattern recognition or by means of a pattern comparison), in particular a similarity measure (e.g. an indicator which is greater the more similar the operating processes are), has a value that is greater than a certain threshold value.

The method 100 further includes executing (in particular also to be understood as initiating or causing) 103 a measure in relation to an (initially possible or to be considered possible) occupant error in the second operating process. This may be the occupant error of an occupant that is different from the one or more users of the one or more user devices (the one or more first operations).

Thus, at least as a result of method 100, there is the effect that a possible occupant error of a second occupant is prevented before the occupant of the vehicle F10, F11, F12 has even made an occupant error or before the occupant has started a specific operating procedure. An operating process can be related to an input and/or output of information using the respective vehicle F10, F11, F12.

The measure can be carried out depending on whether a predetermined precondition or one that can be changed in the process is met. In particular, the precondition can exclude potentially disadvantageous, in particular dangerous, measures (in the vehicle, in particular when the vehicle is being driven).

The measure is carried out in particular in response to one or more previous steps of the method 100, in particular in a predetermined dependency on the data or information determined. This is preferably done dynamically and, so to speak, in a manner tailored to the relevant case. Furthermore, the method 100 may also include initiating (effecting) a measure that changes the behavior of the vehicle, in particular temporarily and/or permanently. This can be done in relation to certain inmates and/or groups of inmates who are assigned to certain groups of people. The measure can (also) be an update of the at least one second vehicle that is tailored to specific occupants or occupants belonging to specific groups of people (i.e. tailored to them).

The method 100 can detect 101 (the term “recognize” can in particular also be understood as predictive recognition or prediction) of a second operating process on the vehicle AG10, AG11, AG12, F10, F11, F12, which has a similarity to at least one or has several first operations; wherein at least primarily first operating processes are selected that had a user error (operating error). However, it can also be first data that corresponds to very successfully carried out first operating procedures (also) determined and taken into account.

A second operating operation on the vehicle F10, F11, F12 can be recognized, which is at least similar to a first operating operation that has or is likely to have an occupant error (e.g. clumsiness or incorrect operation of the occupant). This can be viewed as an indication (particularly depending on the frequency of the occupant error in a large number of first operating processes) that an occupant error has also occurred or could still occur during the second operating process.

The method 100 may further include, in response to the detected second operation, effecting 103 an action regarding a possible occupant error in the second operation.

In particular, a measure can be implemented 103 in order to avoid (preferably proactively) that the second operating process also has the occupant error (in a simplified example by issuing a warning regarding the possible occupant error). Alternatively or additionally, a measure can be implemented to avoid or at least reduce a possible consequence and/or effect of the occupant error on the vehicle F10, F11, F12, ... (e.g. by taking the occupant error into account when operating the vehicle F10, F11, F12, ...). So the comfort and reliability of operating a vehicle F10, F11, F12, . increase.

The method 100 can in particular also be understood as a method for handling occupant errors. The method 100 may include determining an occupant error when operating a first vehicle... F10, F11, F12,... during one or more first operations on the one or more user devices AG10, AG11, AG12,.... The method 100 can further include detecting, in particular predicting, a second operating process on a vehicle F10, F11, F12, ..., in particular of occupant errors that have a certain (sufficient, e.g. exceeding a predetermined static or dynamic threshold) degree of similarity with the one or several first operations.

Furthermore, the determined measure, in particular determined predictively (e.g. only or primarily in the event that a specific occupant error is made as expected), can be carried out.

The identified measure and/or the initiation of the measure can be linked to a (certain) precondition. For example, the measure is only carried out when and if the precondition is met. This means that no possible disruption needs to be created during the (second) operating process, for example if the operating process will not be similar or it is recognized that the probability of the occupant making an error is low. Such a precondition can be different, for example, for different occupant errors, e.g. for occupant errors with different (expected or maximum expected) consequences. The precondition can, for example, be dependent on a further increasing probability and/or dependent on the exceeding of a probability measure and/or a consequence of the occupant error.

An occupant error is preferably predicted before (e.g. a few seconds or milliseconds before) its expected occurrence, in particular before the second operating process. This alone can save (valuable) time for determining or selecting and executing the measure.

For example, one or more suitable measures or variants of measures can be determined in advance (unnoticed by one or more occupants, in the background, so to speak) even with relatively small error probabilities (which are then only carried out if necessary, for example if the probability increases). Such a measure can include or be a measure for preparing a device of the vehicle for the operating error or a consequence of a (possible) operating error.

By determining a possible measure in advance, a (e.g. predictive) algorithm can be given the necessary time to predict an occupant error. If it is recognized that the precondition for a particular measure applies, then the measure can be selected and carried out.

As part of the method 100, one or more parameters, boundary conditions and/or the context in connection with the occurrence of the at least one occupant error in the one or more first operating processes can be determined and taken into account.

Alternatively or additionally, one or more parameters of the driving situation and/or boundary conditions that precede the occurrence of the occupant error and/or one or more parameters of the boundary conditions that essentially exist during the occurrence of the occupant error can be determined and taken into account.

The first data or one or more parameters can be stored in a memory, for example in a ring memory or in a comparable memory structure, and provided and/or further processed depending on the occurrence or detection of an occupant error.

In particular, one or more parameters of the boundary conditions, preferably a pattern of parameters of the boundary conditions, can be determined and taken into account in connection with the occurrence of the occupant error in the one or more first operating processes.

Preferably, data relating to the one or more parameters of the boundary conditions can be determined, compared and taken into account in a large number of first operating processes.

• - Beleuchtungsverhältnisse, insbesondere eine Stärke und/oder Richtung der Beleuchtung, Veränderung der Beleuchtung im Verlauf der Zeit (z.B. Alleenfahrt, Brücken, kurvige Straße, etc.);

und/oder

• - Lärmverhältnisse, insbesondere Lärm anderer Fahrzeuge, Lärm von einem oder mehreren weiteren Insassen (z.B. bestimmte Stimmen), Radio, etc.;

und/oder

• - eine Bewegung, insbesondere ein Bewegungsmuster, des Anwendergeräts AG10, AG11, AG12, ... bzw. des Fahrzeugs F 10, F11, F12;

und/oder

• - eine Bediensituation, z.B. ein oder mehrere Aktionen, die vor dem ersten Bedienvorgang erfolgen, und/oder ein oder mehrere Aktionen, die (zumindest wahrscheinlich) nach dem ersten Bedienvorgang folgen.

One or more parameters of the boundary conditions can be characteristic, in particular representative, of:

• - Lighting conditions, in particular the strength and/or direction of the lighting, ver changes in lighting over time (e.g. driving on avenues, bridges, winding roads, etc.);

and or

• - Noise conditions, in particular noise from other vehicles, noise from one or more other occupants (e.g. certain voices), radio, etc.;

and or

• - a movement, in particular a movement pattern, of the user device AG10, AG11, AG12, ... or the vehicle F 10, F11, F12;

and or

• - an operating situation, for example one or more actions that take place before the first operation and/or one or more actions that (at least probably) follow the first operation.

Alternatively or additionally, a temporal and/or spatial change characteristic, in particular a gradient, such as a temporal and/or spatial gradient, of a (respective) parameter or a parameter value can be determined and taken into account as a parameter.

As part of the method 100, one or more parameters of the consequence of at least one occupant error in relation to the one or more first and/or second operating processes, in particular several variants of an operating process, can be determined and taken into account.

Preferably, the at least one measure is determined depending on one or more parameters of a corresponding consequence, in particular predicted in the context of the method 100. Depending on a predicted consequence, for example, you can react earlier and/or with a different measure.

For example, a measure is selected from a list of at least two different measures or different types of measures in connection with the at least one second operating process, possibly depending on the data relating to the one or more parameters of the consequences for the one or more first ones operations. In other words, a prediction of a consequence of a specific occupant error can take place depending on data from the one or more first operating processes.

Preferably, the data can be determined, compared and taken into account in relation to the one or more parameters of the consequence of a large number of occupant errors. For example, statistical accumulations of occurrences of certain or consequences of occupant errors of a certain type characterized by certain parameters, in particular with one or more certain criteria, e.g. patterns of corresponding first operating procedures, driving situations, contexts, etc., can be determined and taken into account.

The first data preferably characterize a number or frequency of certain data patterns in connection with the occurrence and/or non-occurrence of the at least one, in particular a specific, operating error or an operating error of a specific type. The first data can occur after and/or approximately simultaneously with the occurrence of the operating error is determined from a large number of initial operating processes. The at least one measure in a second operating process and/or the mathematical relationship can then be determined, for example also depending on the information on a number or frequency of occurrence of the occupant error.

For example, information is determined that is characteristic of the number or frequency of occurrence of a user error, a specific user error or an occupant error of a specific type in a large number of first operating processes, in particular in connection with a specific parameter, in particular pattern of the parameters (e.g. the Movement, the driving situations, boundary conditions, for example a certain pattern of the parameters of the driving situations and/or boundary conditions.

Alternatively or additionally, the first data can be indicative of the number or frequency of non-occurrence of a user error, a specific user error or a user error of a specific type in a large number of first operating processes, in particular in connection with a specific, in particular the same, parameter, in particular parameter patterns of the driving situations and/or the boundary conditions, for example the specific patterns of the parameters of these.

As part of the method 100, one or more parameters of the second operating process can be determined, and based on this, at least one measure of similarity of the one or more parameters of the second operating process with one or more parameters of the one or more first operating processes can be determined and taken into account.

One or more patterns of the parameters of the second operating procedure can particularly preferably be used initially determined and at least one similarity measure of the one or more determined patterns with at least one or more parameters of a first operating process are determined and taken into account. For example, a combination and/or a pattern of parameters that characterize a position and/or a driving profile of the occupant can be determined and taken into account.

As part of the method 100, information characterizing at least one occupant error in the one or more first operating processes can be determined and taken into account, in particular depending on the occupant and/or the group of people to which the occupant belongs or to which the occupant is assigned in the method.

The at least one measure, in particular a variant and/or a parameter of the measure, can be determined depending on a previously determined input or selection of the occupant.

In particular, the occupant can be enabled to decide (in advance) about a measure that should be carried out if a specific occupant error (e.g. an occupant error of a certain type) is recognized or predicted. For example, it can be made possible for the occupant to select, program or put together a measure (possibly configurable and designed by the occupant) from predetermined elements. For example, the occupant can be enabled to select, create or preconfigure reminder information, e.g. a text, a keyword, an announcement, etc.

In other words, the occupant can be given the opportunity to decide how he or she can possibly be helped in the event of a specific occupant error or an occupant error of a certain type.

The at least one measure can be a change to an operating logic and/or a parameter of the user interface of the vehicle F10, F11, F12, . include. For example, depending on a (statistical) accumulation of one or more specific user errors in one or more first operating processes, the behavior of the respective vehicle F10, F 11, F 12, ... in particular with regard to the operating logic and / or a parameter of the user interface of the vehicle F10, F11, F12, ... can be changed.

For example, a menu structure of the vehicle F10, F11, F12, ... can be changed (e.g. in several steps).

As part of the method 100, the at least one measure can include an output of occupant information with a specific suggestion for changing the behavior of the vehicle (e.g. at least one device or functionality of the vehicle) F10, F11, F12. The occupant information can, for example, include a suggestion for changing the operating logic and/or a parameter for interpreting an input from the occupant of the vehicle F10, F11, F12. The (then possibly effective) change in the behavior of the vehicle F10, F11, F12 can be carried out depending on confirmation from the occupant.

A vehicle F10, F11, F12, ... can be set up to receive data relating to an occupant error described in this document directly or indirectly from the one or more user devices AG10, AG11, AG12, .... In particular, this concerns data processed, for example, in the backend BE or as part of data processing DV. Furthermore, it can be configuration data corresponding to the method. For example, the configuration data can include or represent the mathematical relationship already described.

Furthermore, this document describes a system (made up of one or more devices) that is set up to carry out one of the methods 100 described in this document.

In particular, the system is or includes one or preferably a large number of user devices AG10, AG11, AG12, ..., one or more vehicles F10, F11, F12, ... and possibly a backend BE (in particular with data processing DV for Aggregation of the data from a large number of user devices AG10, AG11, AG12. It can be a so-called distributed system. Depending on the result of the data processing or the aggregation of the data, the said mathematical relationship can be determined (and in vehicles F10, F11, F12 can be used).

The system may be configured to perform one or more of the methods 100 described in this document. It can be a distributed system. The system can be designed according to one or more of the features of the method described in the document.

It can be determined, in particular predicted, that the occupant is making (possibly in the near future) an occupant error, in particular an occupant error of a certain type, and/or will make it with at least a certain degree of probability or is already at least partially doing so is to make an inmate mistake. If necessary, it can also be recognized that no occupant error, in particular no occupant error of a certain type, is made. This information can also be taken into account when determining the measures and/or during the general operation of the vehicle F10, F11, F12, ....

As part of the method 100, a measure of a probability and/or a measure of a consequence of an occupant error that has already been made, is currently occurring or has been predicted can be determined and taken into account.

Depending on the information identified, one or more measures can be identified. A measure can be determined that is intended to prevent a (specific, expected) occupant error and/or to avoid or reduce a consequence of the occupant error. Alternatively or additionally, a measure for occupant information and/or for changing the behavior of the vehicle (e.g. one or more specific functionalities or those affected by the operating error) of the vehicle F10, F11, F12 can be determined. Two or more different measures are preferably determined.

A prerequisite can (each) be determined or adjusted for one or more measures. A precondition can identify a specific trigger in which a specific measure, in particular one of several measures, is selected, parameterized, initiated and/or executed. The precondition can be dependent on a parameter of the operating process, the operating situation, the context, the position, the driving situation, a boundary condition in which the vehicle F10, F11, F12, ... is operated, and/or dependent on any other data .

For example, the at least one precondition can be linked to an action and/or an operating action by the occupant. The precondition can also be linked to a specific pattern from the data mentioned in this document. Alternatively or additionally, a precondition can be linked to a certain level of probability and/or to a consequence of an occupant error.

For example, measures can be linked to respective preconditions or measures and preconditions can be assigned to one another, e.g. linked. The measures and/or preconditions can, for example, be initiated before their (possible) execution.

In an example that can be combined with all the features of this document, at least one measure and/or at least one precondition can be determined depending on one or more settings of the vehicle F10, F11, F12, ... and/or on a selection of the occupant.

For example, the precondition or a parameter of the precondition can be selectable (e.g. by the occupant himself). For example, the execution of (certain) measures can be linked to (certain), in particular selectable, preconditions by means of a configuration or settings.

One or more measures can be initiated, each of which is linked to one or more preconditions. Different measures with different preconditions can preferably be initiated for several (currently probable or impending) variants of an occupant error.

For example, the measures can be determined in the backend BE and transmitted to the respective vehicle F10, F11, F12, .... This has the advantage that relatively complex data aggregation DV and/or prediction can also be determined in the backend BE. This does not necessarily have to be provided in the vehicles.

For example, a list of instructions that is characteristic of carrying out and/or triggering certain measures in the event of certain events is determined and initiated on the vehicle F10, F11, F12.

Depending on whether the occupant error and, if applicable, which variant of the occupant error is to be expected, a measure can then be selected and carried out (in real time, with low resource consumption). This can result in a decision regarding a measure being delayed. The number of probable occupant errors and/or measures can, for example, be gradually reduced to particularly likely variants. This enables improved, precise and effective functionality.

As part of the method 100, a parameter of a further course of the operating process, the operating situation, the context, the position, the movement profile, a driving situation, a boundary condition in which the vehicle F10, F11, F12, ... is operated, and / or further data can be determined. One or more preconditions can be checked (e.g. continuously, cyclically).

As part of the method 100, a measure can be initiated and/or carried out when a (certain, expected) occupant error is detected and/or when the at least one precondition is met and/or when a certain measure of a probability and/or a consequence of a (possible, expected) occupant error increases.

As part of the method 100, depending on the information determined and/or depending on the progress of the occupant error, one or more different measures can be carried out with respect to the vehicle F10, F11, F12.

The method 100 described in this document provides, so to speak, individually tailored support for (e.g. human, particularly typical for certain people or groups of people) occupant errors. Furthermore, the performance of a vehicle (also to be understood as: vehicle functionality or vehicle system) F10, F11, F12, ... can be increased. Furthermore, the subjective experience when using the vehicle can be improved.

The system can also be designed in such a way that the detection of occupant errors and the reaction to occupant errors can be continuously improved. The measures described can reduce the consequences of occupant errors. A correspondingly set-up prediction algorithm can be used to recognize the consequences of (certain) occupant errors.

In particular, the invention also includes the case in which the vehicle F10, F11, F12 is the user device AG10, AG12, AG12 (at a later point in time) or is viewed as such. Alternatively or additionally, the occupant of the vehicle may be the user of the user device (at a later time). Particularly in such a case, person-specific operating errors, in particular those that are repeated particularly frequently by the occupant, can be handled.

The present invention is not limited to the exemplary embodiments shown. In particular, it should be noted that the description and the figures are only intended to illustrate the principle of the proposed methods, devices and systems by way of example.

Claims (15)

Method (100) for supporting an occupant of a vehicle (F10, F11, F12, ...) when operating the vehicle (F10, F11, F12, ...), comprising: - Determining (101) the first data characterizing one or more first operations of a first mobile user device (AG10, AG11, AG12); - Detecting (102) a second operating process of a vehicle (F10, F11, F12); wherein the second operation has at least some degree of similarity to the one or more first operations; and then: - Execute (103) a measure in relation to the detected second operating operation of a vehicle (F10, F11, F12). Procedure (100) according to Claim 1 , wherein the first data characterizing one or more first operating processes of one or more first mobile user devices (AG10, AG11, AG12) are indicative of: - a user error when operating the mobile user device (AG10, AG11, AG12); and/or - one or more information about the settings and/or operating logic of the respective mobile user device (AG10, AG11, AG12). Procedure (100) according to Claim 1 or 2 , wherein the first data characterizing one or more first operating processes of one or more first mobile user devices (AG10, AG11, AG12) are characteristic of: - one or more first measures carried out to prevent and / or correct one or more user errors in one or more first Operating processes of one or more mobile user devices (AG10, AG11, AG12); and/or - a result, in particular a measure of the success, prevention and/or correction of one or more user errors during one or more first operations of one or more mobile user devices (AG10, AG11, AG12). Method (100) according to one of the preceding claims, wherein depending on the first data characterizing one or more first operating processes of one or more first mobile user devices (AG10, AG11, AG12): - Training data for training a neural network (DNN1, DNNZ, DNN2) to carry out (103) a measure in relation to the recognized or predicted second operating operation of a vehicle (F20, F21, F22); and or - Learning data of a neural network (DNN1, DNNZ, DNN2) for executing (103) a measure in relation to the recognized or predicted second operating operation of a vehicle (F20, F21, F22); Depending on the first data, one or more first operating processes of a first mobile user device (AG10, AG11, AG12) are determined. Method (100) according to one of the preceding claims, comprising: - determining first user information in relation to a user of the user device (AG10, AG11, AG12), in particular characterizing a personality profile of the user and / or a group of people to which the user of the user device ( AG10, AG11, AG12); and/or - determining second occupant information relating to an occupant of the vehicle (F10, F11, F12), in particular characterizing a personality profile of the occupant of the vehicle (F10, F11, F12) and/or a group of people belonging to the occupants of the vehicle ( F10, F 11, F12); and - taking the first user information and/or the second occupant information into account in the method. Method (100) according to Claim 5 , comprising: - determining a similarity measure between the personality profile of the user of the user device and the personality profile of the occupant of the vehicle (F10, F11, F12); and/or - determining a similarity measure between the group of people to which the user of the user device (AG10, AG11, AG12) belongs and the group of people to which the occupant of the vehicle (F10, F11, F12) belongs; and: - Taking into account at least one similarity measure in the procedure. Method (100) according to Claim 5 or 6 , comprising: - recognizing (102) the second operating process; and/or - executing (103) the measure in relation to the detected second operating operation of a vehicle (F10, F11, F12); depending on the first user information of the user of the user device (AG10, AG11, AG12); and/or depending on the second occupant information relating to an occupant of the vehicle (F10, F11, F12); and/or depending on the degree of similarity between the personality profile of the user of the user device and the personality profile of the occupant of the vehicle (F10, F11, F12); and/or depending on the degree of similarity between the group of people to which the user of the user device (AG10, AG11, AG12) belongs and the group of people to which the occupants of the vehicle (F10, F11, F12) belong. Method (100) according to Claim 5 or 6 , comprising: - Determining training data for training a neural network (DNN1, DNNZ, DNN2), the neural network (DNN1, DNNZ, DNN2) being used to carry out (103) a measure in relation to the recognized or predicted second operating operation of a vehicle ( F20, F21, F22); depending on the first user information of the user of the user device (AG10, AG11, AG12); and/or depending on the second occupant information relating to an occupant of the vehicle (F10, F11, F12); and/or depending on the degree of similarity between the personality profile of the user of the user device and the personality profile of the occupant of the vehicle (F10, F11, F12); and/or depending on the degree of similarity between the group of people to which the user of the user device (AG10, AG11, AG12) belongs and the group of people to which the occupants of the vehicle (F10, F11, F12) belong. Method (100) according to one of the preceding claims, comprising: - Determining an input from an occupant of the vehicle (F10, F11, F12, ...) regarding a measure to be carried out, in particular an association between a specific operating error or an operating error of a certain type and the measure then to be carried out or a type of measure to be carried out; and - Execute (103) a measure dependent on the occupant's input. Method (100) according to one of the preceding claims, wherein the measure is aimed at: - to change an operating logic and/or a parameter of a user interface of the vehicle (F10, F11, F12), in particular temporarily and/or in a passenger-specific manner; and or - the measure includes a, in particular temporary and/or occupant-specific, change to an operating logic and/or a parameter of the operating interface of the vehicle (F10, F11, F12, ...). Method (100) according to one of the preceding claims, comprising - Generating a proposal for changing a behavior of the vehicle (F10, F11, F12), in particular the behavior of an information output device, an information input device and / or a user interface of the vehicle (F10, F11, F12, ...). Method (100) according to one of the preceding claims, in which one or more user devices (AG10, AG11, AG12) are interpreted as one or more vehicles (F10, F11, F12), and: the one or more vehicles (F10, F 11, F12) can be understood as one or more user devices (AG10, AG11, AG12); and the method (100) supports the user of one or several user devices (AG10, AG11, AG12) by carrying out the at least one measure in relation to the operating process of the one or more user devices (AG10, AG11, AG12). Neural network (DNN1, DNNZ, DNN2) for supporting an occupant of a vehicle (F10, F11, F12, ...) when operating the vehicle (F10, F11, F12, ...), wherein the neural network (DNN1, DNNZ , DNN2) is trained depending on the first data characterizing one or more first operating processes of a mobile user device (AG10, AG11, AG12), with the neural network (DNN1, DNNZ, DNN2) being used to carry out a measure in relation to a second operating process of a vehicle ( F10, F11, F12) can be carried out, which has at least a certain degree of similarity with the one or more first operating processes of a mobile user device (AG10, AG11, AG12). System for supporting an occupant of a vehicle (F10, F11, F12) when operating the vehicle (F10, F11, F12, ...); wherein the system is set up: for determining (101) the first data characterizing one or more first operating processes of a mobile user device (AG10, AG11, AG12); and is set up: for recognizing (102) a second operation of a vehicle (F10, F11, F12); wherein the second operation has at least some degree of similarity to the one or more first operations; and is also set up: then carry out a measure in relation to the detected second operating operation of the vehicle (F10, F11, F12) (103). Computer program product, comprising a computer program, wherein the computer program product is designed, when executed on at least one computing unit, one or more steps of the method (100) according to at least one of Claims 1 until 12 to carry out.

Images