EP4790223A1

EP4790223A1 - Method for improving an opening of a flap of a vehicle, method for training a machine-learning model, apparatus, vehicle and computer program

Info

Publication number: EP4790223A1
Application number: EP25157124.6A
Authority: EP
Inventors: Annabel Matz; Fanny Heidmann; Kristina Naskovska
Original assignee: Bayerische Motoren Werke AG
Current assignee: Bayerische Motoren Werke AG
Priority date: 2025-02-11
Filing date: 2025-02-11
Publication date: 2026-08-12

Abstract

Embodiments relate to a method for improving an opening of a flap of a vehicle. The method comprises obtaining a movement profile indicative of a trajectory of a user relative to the vehicle, generating an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle, estimating, by a trained machine-learning model and based on the adjusted movement profile, whether the flap is to be opened and, if it is estimated by the trained machine-learning model that the flap is to be opened, triggering the opening of the flap.

Description

The present disclosure relates to the field of intent estimation. Embodiments relate to a method for improving an opening of a flap of a vehicle, a method for training a machine-learning model, an apparatus, a vehicle and a computer program.
In trajectory-based intent prediction for automotive applications (e.g., predicting charging intent), the varying dimensions of different vehicle models can result in distinct trajectories (e.g., coordinates) for each vehicle. Despite this, the actual positions represented by these trajectories relative to the vehicle boundaries may be similar. For instance, the origin of the coordinate system used to measure trajectories might be fixed at a specific point on or within the vehicles. Conventional techniques for employing a machine-learning model may collect training data for each of the vehicle models and independently train the models. This approach can lead to time loss due to data collection, training and testing of the respective models. Thus, there is a demand for improved training and inference in trajectory-based intent predictions for automotive applications.
It is a finding of the inventors that training data can be collected for one or more vehicles; this training data can be pre-processed separately for these vehicles, and a shared AI (artificial intelligence) model can be trained and rolled out on any vehicle.
Examples provide a method for improving an opening of a flap of a vehicle. The method comprises obtaining a movement profile indicative of a trajectory of a user relative to the vehicle, generating an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle, and estimating, by a trained machine-learning model and based on the adjusted movement profile, whether the flap is to be opened. The method further comprises, if it is estimated by the trained machine-learning model that the flap is to be opened, triggering the opening of the flap.
For instance, a vehicle-specific normalization pre-processing step can be applied at inference time before the AI prediction model estimates the flap opening. Similarly, such a pre-processing step may be applied at the training stage of the model. This may enable training and roll out only few or even a single AI prediction model which scales to multiple vehicles models.
In some examples, generating the adjusted movement profile comprises normalizing coordinates of the trajectory based on the dimensions of the vehicle. This may ensure that the normalized trajectory is expressed on a consistent, vehicle-relative scale, which allows for seamless adaptation to different vehicle sizes and shapes.
In some examples, the trained machine-learning model is trained based on training data measured with a reference vehicle. Generating the adjusted movement profile may comprise adjusting the trajectory based on dimensions of the reference vehicle. This may ensure that the model which has learned trajectory patterns and behaviors specific to one or more specific (reference) vehicles, can be applied to various vehicles of varying dimensions.
In some examples, generating the adjusted movement profile comprises at least one of a translation and scaling of the trajectory or a part thereof based on the dimensions of the vehicle. This may allow for customization of the movement profile to account for the specific size and geometry of the vehicle.
In some examples, a point of the trajectory is translated if a value of its coordinates falls into a first range and a point of the trajectory is scaled if a value of its coordinates falls into a second range different from the first range. For example, the first range may correspond to an area extending in front or behind the vehicle and the second range may correspond to an area adjacent to a longitudinal side of the vehicle. This approach may take into consideration the varying proportions of vehicles in terms of length and width.
Examples provide a method for training a machine-learning model. The method comprises obtaining training data. The training data comprises at least one movement profile indicative of a trajectory of a user relative to a vehicle and at least one corresponding label indicative of whether a flap of the vehicle is to be opened. The method further comprises generating an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle and training, based on the adjusted movement profile and the label, a machine-learning model to estimate whether the flap of the vehicle is to be opened. This approach may allow the machine-learning model to be trained with a reduced influence of specific vehicle dimensions on the accuracy of its predictions.
Examples provide an apparatus comprising processing circuitry configured to perform one or more of the methods described herein.
Examples further provide a vehicle comprising the apparatus as described herein.
Examples further provide a computer program having a program code for performing the method described above, when the computer program is executed on a computer, a processor, or a programmable hardware component.
Some examples of apparatuses, methods and/or computer programs will be described in the following by way of example only, and with reference to the accompanying figures, in which

Fig. 1 shows an example of a method for improving an opening of a flap of a vehicle;
Fig. 2 shows a juxtaposition of two examples of vehicles with different dimensions and respective trajectories;
Fig. 3 shows an example of a method for improving an opening of a flap of a vehicle comprising normalization;
Fig. 4 shows a juxtaposition of two examples of vehicles with different dimensions and respective semantic points;
Fig. 5 shows an example of a separation of areas around a vehicle for applying different coordinate transformation techniques in the x direction;
Fig. 6 shows an example of a separation of areas around a vehicle for applying different coordinate transformation techniques in the y direction;
Figs. 7a-7c shows examples of implementation of a coordinate transformation from respective vehicles to respective reference vehicles;
Fig. 8 shows an example of a method for training a machine-learning model to be used for interpreting a movement profile; and
Fig. 9 shows an example of an apparatus for a vehicle to implement a method as described herein.

As used herein, the term "or" refers to a non-exclusive or, unless otherwise indicated (e.g., "or else" or "or in the alternative"). Furthermore, as used herein, words used to describe a relationship between elements should be broadly construed to include a direct relationship or the presence of intervening elements unless otherwise indicated. For example, when an element is referred to as being "connected" or "coupled" to another element, the element may be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. Similarly, words such as "between", "adjacent", and the like should be interpreted in a like fashion.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes", or "including", when used herein, specify the presence of stated features, integers, steps, operations, elements or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Fig. 1 shows a block diagram of an example of a method 100 for improving an opening of a flap of a vehicle. The method 100 may be executed by a computer system or a control device of the vehicle, for instance. "Opening the flap" may be understood as unlocking the flap, e.g., for enabling a manual opening of the hinge of the flap, and/or automatic movement of the hinge of the flap. The flap may be a charging port or a trunk tailgate, for instance. An example application of the method 100 may be intent classification of trajectories for identifying charging intents.
The method 100 comprises obtaining 110 a movement profile indicative of a trajectory of a user relative to the vehicle. The trajectory may mainly include points in an environment outside and close to the vehicle. The movement profile may be a quantitative representation of the user's motion characteristics as they approach or interact with the vehicle for understanding whether they intend to open the flap. It may encompass various parameters that describe the user's trajectory, e.g., position, orientation, and timing information. The trajectory may be described by at least one of vectors and waypoints, e.g., absolute coordinates (e.g., GPS, Global Positioning System), relative coordinates (e.g., relative to a predefined coordinate system which may be fixed to the vehicle), distance-based metrics (e.g., describing the distance of a waypoint to a specific part of the vehicle). In the following, the trajectory may be mainly described as being represented by relative coordinates, however, other forms of trajectory representation can similarly be implemented.
The trajectory may be measured by a tracking system, e.g., based on a signal (e.g., UWB, Ultra-Wideband) from the user's mobile phone, smartwatch or alike, or based on an optical system like a camera tracking the movement of the user. Obtaining 110 the movement profile may comprise receiving the movement profile from an external tracking system and/or determining the movement profile in case the tracking system is integrated into the apparatus executing the method 100.
The method 100 further comprises generating 120 an adjusted movement profile by adjusting the trajectory based on dimensions (e.g., length and width of an outer boundary) of the vehicle and estimating 130, by a trained machine-learning model and based on the adjusted movement profile, whether the flap is to be opened. For instance, when the adjusted movement profile and/or the adjusted trajectory or data derived thereof is input to the trained machine-learning model, it may output an estimation of whether the flap is to be opened. For instance, the machine-learning model may output a binary value meaning "flap is to be opened or not", a confidence value or interval or an error margin of its prediction.
The trained machine-learning model may be a data structure and/or set of rules representing a statistical model that is used to perform the task of identifying whether the flap is to be opened, i.e., identifying an intent of the user to open the flap, without using explicit instructions, instead relying on models and inference. The data structure and/or set of rules represents learned knowledge (e.g. based on training performed by a machine-learning algorithm). For example, in machine-learning, instead of a rule-based transformation of data, a transformation of data may be used, that is inferred from an analysis of historical and/or training data. In the proposed technique, the content of the movement profile is analyzed concerning the underlying interpretation of the intent of the user using the machine-learning model.
The machine-learning model is trained by a machine-learning algorithm. The term "machine-learning algorithm" denotes a set of instructions that are used to create, train or use a machine-learning model. For the machine-learning model to analyze the movement profile the machine-learning model may be trained using training and/or historical movement profiles as input and training content information (e.g., labels indicating whether the flap is to be opened for each of the training or historical movement profiles) as output. By training the machine-learning model with a large set of training data and associated training content information (e.g. labels or annotations), the machine-learning model "learns" to recognize the content of the movement profiles, so the content that are not included in the training data can be recognized using the machine-learning model. By training the machine-learning model using training data and a desired output, the machine-learning model "learns" a transformation between the data and the output, which can be used to provide an output based on non-training data provided to the machine-learning model.
The machine-learning model may be trained using training input data (e.g. training movement profiles). For example, the machine-learning model may be trained using supervised learning, semi-supervised learning, unsupervised learning, or reinforcement learning. Furthermore, additional techniques may be applied to some of the machine-learning algorithms. For example, feature learning may be used. In some examples, the machine-learning algorithm may use a decision tree, association rules or alike as a predictive model.
For example, the machine-learning model may be an Artificial Neural Network (ANN). ANNs comprise a plurality of interconnected nodes and a plurality of connections, so-called edges, between the nodes. There are usually three types of nodes, input nodes that receive input values (the movement profile or data derived thereof), hidden nodes that are (only) connected to other nodes, and output nodes that provide output values (an indication whether the flap is to be opened). Each node may represent an artificial neuron. Each edge may transmit information from one node to another. The output of a node may be defined as a (non-linear) function of its inputs (e.g. of the sum of its inputs). The inputs of a node may be used in the function based on a "weight" of the edge or of the node that provides the input. The weight of nodes and/or of edges may be adjusted in the learning process. In other words, the training of an ANN may comprise adjusting the weights of the nodes and/or edges of the ANN, i.e., to achieve a desired output for a given input. Alternatively, the machine-learning model may be a support vector machine, a random forest model, a gradient boosting model, a Bayesian network, a genetic algorithm, or alike.
In some examples, the trained machine-learning model may comprise a long short-term memory (LSTM). LSTMs are particularly effective at handling sequential data, which may be useful for understanding patterns over time, such as user movements or behavior. This is because LSTMs can remember important information from earlier steps in a sequence, while also forgetting irrelevant details, allowing for more accurate predictions. Another advantage may be that LSTMs can manage varying time intervals between inputs, making them robust when interpreting trajectories where the timing of events may be inconsistent.
Optionally, the movement profile may also indicate a speed of a movement of the user along the trajectory. The trained machine-learning model may be configured to estimate whether the flap is to be opened based on the adjusted trajectory and the speed. In such cases, the speed may be part of the training, too, i.e., the training data may include trajectories and associated speed. An advantage of incorporating the user's movement speed along the trajectory in the ML model is that it may enhance the accuracy of intent estimation. By considering both the trajectory and the speed, the method may gain a more comprehensive understanding of the user's behavior, allowing it to better distinguish between intentional and unintentional actions. This additional data may help reduce false positives and improve the overall robustness, especially in scenarios where similar trajectories may result in different intents based on speed.
Adjusting the trajectory may be done such that the subsequent estimation is more independent from the specific vehicle dimensions, i.e., such that the resulting adjusted movement better generalizes the trajectory relative to the flap across varying vehicle dimensions of different vehicle models - this generalized motion pattern may lead to a better prediction accuracy.
The adjustment may ensure the intent estimation less depends on the vehicle dimensions. For example, on a small vehicle, some waypoints of the trajectory with the user having charging intent might be closer to a specific reference point on the vehicle (e.g., origin of the coordinate system) if the reference point is set on the geometric center, the front middle, the axle or alike. On a large vehicle, some waypoints of the trajectory - also with the user having charging intent - might be further away. Depending on how the machine-learning model is trained, the one or the other case (small or large vehicle) might not lead to a correct estimation of the charging intent since the trajectory can be interpreted by the model as too far away from the flap or within the vehicle, for instance. Additionally, the flap may have also varying places on the vehicle across different models, not only concerning the length of the vehicle. For instance, the flap of one vehicle might be further at the rear corner, where another vehicle might have the flap more at its center. Thus, the adjustment of the trajectory may improve the model's prediction accuracy because it is less influenced by raw vehicle dimensions, which otherwise may introduce noise into the estimation process.
The challenge of intent estimation with varying vehicle dimensions is illustrated by Fig. 2. Fig. 2 shows a juxtaposition of two examples of vehicles 210, 220 with different dimensions and respective trajectories represented by a plurality of waypoints around the vehicles 210, 220, e.g., waypoints 231, 232 at vehicle 210 and waypoints 241, 242 at vehicle 220.
The trajectories around both vehicles are similar in their general pattern, however, since vehicle 220 is smaller than vehicle 210, the coordinates of the trajectory are different. In the example of Fig. 2, the coordinate system has its origin being fixed at the front middle of each of the vehicles 210, 220, the x axis runs along the length of each of the vehicles 210, 220, while the y axis is perpendicular to the x axis and points toward the side of each of the vehicles 210, 220. In this example coordinate system, the waypoints 231, 232 have the coordinates x = 113 cm (centimeters), y = -140 cm and x = 576 cm, y = 8 cm, respectively. The waypoints 241, 242 have the coordinates x = 109 cm, y = -132 cm and x = 506 cm, y = 7 cm, respectively.
Since the trained machine-learning model may deduct from, e.g., the waypoints' coordinates, their distances to the origin and/or their angles etc. whether the flap has to be opened. The characteristics of the waypoints may, for example, carry semantic meaning, such as indicating whether the user is sufficiently close to the flap or is following a specific path along the flap that suggests an intent to open it. Similarly, the fact that the user has exited the vehicle through the driver's door could provide a clue about whether the flap should be opened. When the waypoints are distorted due to varying vehicle dimensions, as illustrated in the example of Fig. 2, the machine-learning model may be unable to interpret their characteristics, and the prediction accuracy of the machine-learning model may therefore decrease. This is especially the case when the machine-learning model has been trained with trajectories relative to one or more specific vehicles, and when this trained machine-learning model then is to be applied to other vehicle models.
Referring back to Fig. 1, generating 120 the adjusted movement profile may include scaling at least some waypoints (or vectors) of the trajectory. For instance, the trajectory coordinates may be scaled proportionally based on the dimensions of the vehicle or scaled directionally, e.g., by applying nonuniform scaling such as extending the trajectory longitudinally for longer vehicles or laterally for wider vehicles. The adjustment may additionally or alternatively include offset-shifting, e.g., lateral shifting of the coordinates to align with the position of the flap relative to the vehicle's width, or longitudinally shifting the trajectory forward or backward based on the location of the flap along the vehicle's length. Additionally or alternatively, normalization techniques may be applied to the trajectory to normalize it to a consistent reference point, such as the geometric center of the vehicle or the flap's location. Geometric adjustments of the trajectory may be made, e.g., path expansion or contraction of the trajectory to account for the size of the vehicle. In case of coordinates, coordinate transformation such as translation, rotation, or scaling can be applied to the trajectory. The latter may preserve relative relationships between the waypoints of the trajectory.
In some examples, generating 120 the adjusted movement profile comprises standardizing or normalizing coordinates of the trajectory based on the dimensions of the vehicle. For example, the coordinates can be standardized or normalized to match or at least approach a reference's vehicle dimensions used for producing training data, an imaginary vehicle's dimensions or unit dimensions used for pre-processing training data before training or alike. The coordinate transformation technique used may depend on how the coordinate system is set up relative to the vehicle or on the placement of the origin.
In cases where the trained machine-learning model is trained based on training data measured with a reference vehicle, generating 120 the adjusted movement profile may in some examples comprise adjusting the trajectory based on dimensions of the reference vehicle. Alternatively, the measured trajectory may have been normalized for training the machine-learning model, as well, then the trajectory during inference may similarly be normalized (e.g., to unit dimensions or any other specific vehicle dimensions used for normalization as pre-processing for training the machine-learning model).
An example of the normalization is illustrated by Fig. 3. Fig. 3 shows a flow chart of an example of a method 300 for improving an opening of a flap of a vehicle. The method 300 comprises obtaining a first movement profile 311 indicative of a trajectory of a user relative to a first vehicle 310a and a second movement profile 312 indicative of a trajectory of the or another user relative to a second vehicle 31 0b. The first vehicle 310a and the second vehicle 310b have different dimensions. The trajectories of the first and second movement profile 311, 312 are similar in pattern (e.g., equal relative to the respective vehicle dimensions) but differ in the coordinates of their waypoints due to the different vehicle dimensions.
The method 300 further comprises generating 320 an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle. Generating 320 the adjusted movement profile comprises normalizing coordinates of the trajectory based on the dimensions of the vehicle.
The method 300 further comprises inputting 330 the adjusted movement profile into a trained machine-learning model and outputting 340, by the trained machine-learning model, an estimation of whether the flap is to be opened (here: an estimation of whether a charging intent is present and thus a charging flap is to be opened).
The method 300 may enable charging intent prediction for multiple vehicle models. The method 300 may enable the use of only one trained machine-learning model (or at least only few machine-learning models) for multiple vehicle models (derivates). The pre-processing of the movement profile may enable that the machine-learning model "thinks" that it only sees trajectories relative to one car (reference car) instead of having training trajectories and trajectories in field from different derivates.
The normalization of the trajectory may include virtually "blowing up" (enlarging) a vehicle's dimensions (and according trajectories) if vehicle is smaller than the reference car and virtually "shrinking" (downsizing) a vehicle's dimensions (and according trajectories) if vehicle is larger than the reference car. In this way, the model may get trained and applied on trajectories which look like they have been collected with a specific reference vehicle.
In some examples, the normalization of the coordinates is done with a certain coordinate transformation based on the vehicle dimensions. The coordinate transformation may be chosen such that one or more semantic points in the original coordinate system (in which the coordinates are recorded) and corresponding one or more semantic points of the training or reference vehicle (with which the machine-learning model was trained) have equal or substantially equal (e.g., equal after rounding) coordinates. The normalization therefore depends on the differences between the vehicle's dimensions and the training vehicle's dimensions.
The semantic point may be any point which represents a critical element in feature spaces of the machine-learning model for its downstream task of classification or decision-making which can be used to estimate whether the flap is to be opened. The semantic point may be at least one of the flap location, an edge of the vehicle and a point behind the vehicle with a certain distance to it. Examples of semantic points are illustrated by Fig. 4. Fig. 4 shows a juxtaposition of two examples of vehicles 410, 420 with differing dimensions and corresponding semantic points 431, 432, 433 and 441, 442, 443. In the examples of Fig. 4, the semantic points represent a front left corner (431, 441), the charging flap on the rear right side (432, 442) and a point 50 cm behind the respective car (433, 443).
The coordinate systems of both vehicles 410, 420 are chosen similarly like in Fig. 2. The coordinate system for the position of the measurement device (digital key or alike) is attached to the respective vehicle 410, 420 as follows: y axis = front wheel axis, x axis = middle of the car in the length. In other examples however, the origin may be attached to other locations, such as the exact middle of the car on both axes, the front left corner or alike, or the x and y axes may have a different orientation. The dimensions of vehicle 410 are 200 cm width and 400 cm length (100 cm from the origin of the coordinate system to the vehicle's front and 300 cm from the origin to the vehicle's back). The dimensions of vehicle 420 are 100 cm width and 200 cm length (50 cm from the origin of the coordinate system to the vehicle's front and 150 cm from the origin to the vehicle's back. Thus, vehicle 420 is half the size in length and width of vehicle 410.
In the example of Fig. 4, a machine-learning model could have been trained on training trajectories measured with vehicle 410. The semantic points of vehicle 410 have different coordinates than the semantic points of vehicle 420 due to their different vehicle dimensions. In order to embed the machine-learning model, pre-trained on training data from vehicle 410, into vehicle 420 (e.g., without retraining to avoid additional validation efforts), the coordinates of vehicle 420 (positions around vehicle 420) can be normalized such that the coordinates of the normalized semantic points of vehicle 420 match the coordinates of the semantic points of vehicle 410.
Referring back to Fig. 1, generating 120 the adjusted movement profile may, in some examples, comprise at least one of a translation and scaling of the trajectory or a part thereof (e.g., semantically important waypoints) based on the dimensions of the vehicle.
Translation shifts (data) points of the trajectory by adding a constant value to their coordinates. This moves the points in the same direction by the same amount without changing their relative distances. Translation may be applied to points of the trajectory when they fall close to the vehicle's boundary. The threshold of proximity to the car for using translation may be determined heuristically based on the desired accuracy of matching trajectories for different vehicles and the desired accuracy of prediction of the machine-learning model.
Scaling resizes points of the trajectory by multiplying one or more of their coordinates by a scaling factor. This changes distances between points but retains their relative positions if scaling is uniform. Scaling (proportional adjustment) may be applied to points of the trajectory if they extend beyond a certain threshold of proximity to the vehicle.
For instance, a point of the trajectory may be translated if its coordinates fall into a first range, and a point of the trajectory may be scaled if its coordinates fall into a second range different from the first range. The ranges may be chosen differently for the x and the y axes. The ranges may be chosen such that - after translation and scaling - the adjusted coordinates of the semantic points of the vehicle (which are specific for the machine-learning model and its application) match the reference vehicle's semantic points.
An example of a coordinate normalization on the x axis is illustrated by Fig. 5. Fig. 5 shows an example of a vehicle 510 and points 521-524 of a trajectory around the vehicle 510 which are to be normalized to a reference vehicle with the illustrated outer boundary 530. A simplified approach to normalize the trajectory is described in the following: To normalize the x coordinates of the points 521-524, it can be assumed - for simplification - that the dimensions of the vehicle 510 and the dimensions 530 of the reference vehicle match except that they are scaled. For example, the position of the charging flap of vehicle 510 compared to the coordinate system origin may differ by a scaling factor. The same may apply for the position of any door. In this suggestion of implementation, the x coordinates of the points 521-524 in the coordinate system of vehicle 510 are converted to the coordinate system of the reference vehicle via scaling and addition.
In the example of Fig. 5, the x coordinate of a point of the trajectory is translated if it falls into a first range corresponding to an area extending in front or behind the vehicle. This applies to the point 521 which is in front of the vehicle 510 and to the point 524 which is behind the vehicle 510. In the example of Fig. 5, the x coordinate of a point of the trajectory is scaled if it falls into a second range corresponding to an area adjacent to a longitudinal side of the vehicle 510. This applies to the points 522 and 523 on the right side of the vehicle 510. The first and second ranges for the x coordinates are indicated in Fig. 5 by two lines 541, 542, one separating the front of the vehicle 520 from its sides and one separating the back of the vehicle 520 from its side.
The translation and scaling in the x direction may be implemented by the following equations:

If the x coordinate of a point of the trajectory is smaller than VEH_FL_X (the x coordinate of the front edge of the vehicle 510, i.e., the point is in front of the vehicle 510): $x_Coordinate = x_Coordinate + d 1$
If the x coordinate of a point of the trajectory is greater than VEH_RR_X (the x coordinate of the back edge of the vehicle 510, i.e., the point is behind the car): $x_Coordinate = x_Coordinate + d 2$
Otherwise, i.e., the point is next to the car (left or right) $x_Coordinate = x_Coordinate \cdot length_scale + length_shift$
The values of d1, d2, length_scale and length_shift are determined as follows: $\begin{array}{l} reference_vehicle_length = REF_VEH_RR_X (\begin{array}{l} x coordinate of the back edge of the reference \\ vehicle \end{array}) - REF_VEH_FL_X (x coordinate of the front edge of the reference vehicle) \\ vehicle_length = VEH_RR_X - VEH_FL_X \\ length_scale = reference_vehicle_length / vehicle_length \\ length_shift = REF_VEH_FL_X - VEH_FL_X \cdot length_scale (\begin{array}{l} this is in case the scaling is different \\ for the front and back of the vehicle 510, if not, length_scale = 0; a width shift is not considered if \\ the vehicle 510 is symmetric with regards to the x axis \end{array}) \\ d 1 = REF_VEH_FL_X - VEH_FL_X \\ d 2 = REF_VEH_RR_X - VEH_RR_X \end{array}$

The y coordinates of the trajectory in the example of Fig. 5 can be adapted similarly. This is illustrated by Fig. 6. Fig. 6 shows an example of a vehicle 610 and two lines 621, 622 indicating the y ranges used for the adaptation of the y values of the coordinates. The line 621 separates the right side of the vehicle 610 from its center part, and the line 622 separates the left side of the vehicle 610 from its center part.
The translation and scaling in the y direction may be implemented by the following equations:

If y coordinate of a point of a trajectory is smaller than VEH_FL_Y (y coordinate of the left edge of the vehicle's outer borderline, i.e., the point is on the left): $y_Coordinate = y_Coordinate - d 3$
If y coordinate of the point is greater than VEH_RR_Y (y coordinate of the right edge of the vehicle's outer borderline, i.e., the point is on the right): $y_Coordinate = y_Coordinate + d 3$
Otherwise, i.e., the point is behind or in front of the vehicle 610: $y_Coordinate = y_Coordinate \cdot width_scale$
The values of d3 and width_scale can be determined as follows: $\begin{array}{l} reference_vehicle_width = REF_VEH_RR_Y - REF_VEH_FL_Y (\begin{array}{l} y coordinates of the right and left \\ edge of an assumed reference vehicle, respectively \end{array}) \\ vehicle_width = VEH_RR_Y_- VEH_FL_Y \\ width_scale = " reference_vehicle_width " / " vehicle_width " \\ d 3 = VEH_FL_Y - REF_VEH_FL_Y \end{array}$

Figs. 7a-7c illustrate diagrams 700 of three examples of normalizations of trajectory points 1-5 from a first vehicle (current car) to a second vehicle (reference car) with differing dimensions. The equations as described with reference to Fig. 5 and Fig. 6 are applied to realize the normalization. The diagrams 700 show the first and second vehicle's dimensions indicated as bounding boxes. The trajectory points 1-5 of the first vehicle are indicated by circles, while the adapted trajectory points 1-5 after normalization are indicated by stars.
For the example of Fig. 7a where the first vehicle is smaller than the second vehicle, a length scale of 2.0, d1 = -50.0, d2 = 150.0, and d3 = 50.0 are assumed. This results in the following normalization for the points 1-5:

(1) Original point (-50, 50) gets scaled to (-100.0, 100.0).
(2) Original point (-75, -20) gets scaled to (-125.0, -40.0).
(3) Original point (100, -60) gets scaled to (200.0, -110.0).
(4) Original point (200, 20) gets scaled to (350.0, 40.0).
(5) Original point (75, 70) gets scaled to (150.0, 120.0).

For the example of Fig. 7b where the first vehicle is bigger than the second vehicle, a length scale of 0.5, d1 = 50.0, d2 = -150.0, and d3 = -50.0 are assumed. This results in the following normalization for points 1 to 5:

(1) Original point (-100, 100) gets scaled to (-50.0, 50.0).
(2) Original point (-150, -20) gets scaled to (-100.0, -10.0).
(3) Original point (200, -120) gets scaled to (100.0, -70.0).
(4) Original point (350, 20) gets scaled to (200.0, 10.0).
(5) Original point (150, 120) gets scaled to (75.0, 70.0).

For the example of Fig. 7c where the first vehicle's front is bigger than the second vehicle's front, but the back is shorter, the following is assumed:

FL_X of first vehicle = 1.5 x FL_Y of second vehicle
RR_X of current car = 0.66 x RR_X of second vehicle
length scale = 1.14
width scale = 2
d1 = 50
d2 = 100
d3 = 50
length shift: 71.43

This results in the following normalization for points 1 to 5:

(1) Original point (-150, 50) gets scaled to (-100.0, 100).
(2) Original point (-170, -20) gets scaled to (-120, -40).
(3) Original point (-75, -60) gets scaled to (-14.29, -110).
(4) Original point (250, 20) gets scaled to (350, 40).
(5) Original point (150, 70) gets scaled to (242.86, 120).

Referring back to Fig. 1, the method 100 further comprises, if it is estimated by the trained machine-learning model that the flap is to be opened, triggering 140 the opening of the flap. For instance, the method 100 may comprise, if the machine-learning model has detected an intent to open the flap, sending a signal to an actuator for opening or unlocking the flap. This may be implemented through a remote trigger or through a wired connection between the method-executing device with the flap.
Fig. 8 illustrates a flowchart of an example of a method 800 for training a machine-learning model. The training of the machine-learning model may yield a trained machine-learning model as described above. The method 800 may be executed by a processing circuit.
The method 800 comprises obtaining 810 training data comprising at least one movement profile indicative of a trajectory of a user relative to a vehicle and at least one corresponding label indicative of whether a flap of the vehicle is to be opened, generating 820 an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle and training 830, based on the adjusted movement profile and the label, a machine-learning model to estimate whether the flap of the vehicle is to be opened.
The training 830 may be implemented by any training technique such as those described with reference to Fig. 1.
The adjustment of the movement profile may be an adaptation of the trajectory to match a reference or imaginary vehicle's dimensions, such as normalization as described above. That is, training data measured with various cars of different dimensions may be used and normalized (e.g., scaled) before using it as training input to the machine-learning model. This may simplify the training process since the machine-learning model does not have to learn handling different vehicle dimensions and their effect on the trajectory. The method 800 may also enable the usage of many vehicles of different dimensions, e.g., of a vehicle fleet, to produce the movement profiles for training which increases the ground truth and thus the accuracy of the machine-learning model.
Fig. 9 shows a block diagram of an example of an apparatus 30, e.g., for a vehicle 40. The apparatus 30 comprises interface circuitry 32 configured to communicate with at least one of a communication device, user equipment or backend to obtain a movement profile and processing circuitry 34 configured to perform one or more steps of a method as described above, e.g., the method for improving an opening of a flap as described with reference to Fig. 1 or the method for training a machine-learning model as described with reference to Fig. 8. The methods, for instance, the method for training a machine-learning model, may also be executed with the apparatus 30 being external to the vehicle 40, such as implemented in a backend or alike.
For example, the vehicle 40 may be a land vehicle, such as a road vehicle, a car, an automobile, an off-road vehicle, a motor vehicle, a bus, a robo-taxi, a van, a truck or a lorry. Alternatively, the vehicle 40 may be any other type of vehicle, such as a train, a subway train, a boat or a ship. For example, the proposed concept may be applied to public transportation (trains, bus) and future means of mobility (e.g., robo-taxis).
As shown in Fig. 9, the respective interface circuitry 32 is coupled to the respective processing circuitry 34 at the apparatus 30. In examples the processing circuitry 34 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. Similar, the described functions of the processing circuitry 34 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. The processing circuitry 34 is capable of controlling the interface circuitry 32, so that any data transfer that occurs over the interface circuitry 32 and/or any interaction in which the interface circuitry 32 may be involved may be controlled by the processing circuitry 34.
In an embodiment the apparatus 30 may comprise a memory and at least one processing circuitry 34 operably coupled to the memory and configured to perform the method described above.
In examples the interface circuitry 32 may correspond to any means for obtaining, receiving, transmitting or providing analog or digital signals or information, e.g. any connector, contact, pin, register, input port, output port, conductor, lane, etc. which allows providing or obtaining a signal or information. The interface circuitry 32 may be wireless or wireline and it may be configured to communicate, e.g., transmit or receive signals, information with further internal or external components.
The apparatus 30 may be a computer, processor, control unit, (field) programmable logic array ((F)PLA), (field) programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASICs), integrated circuits (IC) or system-on-a-chip (SoCs) system.
The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.
Examples may further be or relate to a (computer) program including a program code to execute one or more of the above methods when the program is executed on a computer, processor or other programmable hardware component. Thus, steps, operations or processes of different ones of the methods described above may also be executed by programmed computers, processors or other programmable hardware components. Examples may also cover program storage devices, such as digital data storage media, which are machine-, processor- or computer-readable and encode and/or contain machine-executable, processor-executable or computer-executable programs and instructions. Program storage devices may include or be digital storage devices, magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives, or optically readable digital data storage media, for example. Other examples may also include computers, processors, control units, (field) programmable logic arrays ((F)PLAs), (field) programmable gate arrays ((F)PGAs), graphics processor units (GPU), application-specific integrated circuits (ASICs), integrated circuits (ICs) or system-on-a-chip (SoCs) systems programmed to execute the steps of the methods described above.
It is further understood that the disclosure of several steps, processes, operations or functions disclosed in the description or claims shall not be construed to imply that these operations are necessarily dependent on the order described, unless explicitly stated in the individual case or necessary for technical reasons. Therefore, the previous description does not limit the execution of several steps or functions to a certain order. Furthermore, in further examples, a single step, function, process or operation may include and/or be broken up into several sub-steps, -functions, - processes or -operations.
If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method and vice versa. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.
The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.
The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.

References

30 apparatus
32 processing circuitry
34 interface circuitry
40 vehicle
100 method
110 obtaining
120 generating
130 estimating
140 triggering
210 vehicle A
220 vehicle B
231, 232 waypoints
241, 242 waypoints
300 method
310a first vehicle
310b second vehicle
311 first movement profile
312 second movement profile
320 generating
330 inputting
340 outputting
410 vehicle
420 vehicle
431-433 semantic points
441-443 semantic points
510 vehicle
520 vehicle
521-524 points of trajectory
530 dimensions of the vehicle
541, 542 range lines
610 vehicle
621, 622 range lines
700 diagram
800 method
810 obtaining
820 generating
830 training

Claims

A method (100) for improving an opening of a flap of a vehicle, comprising:
obtaining (110) a movement profile indicative of a trajectory of a user relative to the vehicle;

generating (120) an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle;

estimating (130), by a trained machine-learning model and based on the adjusted movement profile, whether the flap is to be opened; and

if it is estimated by the trained machine-learning model that the flap is to be opened, triggering (140) the opening of the flap.
The method (100) of claim 1, wherein generating (120) the adjusted movement profile comprises normalizing coordinates of the trajectory based on the dimensions of the vehicle.
The method (100) of any one of the previous claims, wherein the trained machine-learning model is trained based on training data measured with a reference vehicle, wherein generating (120) the adjusted movement profile comprises adjusting the trajectory based on dimensions of the reference vehicle.
The method (100) of any one of the previous claims, wherein generating (120) the adjusted movement profile comprises at least one of a translation and scaling of the trajectory or a part thereof based on the dimensions of the vehicle.
The method (100) of claim 4, wherein a point of the trajectory is translated if a value of its coordinates falls into a first range, and wherein a point of the trajectory is scaled if a value of its coordinates falls into a second range different from the first range.
The method (100) of claim 5, wherein the first range corresponds to an area extending in front or behind the vehicle or an area adjacent to a longitudinal side of the vehicle, and wherein the second range corresponds to the other one of the area extending in front or behind the vehicle or the area adjacent to the longitudinal side of the vehicle.
A method (800) for training a machine-learning model, comprising:
obtaining (810) training data comprising at least one movement profile indicative of a trajectory of a user relative to a vehicle and at least one corresponding label indicative of whether a flap of the vehicle is to be opened;

generating (820) an adjusted movement profile by adjusting the trajectory based on dimensions of the vehicle; and

training (830), based on the adjusted movement profile and the label, a machine-learning model to estimate whether the flap of the vehicle is to be opened.
An apparatus (30), comprising processing circuitry (32) configured to perform the method according to any of the previous claims.
A vehicle (40) comprising the apparatus according to claim 8.
A computer program having a program code for performing the method according to any one of claims 1 to 8, when the computer program is executed on a computer, a processor, or a programmable hardware component.