DE102022103821A1 - METHODS AND DEVICES FOR THE RADAR-ASSISTED DETECTION OF A SEAT OCCUPANCY STATE OF A SEAT ARRANGEMENT - Google Patents

Abstract

A method for automatically detecting a seat occupancy state of a seating arrangement with at least one seat comprises: receiving or generating measurement data which represents a radar point cloud obtained on the basis of a radar scan of a spatial region at least partially surrounding the seating arrangement and which has a plurality of radar points; for each feature from a defined, non-empty set of features for characterizing properties of radar point clouds, determining a value, in particular a numerical value, assigned to the respective feature as a function of the radar point cloud represented by the measurement data; Determining, in particular estimating, a seat occupancy state of the seating arrangement using a trained machine learning model, the determined values of the characteristics being made available to the machine learning model as input data in order to output a result characterizing a seat occupancy state of the seating arrangement, in particular classification result; and outputting information defined depending on the result. In particular, the information can represent the result itself. It can also be a detectable signal, in particular one that can be detected with a human sense, such as a warning, or a control signal for controlling a signal source, or a data signal carrying the information. A system for carrying out the method is also described.

Description

The present invention relates to a method based on machine learning and a system configured to carry out the method, in each case for automatically recognizing a seat occupancy state of a seating arrangement with at least one seat. Furthermore, the invention relates to a method for training a system for seat occupancy detection, in particular for a motor vehicle, with a machine learning model (hereinafter referred to as a "machine learning model" or "ML model" in accordance with a common term from the technical jargon) for detection a seat occupancy state of a seat arrangement with at least one seat. The machine learning model can be used in particular within the framework of the aforementioned method for recognizing a seat occupancy status.

In various situations, it may be necessary to automatically determine a current seat occupancy status of a seat arrangement with at least one seat. Such a situation can occur in particular in vehicles, for example in motor vehicles, where the vehicle is to be configured or one or more vehicle functionalities are to be activated, deactivated or controlled as a function of a current seat occupancy status. For example, it is known in motor vehicles to output an acoustic or visual indication to vehicle occupants to fasten seat belts or to control the activation or deactivation of airbags as a function of a detected seat occupancy state.

For the automated detection of a current seat occupancy status of one or more seats, in particular an arrangement of vehicle seats in a vehicle, so-called seat occupancy mats are known, which are integrated into the seats (usually one for each seat) and pressure-sensitive sensors for detecting an occupancy of the respective seat use. A seat occupancy state of the seat is determined as a function of the sensor signals or sensor data from these sensors, generally by means of a threshold value check.

These known solutions therefore require the seats to be equipped with a sensor system built into them and are also usually not able to distinguish between different seat occupancy states beyond “occupied” and “not occupied”. Furthermore, the sensors typically have to be integrated into the seats at the factory, so that retrofitting is difficult or impossible and there is no need to detect a seat occupancy status of seats that are not equipped in this way.

It is an object of the present invention to specify an improved solution for automatically recognizing a seat occupancy state of a seat arrangement with at least one seat.

The solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and developments of the invention are the subject matter of the dependent claims.

A first aspect of the solution presented here relates to a method, in particular a computer-implemented method, for training a system for seat occupancy detection for a motor vehicle with a machine learning model for detecting a seat occupancy state of a seat arrangement with at least one seat, in particular with at least one vehicle seat in one or for a vehicle. The method comprises: training the machine learning model using training data to obtain a trained machine learning model configured based on respective values of various features of a spatial area surrounding the seating arrangement by means of an actual or simulated radar scan obtained radar point cloud to obtain a seat occupancy state of the seating arrangement characteristic result. The training data for various possible predefined seat occupancy states of the seat arrangement each have at least one training data set that represents the respective values of N different features of a total set G for the respective seat occupancy state, which individually or cumulatively characterize a radar point cloud corresponding to the respective seat occupancy state.

The term “machine learning model” or “machine learning model” or “ML model” as used herein refers in particular to at least one machine learning algorithm based on sample data referred to as training data , created mathematical, in particular statistical, model for making predictions or decisions without the algorithm(s) being explicitly programmed for making such predictions or decisions. In particular, decision tree-based models for machine learning ("decision trees”) Machine learning models within the meaning of the invention.

In particular, the term "feature" as used herein means a reference to understand an attribute that can be determined on a radar point cloud, which is suitable for characterizing the radar point cloud with regard to at least one of its properties, in particular its spatial position or extent. Concrete examples of such features will be given below.

The term "seat occupancy status" of a seating arrangement with at least one seat, as used herein, is to be understood in particular as information that indicates whether or to what extent the seating arrangement or at least one of its seats is occupied or occupied by an object, in particular an item or a person .is occupied. In a simple example, the seat occupancy status can only indicate the presence or absence of an object, or in a more advanced example, in the case of the presence of at least one object on the seating arrangement or one or more of its seats, a statement on the type or another Specify a property, such as a spatial extent, of the object.

The term “radar point cloud” as used herein is to be understood in particular as meaning a set of points in a vector space obtained by means of radar scanning of at least one object surface, which has an unorganized spatial structure (“cloud”). In the case of a radar point cloud, the points of the radar point cloud can be referred to as "radar points". A (radar) point cloud can be described in particular by the (radar) points it contains, which are each recorded in particular by their spatial coordinates, which indicate for each radar point a location of the reflection of a radiated radar signal on an object surface measured during radar scanning. In addition to the radar points, attributes such as e.g. B. measured Doppler velocity or a signal-to-noise ratio (SNR) can be detected.

As used herein, the terms “comprises,” “includes,” “includes,” “has,” “has,” “having,” or any other variation thereof, as appropriate, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or has a list of elements is not necessarily limited to those elements, but may include other elements that are not expressly listed or that are inherent in such method or apparatus.

Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not to an exclusive "or". For example, a condition A or B is satisfied by one of the following conditions: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present).

As used herein, the terms "a" or "an" are defined to mean "one or more". The terms "another" and "another" and any other variant thereof shall be construed to mean "at least one other".

The term "plurality" as used herein means "two or more".

The term "configured" or "set up" to perform a specific function (and respective modifications thereof) is to be understood within the meaning of the invention that the corresponding device is already in a configuration or setting in which it can or can perform the function it is at least adjustable - i.e. configurable - so that it can carry out the function after the appropriate setting. The configuration can take place, for example, via a corresponding setting of parameters of a process flow or of switches or the like for activating or deactivating functionalities or settings. In particular, the device can have a plurality of predetermined configurations or operating modes, so that the configuration can take place by selecting one of these configurations or operating modes.

Using the method for training according to the first aspect, a system, in particular a data processing system, more precisely its machine learning model, can be trained so that after training it is able, depending on training data of the aforementioned type, on the basis of respective Values of various features of a radar point cloud obtained by means of an actual or simulated radar scan of a spatial area surrounding the seating arrangement to obtain a result characterizing a seat occupancy state of the seating arrangement (in particular in the sense of a prediction or classification). In particular in the vehicle context (especially for automobiles), radar-based solutions can be implemented on the basis of such trained machine learning models that can reliably detect a seat occupancy status (in particular exclusively) based on radar and on the basis of this certain functionalities or systems, such as a seat belt reminder or activate, deactivate, or control an airbag system, all or selectively.

Various exemplary embodiments of the method for Trai Kidneys described, each of which, unless this is expressly excluded or technically impossible, can be combined with each other and with the other described other aspects of the present solution.

In some embodiments, training the machine learning model comprises (i) selecting a subset T of M features from the total set G, where M<N, and selecting based on the respective values of the N features of the total set represented in the training data G occurs; and (ii) adjusting the machine learning model in such a way that it relies solely on the selected M features of the subset T for its dependency on features of the total set G after the adjustment. Depending on the type of machine learning model used, the number M can be specified in advance, or only result during the training itself, the latter in particular when using a decision tree-based machine learning model.

In these embodiments, therefore, a subset of the overall available features is selected, so that the number of features to be taken into account in the trained machine learning model resulting from the training is reduced. This can serve to improve the performance of the machine learning model and thus of the method described below for automatically recognizing a seat occupancy state of a seating arrangement. The gain in performance can relate to a reduction in the execution time of the method or a reduction in the computing resources required for its execution and/or to the quality of the result. The latter can be caused in particular by the fact that, due to the reduction or limitation of the number of features to be taken into account, a consideration of features that lead to an increase in complexity and distraction of the machine learning model from the actually particularly decisive features without making a major contribution to solving the problem could be counteracted. In particular, so-called "overfitting", i.e. over-adaptation to the training data used, can be counteracted effectively.

In some embodiments, training the machine learning model further comprises training the already adjusted machine learning model based on the training data regarding the selected M features of the subset T. In this way, the quality of the machine learning model, in particular with regard to the reliability of its results, can be further increased, since the model, which was limited by the previous selection of M features, is trained again as such and is thus optimized.

In some embodiments (“Group 1”), the subset T of features is selected iteratively from the total set G. With each iteration run: (i) starting from a starting set that contains a predetermined preselection of K of the N features of the total set G with 0 ≤ K ≤ M, each of the other N-K features of the total set G is initially assigned temporarily as a candidate feature of the starting set , so as to form a test set P with K+1 features; (ii) the machine learning model is trained with values given by the training data for each training data set for the K+1 features of the test set P and the associated seat occupancy status; (iii) the machine learning model trained in this way is validated on the basis of validation data that corresponds to the type of training data but is at least partially different from the latter and is thereby evaluated using at least one predetermined score, e.g. accuracy score or F-score (or . F1 score), is evaluated. In particular, the preselection can be an empty set for the first iteration run, i.e. K = 0.

Furthermore, depending on the respective scores of the various candidate characteristics of the test sets P that occurred during the iteration, one of the candidate characteristics is determined as a selection candidate and the selection candidate is checked based on its score to determine whether it meets a defined inclusion criterion. If this is the case, the selection candidate is included as a new element in the subset T and a new iteration begins without considering the now selected selection candidate as a candidate feature, while otherwise the iteration and thus the definition of the subset T is completed. The starting set for the first iteration is predetermined, while the starting set of each subsequent iteration is determined by the starting set of the iteration immediately preceding it plus its candidate choice.

In this way, the M features of the subset are gradually determined as part of the iteration, depending on their relevance for the machine learning model or its quality, for which purpose the scores and the inclusion criterion for selecting the M features are used.

In some embodiments of group 1, the machine learning model includes an ensemble with a plurality L of mutually similar individual machine learning models, eg Ensemble of decision trees based on support vector machine or random forest. The training of the machine learning model includes a corresponding training of the L individual machine learning models for each test set, the training of the L individual machine learning models differing in that for at least two, in particular each, the L individual Machine learning models each use a subset of the training data, with the subsets for the at least two individual machine learning models differing from one another. Due to the different subsets, the resulting individual machine learning models also differ, at least as a rule. Each of the L individual machine learning models trained in this way is validated on the basis of validation data which, in terms of their type, corresponds to the training data but at least partially differs from this and is thereby evaluated using at least one individual score. For the resulting distribution of the L individual scores of the individual machine learning models, the respective value for at least one statistical measure that characterizes the distribution is determined, and among all selection candidates of the test sets, an individual selection candidate is selected depending on the one or more values assigned to it of the at least one measure as a candidate for the subset T. Additionally or alternatively, the machine learning models can also be trained using so-called boosting, in which the training data is weighted differently.

These embodiments are therefore ensemble-based and are therefore suitable, due to the associated larger training basis (diversity of the individual models), for further increasing the quality of the machine learning model that can be achieved using the method for training.

In some of the ensemble-based embodiments, a mean value of the distribution is determined as a characteristic statistical measure of the distribution of the individual scores and that selection candidate is selected as a candidate for the subset T whose mean value is the best according to a definition of the score, in particular the largest depending on the definition of the score or smallest, value among all means of the selection candidates. In this way, from the set of selection candidates determined by the ensemble, a single selection candidate can be determined in a simple and effective manner as a candidate for the subset T, which also corresponds to an optimized selection.

In some embodiments of group 1, the inclusion criterion is or will be defined as a function of a confidence interval, in particular a Wald confidence interval. In particular, this can be done in such a way that the candidate to be potentially included is only included in the subset T if its inclusion results in a minimum improvement defined by a confidence interval being able to be achieved.

In some embodiments, which can be used in particular as an alternative to the aforementioned embodiments as part of the method for selecting or reducing features (“group 2”), the selection of the features of the subset T from the total set G includes: (i) grouping of features of the total set G into different groups such that the features of each group are mutually correlated with respect to the training data and a certain correlation measure; and (ii) selecting one feature from each group as a representative of the group, the subset T being formed only from representatives. These embodiments can be implemented in a particularly computationally efficient manner, since in particular no iteration is required.

In these embodiments, the method may further include: (i) training an ensemble of different individual machine learning models of the same type based on training vectors extracted from the training data and having only representatives as features; (ii) the training of the individual machine learning models differs in that a subset of the training data is used for at least two, in particular each, of the individual machine learning models and the subsets differ from one another. These embodiments are therefore ensemble-based and are therefore suitable, due to the associated larger training basis (diversity of the individual models), for further increasing the quality of the machine learning model that can be achieved using the method for training.

In some of these embodiments, when training the individual machine learning models as part of the training, a selection is made as to which of the representatives are to be included as a feature in the respective individual machine learning model, and these recorded features are weighted such that in As part of the training of the individual machine learning models, each initial inclusion of a feature in one of these machine learning models that has not yet been included by any of the individual machine learning models is linked to a weighting that makes such an inclusion more difficult than an inclusion of a feature already used becomes. Only those of the representatives in the part quantity T, which was included as a feature in more than x% of the individual machine learning models, where x is a fixed or variably adjustable parameter with 30 ≤ x ≤ 70, in particular with x = 50. This makes it difficult to add new features or "expensive", so that the machine learning model resulting from the ensemble is optimized after training at a given quality level with regard to the smallest possible number of different features, which is particularly advantageous with regard to high processing efficiency.

In some embodiments (applicable in particular to both groups 1 and 2), different training sets from the training data are taken into account to different degrees depending on the seat occupancy status assigned to them according to a predetermined weighting of the seat occupancy statuses when training the machine learning model. In particular, the training can be particularly strong with regard to selected seat occupancy states and/or inequalities in the frequency of different training sets per seat occupancy state can be compensated, in particular proportionally.

A second aspect of the present solution relates to a method, in particular a computer-implemented one, for the automated detection of a seat-occupancy status, in particular a seat-related one, in a seat arrangement with at least one seat, in particular a seat in or for a vehicle, such as an automobile (e.g. truck, car or bus). The method has: (i) receiving or generating measurement data which represent a radar point cloud obtained on the basis of a radar scan of a spatial region which at least partially surrounds the seating arrangement and which has a plurality of radar points; (ii) for each feature from a defined, non-empty set of features for characterizing properties of radar point clouds, determining a value, in particular a numerical value, assigned to the respective feature as a function of the radar point cloud represented by the measurement data; (iii) Determining, in particular estimating, a seat occupancy state of the seating arrangement using a trained machine learning model, the determined values of the characteristics being made available to the machine learning model as input data in order to output a seat occupancy state of the seating arrangement obtain characteristic result, in particular classification result; and (iv) outputting information defined depending on the result. In particular, the information can represent the result itself. It can also be a detectable signal, in particular one that can be detected with a human sense, such as a warning, or a control signal for controlling a signal source, or a data signal carrying the information.

Various exemplary embodiments of the method according to the second aspect are described below, each of which can be combined with one another and with the other aspects of the present solution described herein, unless expressly excluded or technically impossible.

In some embodiments, the seat arrangement has a plurality of seats for which a seat occupancy state is to be determined individually or cumulatively as part of the method. The set of radar points in the radar point cloud is divided into several clusters, each containing a subset of the radar points, by means of cluster formation depending on the respective spatial position of the radar points in relation to the seats, in order to individually assign one of the clusters closest to it to each of the seats, in particular this way , that each radar point is assigned to exactly one cluster. The respective values of the features for each of the seats are determined individually on the basis of the cluster of radar points assigned to the respective seat. Determining the seat occupancy status of the seating arrangement includes individually determining a respective individual seat occupancy status for each of the seats using the trained machine learning model, with the respective values of the features determined for the cluster assigned to the respective seat as input data for the machine learning model be made available in order to obtain as its output a result characterizing a seat occupancy state of the respective seat, in particular a classification result. The information to be output is defined depending on the respective individual results for the different seats. In this way, even in the case of seating arrangements with several seats, individual statements can be obtained for each seat with regard to its respective seat occupancy status.

In some embodiments, the radar point cloud is segmented into several clusters by assigning a subset of the radar points to each of the seats as a cluster depending on their respective position, in particular clearly for each radar point, so that the radar points of the cluster are in a defined, closed, in particular cuboid, spatial area in the vicinity of the seat. The assignment can be done in particular so that each radar point cluster of the seat closest to it is assigned. The radar point cloud can be divided into clusters, ie subsets of the radar point cloud located in the vicinity of the respective seats, so that seat-specific seat occupancy states can be determined in a targeted manner and therefore with high reliability on the basis of the cluster assigned to the respective seat.

In some embodiments, the outputting of the information includes driving a signal source depending on the information in order to cause the signal source to output a defined signal depending on the driving. The signal source can in particular be an audio source, an optical signal source, in particular a display device for image or text, and/or a haptic actuator or a combination of at least two of the aforementioned signal sources. Based on the signaling, the recognized seat occupancy status can be communicated to a user or used to control another technical system, such as an airbag system.

In some of these embodiments, the signal source is controlled as a function of the information in such a way that it emits a signal, defined in particular by the control, if the information results from a result according to which at least one seat in the seating arrangement is occupied and/or a selected predetermined one Seat occupancy status is present.

In some embodiments, the method further comprises: (i) detecting a seat belt fastening state of at least one seat of the seating arrangement or receiving seat belt information characterizing this seat belt fastening state; (ii) the signal source being controlled as a function of the seat belt information and the information from the result in such a way that it emits a seat belt notice signal if, according to the information, at least one seat in the seating arrangement is occupied and/or a selected predetermined seat occupancy state is present and seat belt information is given, that the associated seat belt of the seat is not fastened. In this way, a radar-based seat belt check and warning system can be achieved, particularly with regard to detection.

In some embodiments, the individual radar points of the radar point cloud are each represented by one or more of the following data: (i) a position of the respective radar point in three-dimensional space; (ii) a Doppler shift value of the radar signal at the respective radar point; (iii) a signal-to-noise ratio value of the radar signal to the respective radar point.

In particular, the position of the respective radar point according to option (i) can be used to determine the features, while options (ii) and (iii) can be used in particular to pre-filter the radar point cloud as part of a pre-processing preceding the feature determination. Alternatively or additionally, options (ii) and (iii) can also be used to determine features.

In some embodiments, the machine learning model is or will be trained using the method of training according to the first aspect. In particular, the two methods according to the first aspect and the second aspect can also be combined into a single method, in particular in such a way that this combined method can be configured, depending on the situation, in a training mode corresponding to the method according to the first aspect or in a Method according to the second aspect to run corresponding detection mode.

Various exemplary embodiments are described below, which can each be used for the method according to the first aspect and/or for the method according to the second aspect, unless this is expressly excluded or is technically impossible.

• - Merkmale, die eine räumliche Lage eines Clusters aus Radarpunkten im räumlichen Umfeld eines Sitzes der Sitzplatzanordnung beschreiben;
• - Merkmale, die die Anzahl der Radarpunkte der Radarpunktwolke oder eines definierten Ausschnitts davon beschreiben;
• - Merkmale, die für eine bestimmten Cluster aus Radarpunkten im räumlichen Umfeld eines Sitzes der Sitzplatzanordnung beschreiben, welchem Sitz der Sitzplatzanordnung der Cluster zugeordnet ist;
• - Merkmale, die Doppler-Verschiebungswerte und/oder einen Signal-zu-Rausch-Abstands-Wert des Radarsignals zu einem jeweiligen Radarpunkt der Radarpunktwolke beschreiben;
• - Merkmale, die einen quantitativen Vergleich von zwei oder mehr Cluster aus Radarpunkten der Radarpunktwolke beschreiben;
• - Merkmale, die eine Orientierung und/oder räumliche Ausdehnung eines Clusters aus Radarpunkten der Radarpunktwolke beschreiben;
• - Merkmale, welche die Radarpunktwolke insgesamt beschreiben.

In some of these embodiments, the set of features used in the machine learning model, in particular the total set G or set C, contains at least one feature that falls into one of the following feature categories:

• - Features that describe a spatial position of a cluster of radar points in the spatial environment of a seat of the seating arrangement;
• - Characteristics that describe the number of radar points in the radar point cloud or a defined section thereof;
• - Characteristics that describe, for a specific cluster of radar points in the spatial environment of a seat of the seating arrangement, which seat of the seating arrangement is assigned to the cluster;
• - features that describe Doppler shift values and/or a signal-to-noise ratio value of the radar signal to a respective radar point of the radar point cloud;
• - features describing a quantitative comparison of two or more clusters of radar points of the radar point cloud;
• - Features that indicate an orientation and/or spatial extent of a cluster describe radar points of the radar point cloud;
• - Features that describe the radar point cloud as a whole.

In some embodiments, the result for the seating arrangement or a respective seating position thereof is or will be defined as indicating a class according to a predetermined classification of possible seat occupancy possibilities.

• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist unbelegt, insbesondere wenn kein einziger ihrer Sitze belegt ist;
• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist belegt, insbesondere wenn zumindest ein Sitzplatz belegt ist oder wenn alle belegt sind oder zumindest ein bestimmter Anteil der Sitze;
• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist mit einer Person belegt;
• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist mit einer Person mit einer Größe oberhalb einer Größenschwelle belegt;
• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist mit einer Person mit einer Größe unterhalb einer Größenschwelle belegt;
• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist mit einer Sache belegt;
• - Sitzplatzanordnung bzw. der jeweilige Sitzplatz ist mit einem Kindersitz, insbesondere einer Kindertrageschale, belegt.

In particular, the classification can be or become predetermined in such a way that it contains at least two of the following classes:

• - Seating arrangement or the respective seat is unoccupied, especially if not a single one of their seats is occupied;
• - Seating arrangement or the respective seat is occupied, in particular if at least one seat is occupied or if all are occupied or at least a certain proportion of the seats;
• - Seating arrangement or the respective seat is occupied by one person;
• - Seating arrangement or the respective seat is occupied by a person with a size above a size threshold;
• - Seating arrangement or the respective seat is occupied by a person below a height threshold;
• - Seating arrangement or the respective seat is occupied by one thing;
• - Seat arrangement or the respective seat is occupied by a child seat, in particular a child carrier.

In some embodiments, the machine learning model is based on at least one decision tree, in particular with regard to the ensemble embodiments mentioned above, on an ensemble of decision trees.

A third aspect of the present solution relates to a system for automatically detecting a, in particular respective, seat occupancy state of a seating arrangement with at least one seat, in particular with at least one vehicle seat in or for a vehicle, the system having a data processing device that is configured to Carrying out the method for training according to the first aspect and/or the method according to the second aspect.

A fourth aspect of the present solution relates to a computer program or computer program product having instructions which, when executed on the data processing device of the system according to the third aspect, cause the system to carry out at least one of the methods according to the first aspect and/or according to the second aspect.

The computer program can in particular be stored on a non-volatile data medium. This is preferably a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program as such is to be traded independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can be present as a file on a data processing unit, in particular on a server, and can be downloaded via a data connection, for example the Internet or a dedicated data connection, such as a proprietary or local network. In addition, the computer program can have a plurality of interacting individual program modules. In particular, the modules can be configured or at least used in such a way that they are executed in the sense of distributed computing on different devices (computers or processor units that are geographically spaced apart and connected to each other via a data network.

The system according to the third aspect can accordingly have a program memory in which the computer program is stored. Alternatively, the system can also be set up to access a computer program available externally, for example on one or more servers or other data processing units, via a communication connection, in particular in order to exchange data with it that are used during the course of the method or computer program or outputs of the computer program represent.

A fifth aspect of the present solution relates to a vehicle, comprising: (i) a seating arrangement with at least one seating space; (ii) a radar sensor for at least partial radar scanning of the seating arrangement; and (iii) a system according to the second aspect for the automated detection of an, in particular respective, seat occupancy state of the seating arrangement as a function of at least partial radar scanning of the seating arrangement carried out by the radar sensor.

The features and advantages explained in relation to the first aspect and the second aspect of the invention also apply correspondingly to the further aspects of the invention.

Further advantages, features and application possibilities of the present invention result from the following detailed description in connection with the figures.

• 1A schematisch eine beispielhafte Ausführungsform eines Fahrzeugs, das mit einem System zum automatisierten Erkennen eines Sitzbelegungszustands einer Sitzplatzanordnung im Fahrzeug ausgerüstet ist;
• 1 B schematisch das Fahrzeug aus 1A, wobei hier auf dem Beifahrersitz als zu erkennendes Objekt eine Babyschale mit einem darin liegenden Baby vorhanden ist;
• 2 ein Flussdiagramm zur Veranschaulichung einer beispielhaften Ausführungsform eines kombinierten Verfahrens (i) zum automatisierten Erkennen eines Sitzbelegungszustands einer Sitzplatzanordnung sowie (ii) zum Trainieren eines dazu verwendeten Machine-Learning-Modells;
• 3A eine beispielhafte zweidimensionale Darstellung einer durch einen Radarsensor des Fahrzeugs aus 1 aufgenommenen Radarpunktwolke;
• 3B eine beispielhafte Darstellung einer Clusterung der akkumulierten Radarpunktwolke Ausführung 3B gemäß den Positionen der einzelnen Sitzplätze der Sitzplatzanordnung;
• Die 4 eine Tabelle zur Illustration einer beispielhaften Struktur von Trainingsdaten zum Trainieren eines Maschine-Learning Modells gemäß einem lösungsgemäßen Trainingsverfahren, insbesondere gemäß dem Verfahren zum Trainieren aus 2;
• Die 5 einen Pseudocode zur Illustration einer ersten beispielhaften Ausführungsform des Verfahrens zum Trainieren aus 2; und
• Die 6 ein Flussdiagramm zur Veranschaulichung einer zweiten beispielhaften Ausführungsform des Verfahrens zum Trainieren aus 2.

while showing

• 1A schematically shows an exemplary embodiment of a vehicle that is equipped with a system for automatically recognizing a seat occupancy state of a seating arrangement in the vehicle;
• 1 B schematic of the vehicle 1A , with a baby seat with a baby lying in it being present on the front passenger seat as an object to be recognized;
• 2 a flowchart to illustrate an exemplary embodiment of a combined method (i) for automated detection of a seat occupancy state of a seating arrangement and (ii) for training a machine learning model used for this purpose;
• 3A an exemplary two-dimensional representation of a radar sensor of the vehicle 1 recorded radar point cloud;
• 3B an exemplary representation of a clustering of the accumulated radar point cloud embodiment 3B according to the positions of the individual seats of the seating arrangement;
• The 4 a table to illustrate an exemplary structure of training data for training a machine learning model according to a solution-based training method, in particular according to the method for training 2 ;
• The 5 a pseudo code for illustrating a first exemplary embodiment of the method for training 2 ; and
• The 6 a flowchart to illustrate a second exemplary embodiment of the method for training 2 .

In the figures, the same reference numbers denote the same, similar or corresponding elements. Elements depicted in the figures are not necessarily drawn to scale. Rather, the various elements shown in the figures are presented in such a way that their function and general purpose can be understood by those skilled in the art. Unless expressly stated otherwise, connections and couplings between functional units and elements illustrated in the figures can also be implemented as indirect connections or couplings. In particular, functional units can be implemented as hardware, software or a combination of hardware and software.

In the 1 schematically illustrated exemplary embodiment of a vehicle 100 has a seating arrangement 105 with five individual seats or seats 105a to 105e. Each of the seats 105a to 105e is suitable for accommodating one person as a passenger of the vehicle 100 . The vehicle 100 further includes a radar sensor 110 mounted within the vehicle cabin on the ceiling thereof and configured to, at least substantially, scan the seating arrangement 105 using radar beams. Accordingly, the seats 105a to 105e, in particular their seat surfaces, are at least predominantly located within an observation field 110a that can be scanned by the radar sensor 110. In addition, vehicle 100 has a system 115 for automatically recognizing a seat occupancy state of seating arrangement 105 as a function of a radar scan of seating arrangement 105 performed by radar sensor 110 at least in sections with respect to observation field 110a.

The system 115 has in particular a data processing unit 115a with at least one microprocessor and a memory 115b signal-connected thereto, in which a memory 115b for carrying out the below with reference to 4 described method for automatically detecting a seat occupancy state of the seat arrangement 105 configured computer program is stored. Furthermore, the sensor data generated by the radar sensor 110 during the radar scan or information already obtained therefrom by further processing can be or will be stored in the memory 115b.

This in 1B illustrated vehicle 100 corresponds to the vehicle 1 , but here on the front passenger seat 105b as an object to be recognized in terms of a seat occupancy check, a baby seat B with a baby lying therein is arranged. In the further following discussion of the other figures, particular attention is paid to the constellation 1 B be referred to.

This in 2 Method 200 illustrated using a flowchart relates to an exemplary combined embodiment both for the method for training according to the first aspect and for the method according to the second aspect of the present solution for automatically recognizing a seat occupancy state of a seat arrangement. The method can be used in particular as a compu term-implemented method be formed. This can be done in particular in the memory 115b of the system 115 1 be stored as a computer program and be executable on the data processing unit 115a. This applies in particular to those parts of the method 200 that are required for the automated detection of a seat occupancy status. The method for training that is also presented in the method 200 can optionally also be carried out elsewhere, for example as part of the development of a system 115, so that it is then already equipped with a trained machine learning model (abbreviated ML model). can be delivered or integrated into a vehicle.

As part of method 200, measurement data PC (for “point cloud”), which were recorded by radar sensor 110 during a radar scan of seating arrangement 105 in observation field 110a, are received in a process 205. The measurement data represent a radar point cloud 300 from a plurality, in particular a large number, of radar points 310, 315, as in 3A illustrated (2D view of the actually three-dimensional radar point cloud 300). Each of the radar points 310, 315 can be represented in the measurement data with its three spatial coordinates in a suitably chosen coordinate system. Additional information can also be assigned to each radar point 310, 315, in particular a respective measured Doppler shift and/or a distance from radar sensor 110 along a straight line connecting radar sensor 110 and the respective radar point 310, 315.

In a further process 210, the radar point cloud 300 represented by the measurement data PC is processed further in order to perform a clustering of the radar point cloud 300, in which each seat 105a to 105e of the seating arrangement 105 is assigned its own cluster as a subset of all points in the radar point cloud 300. This is more accurate in 3B illustrated. To carry out the clustering, each of the seats 105a to 105e is assigned a respective closed, for example cuboid, spatial area 325a to 325e and the clusters are defined in such a way that all radar points 310/315, which are located within each of these spatial areas 325a to 325e, the be assigned to the corresponding cluster for the respective spatial area. The assignment of the radar points 310/315 to their respective cluster can be represented by appropriately defined cluster data CL. It can also be as in 3B illustrates that some of the radar points 310/315 do not fall within any of the spatial regions 325a through 325e and are therefore not assigned to any of the clusters. They are no longer taken into account in the further evaluation of the radar point cloud 300 as part of the method 200 . These radar points 310/315, which do not belong to any cluster, can also optionally be represented in the cluster data CL.

Below is again for the purpose of a more detailed explanation on the 3A and 3B Referred to, which each represent the radar point cloud 300, wherein for the purpose of displayability, the respective three-dimensional radar point cloud 300 was reduced to two dimensions by projecting the positions of the radar points 310/315 of the radar point cloud 300 onto a plane spanned by two of its dimensions.

In 3A an exemplary radar point cloud 300 is illustrated as it was recorded within a single measurement frame, ie as the result of a radar scan of the seating arrangement 105 by the radar sensor 110 during a specified time interval (measurement period). The position of the individual radar points within the radar point cloud 300 can be represented by spatial coordinates, for example Cartesian coordinates X and Y can be assigned to the plane of the drawing and corresponding to each individual point. In fact, if the dimension reduction due to the drawing is disregarded, in reality there is still a third coordinate Z for the third spatial dimension.

If, during radar scanning, not only the spatial positions of the points at which the radar beam is reflected by the scanned objects are recorded as coordinates, but also a respective Doppler shift is measured, then the individual radar points can be classified depending on the amount of this Doppler shift, in particular be divided into two different classes. The latter can be done, for example, by comparing the Doppler shift with a predefined shift threshold that corresponds to a specific shift speed. Depending on the result of the comparison, those radar points 310 that, according to the value of their associated Doppler shift, have no velocity or a velocity of the object surface at the reflection point that is below the shift wave, can be classified as "static" radar points (in the 3A and 3B each shown with a filled black circle). Conversely, those radar points 315 that show a Doppler shift above the shift threshold can be classified as "dynamic" radar points 315 (in the 3A and 3B each shown with a black ring).

The classification of the radar points 310 and 315 according to their Doppler shift is not absolutely necessary, but it can be used to process the radar point cloud 300, in particular as part of a pre-processing that takes place before its evaluation, in particular to filter depending on the classification. For example, such filtering could take place in such a way that only dynamic radar points 315 are taken into account for the evaluation, in order to only detect moving objects, for example.

In 3B is the same radar point cloud 300 as in 3B shown. In addition, however, cuboid (3D case) or, in the present 2D representation, rectangular, selected spatial regions 325a to 325e are drawn in, which are spatially assigned to the respective position of the individual seats 105a to 105e. The definition of these spatial regions 325a to 325e can now be used, as already described above, to cluster the accumulated radar point cloud 300, with each radar point 310 or 315 being assigned, as far as possible, to that spatial region 325a to 325e in which it is located. All radar points that are not in one of the spatial areas 325a to 325e can remain unconsidered. One sees in particular that in the present example the areas 320 with a particularly high radar point density are in the area of the passenger seat 105b, on which according to FIG 1 B the infant carrier with baby B is located.

The clustering according to the process 210 represents an optional step of the method 200, which is used particularly expediently when it comes to determining a seat-specific seat occupancy status for one or more individual seats 105a to 105e. If, on the other hand, it is only a question of determining a seat occupancy state for the seating arrangement 105 as a whole, for example to determine whether at least one of the seats is occupied at all, then such a clustering is not necessary and the radar point cloud 300 can instead be subjected to further processing as a whole . In the following further description of the method 200 it is assumed that the clustering has taken place in accordance with the process 210 .

1. a) Merkmale, die eine räumliche Lage eines Clusters aus Radarpunkten im räumlichen Umfeld eines Sitzes der Sitzplatzanordnung beschreiben;
2. b) Merkmale, die die Anzahl der Radarpunkte der Radarpunktwolke oder eines definierten Ausschnitts davon beschreiben;
3. c) Merkmale, die für eine bestimmten Cluster aus Radarpunkten im räumlichen Umfeld eines Sitzes der Sitzplatzanordnung beschreiben, welchem Sitz der Sitzplatzanordnung der Cluster zugeordnet ist;
4. d) Merkmale, die Doppler-Verschiebungswerte und/oder einen Signal-zu-Rausch-Abstands-Wert des Radarsignals zu einem jeweiligen Radarpunkt der Punktwolke beschreiben;
5. e) Merkmale, die einen quantitativen Vergleich von zwei oder mehr Cluster aus Radarpunkten der Radarpunktwolke beschreiben;
6. f) Merkmale, die eine Orientierung und/oder räumliche Ausdehnung eines Clusters aus Radarpunkten der Radarpunktwolke beschreiben;
7. g) Merkmale, welche die Radarpunktwolke insgesamt beschreiben.

In a further process 215 (unless otherwise stated individually for each of the clusters) different features of a defined set of features are now determined on the basis of the cluster data CL, in particular calculated. In particular, at least some of the features of the sentence may each fall into one of the following feature categories:

1. a) features that describe a spatial position of a cluster of radar points in the spatial environment of a seat in the seating arrangement;
2. b) features that describe the number of radar points in the radar point cloud or a defined section thereof;
3. c) Characteristics which, for a specific cluster of radar points in the spatial vicinity of a seat in the seating arrangement, describe which seat in the seating arrangement the cluster is assigned to;
4. d) features that describe Doppler shift values and/or a signal-to-noise ratio value of the radar signal to a respective radar point of the point cloud;
5. (e) characteristics describing a quantitative comparison of two or more clusters of radar points of the radar point cloud;
6. f) features that describe an orientation and/or spatial extent of a cluster of radar points of the radar point cloud;
7. g) Features that describe the radar point cloud as a whole.

• Zu a): Merkmale, die die Lage des Clusters innerhalb des jeweiligen quaderförmigen Raumbereichs 325a-325e beschreiben (die Koordinaten werden relativ zum Raumbereich angegeben):
  • - Koordinaten x, y und z des Schwerpunkts des Clusters
  • - Koordinaten x, y und z des gewichteten Mittelpunkts des Clusters, wobei jeder Punkt mit seinem Signal-zu-Rausch-Verhältnis (SNR)-Wert gewichtet ist/wird.
  • - Abstand des Schwerpunkts zum linken, vorderen und oberen Rand des Raumbereichs
  • - Abstand des Schwerpunkts und des SNR-gewichteten Zentrums zur Mitte der hinteren und unteren Ebene des Raumbereichs
  • - Höhe des Clusters
• Zu b): Merkmale, die die Anzahl der Radarpunkte beschreiben:
  • - Anzahl der Radarpunkte
  • - Anzahl der statischen Radarpunkte (Punkte mit einem absoluten Dopplerwert unter 0,005 m/s, Radarpunkte mit einem Doppler Wert von mindestens 0,005 m/s werden als dynamische Punkte bezeichnet)
  • - Jeweilige Anzahl der Radarpunkte im Messrahmen, welcher vorzugsweise aus einzelnen zeitlich aufeinanderfolgenden, optional verschachtelten, Subframes besteht, insbesondere mit unterschiedlicher Dauer der Radarmessung von beispielsweise 128 ms, 256 ms, 512 ms, 1024ms oder 2048 ms und einer optionalen Anzahl von fünf Subframes.
• Zu c): Merkmal, das beschreibt, zu welchem Sitz der Cluster gehört:
  • - Kennung des jeweiligen Raumbereichs oder Sitzes
• Zu d): Merkmale zur Beschreibung der Doppler- und SNR-Werte
  • - Minimum, Maximum, Mittelwert, Varianz und Mittelwert der absoluten Dopplerwerte für den gesamten Messrahmen und für seine Subframes
  • - Minimum, Maximum, Mittelwert und Varianz der SNR-Werte
• Zu e): Merkmale im Vergleich zum nächstgelegenen Cluster:
  • - Anzahl der Radarpunkte im Verhältnis zum nächstgelegenen Cluster
  • - Abstand des Schwerpunkts zum Schwerpunkt des nächstgelegenen Clusters
• Zu f): Merkmale, die die Orientierung und Ausdehnung des Clusters beschreiben:
  • - PCA-Eigenwerte (PCA=engl. „principal component analysis“ bzw. auf Deutsch: Hauptkomponentenanalyse)
  • - Winkel des ersten PCA-Eigenvektors
  • - Koordinaten der PCA-Eigenvektoren
  • - Relation der PCA-Eigenwerte
  • - Radarpunktdichte
  • - Anteil der Punkte in diskreten Raumbereichen im Sitzbereich des Sitzes, wobei vorzugsweise dazu der Sitzbereich in gleichgroße Raumbereiche unterteilt wird und die jeweilige Anzahl der Punkte in den Raumbereichen ausgewertet wird.
• Zu g): Merkmale, die die gesamte Radarpunktwolke beschreiben (nicht nur das aktuelle Cluster)
  • - Anzahl aller Radarpunkte in dem aktuellen Messrahmen
  • - Merkmale der dynamischen Punkte: die oben genannten Merkmale werden auch nur für die dynamischen Punkte ausgewertet (alle außer Anzahl der statischen Punkte und Kennung des jeweiligen Raumbereichs oder Sitzes)

In particular, the following concrete characteristics come into question:

• Regarding a): Features that describe the position of the cluster within the respective cuboidal spatial area 325a-325e (the coordinates are given relative to the spatial area):
  • - x, y and z coordinates of the center of gravity of the cluster
  • - Coordinates x, y and z of the weighted center of the cluster, where each point is weighted with its signal-to-noise ratio (SNR) value.
  • - Distance of the center of gravity to the left, front and top edge of the space area
  • - Distance of the center of gravity and the SNR-weighted center to the middle of the back and bottom plane of the spatial area
  • - Cluster height
• Re b): Characteristics that describe the number of radar points:
  • - Number of radar points
  • - Number of static radar points (points with an absolute Doppler value below 0.005 m/s, radar points with a Doppler value of at least 0.005 m/s are called dynamic points)
  • - The respective number of radar points in the measurement frame, which preferably consists of individual, optionally interleaved, subframes that follow one another in time, in particular with different durations of the radar measurement of, for example, 128 ms, 256 ms, 512 ms, 1024ms or 2048 ms and an optional number of five subframes.
• Regarding c): characteristic that describes the seat to which the cluster belongs:
  • - Identification of the respective room area or seat
• Re d): Characteristics for describing the Doppler and SNR values
  • - Minimum, maximum, mean, variance and mean of the absolute Doppler values for the entire measurement frame and for its subframes
  • - Minimum, maximum, mean and variance of the SNR values
• Regarding e): Characteristics compared to the nearest cluster:
  • - Number of radar points in relation to the nearest cluster
  • - Distance of the centroid to the centroid of the nearest cluster
• To f): Characteristics that describe the orientation and extent of the cluster:
  • - PCA eigenvalues (PCA = "principal component analysis" or in German: main component analysis)
  • - Angle of the first PCA eigenvector
  • - Coordinates of the PCA eigenvectors
  • - Relation of the PCA eigenvalues
  • - Radar point density
  • - Percentage of points in discrete spatial areas in the seating area of the seat, with the seating area preferably being divided into spatial areas of equal size for this purpose and the respective number of points in the spatial areas being evaluated.
• To g): features that describe the entire radar point cloud (not just the current cluster)
  • - Number of all radar points in the current measuring frame
  • - Characteristics of the dynamic points: the above characteristics are also only evaluated for the dynamic points (all except number of static points and identifier of the respective room area or seat)

The result of the feature determination in the process 215 is a feature vector FV for each cluster, which summarizes the respective determined values of the features. The selection of the features for which the process 215 is carried out depends on whether the method 200 is used in the respective method run as a method for training or already operationally as a recognition method for recognizing a seat occupancy state of the seating arrangement 105 on the basis of a previously trained machine learning model is used.

In the case of the method for training, all defined features are typically treated as part of process 215 in order to determine their respective values for the radar point cloud currently to be evaluated. In the case of the recognition method, on the other hand, it is sufficient in process 215 to take into account the features already selected during training, which together generally form a real subset of the set of features taken into account during training.

By means of a query 220, in which control information, in particular from a user input, is evaluated to determine whether the method 200 should be continued as a method for training (220 - yes) or as a recognition method (220 - no), the further course of the method 200 fixed.

First, the case of the recognition method (220 - no) is to be explained here: a further process 235 follows here, in which an already trained ML model is used in order to use the feature vectors FV as input data for each of the clusters and thus seats 105a to 105e to individually determine a result which represents a seat occupancy state of the respective seat 105a to 105e. In the simplest case, the result per seat in a class corresponds to a simple classification with only two classes, namely the class "seat occupied" and the other class "seats not occupied". However, more complex classifications are also conceivable. In particular, it is conceivable, using one or more features that indicate a Doppler shift and thus allow a distinction between static and dynamic radar point clouds 310 or 315, to introduce classes that allow a statement to be made as to whether an object that may be detected is on a seat is a moving or stationary object. It is even conceivable, based on certain movement patterns, for example Based on a chronological sequence of radar point clouds, to recognize breathing patterns using the ML model and thus to define corresponding classes that indicate whether the object is a living person or a living animal or not. Other classes are also conceivable, in particular based on an evaluation of a number of different characteristics.

The result of the seat occupancy can be further used in a wide variety of ways in order to control one or more functionalities of the vehicle 100 . In the following, the example will be considered that a warning signal should be output as a functionality if it is determined that a particular seat, for example the one in 1 seat 105b occupied by B the baby in the infant carrier, but the associated safety belt is not fastened. The latter can be determined in a known manner using a corresponding sensor system in a process 240 which, for example, checks whether the belt buckle is correctly inserted in the associated belt lock (240 - yes) or not (240 - no). If the seat belt is put on correctly (240-yes), it is not necessary to issue a seat belt warning signal and the method branches back to process 205 for a new run. Otherwise (240-no, a seat belt warning signal is output in process 240, which can in particular be an acoustic, visual and/or haptic signal, in particular to a driver of vehicle 100. A haptic signal can be sent in particular via the steering wheel, in particular as shaking, The method 200 then returns to the process 205 to run again.

If, on the other hand, the query 220 shows that the method 200 is to be continued as a method for training (220 - yes), a process 221 is carried out in which all feature vectors for the various clusters or seats from a plurality of previous method runs of the Process sequence 205-215 can be summarized on the basis of various radar point clouds. A correct seat occupancy result is assigned to each of the radar point clouds radar point cloud or their respective clusters derived therefrom for each seat. A set of training data TS and validation data VS, on the basis of which a suitable machine learning model is trained in a further process 222, results from appropriate division of the data obtained in this way. The model can in particular be a decision tree-based machine learning model or an artificial neural network. If the model is trained on the basis of the training data TS, it can be validated on the basis of the validation data VS, which were not used for the training, and within the scope of this validation, the model MD can be finally determined, including in particular a selection of the data for the model characteristics to be used is met. The model defined in this way can then be used in process 235 in order, as already described above, to determine a seat occupancy state of the seat arrangement 105, also for individual seats if clustering is used.

A very simple example of the structure of the training data TS and the validation data VS is shown in the table 4 illustrated, each main line, to which a seat occupancy state is assigned, corresponding to a radar point cloud from a sequence of several radar point clouds measured at different points in time.

A classification with four classes, ie possible results, 1, 2, 3 and 4 is used here for the seat occupancy states. For the sake of simplicity, in 4 only three different characteristics A, B and C are defined, in practice there will usually be more characteristics. The respective value determined in the process 215 is assigned to the radar point cloud for each feature. For example, the value 4.2 was determined for feature A in the first radar point cloud, the value 3.2 for feature B and the value 6.1 for feature C, and based on this the seat occupancy status was assigned to class 1. Correspondingly, on the basis of other values for the characteristics A, BC of the second radar point cloud of the sequence, the class 3 seat occupancy status was assigned.

With reference to 5 a first embodiment of a method for training 222 as part of method 200 for training an ML model for recognizing a seat occupancy state of a seat arrangement 105 with at least one seat will now be explained using pseudocode 500 .

In particular, the training data TS serve as input data for the method for training. The purpose of the training is to obtain a trained ML model MD, which is configured to indicate a seat occupancy state of the seating arrangement 105 based on respective values of various features of a radar point cloud 300 (data PC) obtained by means of an actual or simulated radar scan of a spatial area surrounding the seating arrangement to get result.

The training also involves, in particular, selecting a subset T of features that is as optimal as possible from a total set G of available features that are represented in the feature vector FV within the scope of the training select, so that when using the trained model MD as part of the recognition method in the method 200 due to the reduced number of features, on the one hand a very efficient and computationally resource-saving process flow can be used, but on the other hand a high quality and reliability of the result can be achieved because the feature selection (also) optimized to that effect. The adaptation of the ML model MD is therefore carried out in such a way that it is based exclusively on the selected M characteristics of the subset T with regard to its dependency on features of the total set G after the adaptation.

The subset T of M features is selected from the total set G, where M<N applies and the selection is based on the respective values of the N features of the total set G represented in the training data TS. In the pseudocode 500 in 5 G corresponds to the original set of all available features with which the set of candidates is initially initialized.

1. (a) wird ausgehend von einer Startmenge, die eine vorbestimmte Vorauswahl von K der N Merkmale der Gesamtmenge G mit 0 ≤ K ≤ M enthält, jedes der anderen N-K Merkmale der Gesamtmenge G einzeln jeweils als Kandidatenmerkmal der Startmenge zunächst temporär zugeordnet, um so eine Prüfmenge P („temporäre_Merkmale“) mit K+1 Merkmalen zu bilden. In 5 ist K=0 gewählt und die Startmenge somit eine leere Menge (Ausgewählte_Merkmale = { });
2. (b) erfolgt ein Trainieren des ML-Modells MD mit durch die Trainingsdaten TS je Trainingsdatensatz („i“) gegebenen Werten für die K+1 Merkmale der Prüfmenge P und den zugehörigen Sitzbelegungszustand; In 5 dient der Zähler i als Index für n verschiedene Untermengen aus der Menge der in den Trainingsdaten TS repräsentierten Trainingsdatensätze. So lassen sich verschiedene, mit i indizierte Varianten des ML-Modells (Modell i) erzeugen, die sich obwohl ihrerer Art nach gleich (z.B. gleicher Entscheidungsbaum-Algorithmus) unterscheiden können und in der Regel werden, weil sie durch unterschiedliche Gruppen von Trainingsdatensätzen trainiert wurden; Die verschiedenen Varianten bzw. Modelle i bilden somit zusammen ein Ensemble aus verschiedenen Modellen.
3. (c) wird das jeweilige so trainierte ML-Modell zum aktuellen Index i auf Basis von ihrer Art nach den Trainingsdaten TS entsprechenden aber von diesen zumindest teilweise verschiedenen Validierungsdaten VS validiert und dabei mittels zumindest eines vorbestimmten Scores bewertet;
4. (d) wird in Abhängigkeit von den jeweiligen Scores der verschiedenen Kandidatenmerkmale der im Rahmen der Iteration (d.h. in 5 im Rahmen der beiden verschachtelten For-Schleifen) aufgetretenen Prüfmengen P („temporäre_Merkmale“) eines der Kandidatenmerkmale als Auswahlkandidat („ausgewählter Kandidat“) bestimmt. In 5 ist dies derjenige Kandidat mit dem höchsten, über die Menge der n Modelle ermittelnden Mittelwert µ.
5. (e) wird für den Auswahlkandidat anhand seines Scores geprüft, ob er ein in Bezug darauf definiertes Aufnahmekriterium erfüllt (in 5: if-Abfrage); Hier ist das Aufnahmekriterium mittels eines Konfidenzintervalls (zum Wert 95%) definiert.
6. (f) wird, wenn das Aufnahmekriterium erfüllt wird, der Auswahlkandidat als neues Element in die Teilmenge T („Ausgewählte Merkmale”) aufgenommen und eine neue Iteration beginnt ohne Berücksichtigung des nunmehr ausgewählten Auswahlkandidats als Kandidatenmerkmal. Andernfalls wird die Iteration und damit die Definition der Teilmenge T abgeschlossen;

In the procedure after 5 the selection of the subset T of features from the total set G takes place iteratively in several nested loops. Per iteration:

1. (a) Starting from a starting set that contains a predetermined preselection of K of the N features of the total set G with 0 ≤ K ≤ M, each of the other NK features of the total set G is initially assigned temporarily as a candidate feature of the starting set, in order to create a To form test set P (“temporary_characteristics”) with K+1 characteristics. In 5 K=0 is chosen and the starting set is therefore an empty set (selected_features = { });
2. (b) the ML model MD is trained with values given by the training data TS for each training data record (“i”) for the K+1 characteristics of the test set P and the associated seat occupancy state; In 5 the counter i serves as an index for n different subsets from the set of training data sets represented in the training data TS. In this way, different variants of the ML model (model i), indexed with i, can be generated which, although they are of the same type (e.g. same decision tree algorithm), can differ and usually do so because they were trained using different groups of training data sets ; The different variants or models i thus together form an ensemble of different models.
3. (c) the respective ML model trained in this way is validated for the current index i on the basis of the type of validation data VS that corresponds to the training data TS but is at least partially different from these and is evaluated using at least one predetermined score;
4. (d) depending on the respective scores of the different candidate characteristics, the value used in the iteration (i.e. in 5 within the framework of the two nested For-loops), one of the candidate characteristics is determined as a selection candidate (“selected candidate”). In 5 this is the candidate with the highest mean µ determined over the set of n models.
5. (e) the selection candidate is checked on the basis of his score as to whether he fulfills an admission criterion defined in relation thereto (in 5 : if query); Here the inclusion criterion is defined by means of a confidence interval (95% value).
6. (f) if the inclusion criterion is met, the selection candidate is included as a new element in the subset T (“Selected Features”) and a new iteration begins without considering the now selected selection candidate as a candidate feature. Otherwise the iteration and thus the definition of the subset T is completed;

As already mentioned, the starting set for the first iteration process is predetermined (selected_features={ }), while the respective starting set of each subsequent iteration run is determined by the starting set of the iteration run immediately preceding it plus its selection candidate.

With reference to 6 a second embodiment 600 of the method for training 222 as part of the method 200 for training an ML model for recognizing a seat occupancy state of a seat arrangement 105 with at least one seat will now be explained using a flow chart.

The selection of the features of the subset T from the total set G includes a grouping 605 of features of the total set G into different groups, which is done in such a way that the features of each group are correlated with one another in relation to the training data TS and a specific correlation measure. This can be done in particular on the basis of a correlation measure defined on the basis of density connectivity, for example using DBSCAN (Density-Based Spatial Clustering of Applications with Noise), an algorithm for cluster analysis. It is then per formed group in a selection process 610 selects only one feature as a representative of the group, with the subset T being formed only from representatives. In particular, redundancies can be reduced or even avoided, ie the occurrence of different features that essentially characterize the same property(s) of the radar point cloud. For example, coordinates of radar points related to different coordinate systems (eg vehicle coordinate system vs. seat-related coordinate system) could be recognized as being redundant to one another.

In a further process 615, an ensemble of different individual ML models of the same type is now trained on the basis of training vectors extracted from the training data and only having representatives as features. The training of the individual ML models differs in that a subset of the training data is used for at least two of the individual ML models and the subsets differ from one another. The models differ again in that they were defined on the basis of different training data. Especially in the case of decision tree-based ML models, these are of the same type (decision tree-based), but the concrete decision trees can differ because different training data can and usually lead to different "tree growth".

The respective models can be trained using a weighting in such a way that features recorded in the model are weighted in such a way that, during the training of the individual ML models, each initial recording of a feature that has not yet been recorded by any of the individual ML models in one of these ML models is linked with a weighting that makes such a recording more difficult than recording a feature that has already been used. For example, the first inclusion of a new feature that has not yet been used is "punished" based on the weighting, so that multiple use of features is promoted compared to the introduction of new features and thus the lowest possible model complexity.

After the individual ML models have been trained within the scope of the process 615, a feature selection 620 can be carried out in order to determine a final ML model MD, in particular which of the representatives can be adopted as a feature in the final ML model MD. In particular, this can be done in such a way that only those representatives are included in the subset T 625 that were included as a feature in more than x% of the individual ML models, where x is a fixed or variably adjustable parameter with 30≦x≦70 , e.g. x = 50. The characteristics that do not meet this criterion are not included in the final model, but are abandoned 630.

The training of the ML model can, in particular in each of the two aforementioned specific embodiments 500 and 600, also include training the already adapted ML model on the basis of the training data relating to the selected M features of the subset T. This means, for example, that if the model is already per 5 is trained, with the subset T being defined, further training of the model takes place to the effect that only training and possibly validation data are fed in, which relate exclusively to the already preselected features of the subset T. The quality of the training can thus be increased even further, because the features that have now been sorted out as part of the feature selection and are therefore no longer needed for the trained ML model MD to be subsequently used in the recognition method no longer have any influence.

While at least one exemplary embodiment has been described above, it should be appreciated that a large number of variations thereon exist. It should also be noted that the example embodiments described are intended to be non-limiting examples only, and are not intended to limit the scope, applicability, or configuration of the devices and methods described herein. Rather, the foregoing description will provide those skilled in the art with guidance for implementing at least one example embodiment, while understanding that various changes in the operation and arrangement of elements described in an example embodiment may be made without departing from the scope of the appended claims the specified object and its legal equivalents are deviated from.

Reference List

B

Baby in baby seat

CL

cluster data

FV

feature vector

md

Machine Learning (ML) model

personal computer

Measurement data representing radar point cloud(s)

TS

training data

vs

validation data

100

vehicle

105

seating arrangement

105a-e

seats

110

radar sensor

110a

Field of view of the radar sensor 110

115

System for the automated detection of a seat occupancy status

115a

data processing unit

115b

Storage

200

Method for the automated detection of a seat occupancy status

205-240

individual processes or procedural steps as part of the 200 procedure

300

radar point cloud

310

static radar points

315

dynamic radar points

320

Regions of the radar point cloud 300 with high radar point density

325a-e

Spatial areas for cluster definition

400

Table with training data

500

Pseudocode for the first embodiment of the method for training

600

second embodiment of the method for training

605-630

individual processes or procedural steps as part of the 600 procedure

Claims (26)

Method (222) for training a system for seat occupancy detection with a machine learning model for detecting a seat occupancy state of a seating arrangement with at least one seat, the method (222) having: training the machine learning model using training data (TS) to obtain a trained machine learning model (MD) configured based on respective values of various features of an area surrounding the seating arrangement by means of an actual or simulated radar scan spatial region (325a-e) obtained radar point cloud (300) to obtain a seat occupancy state of the seating arrangement characteristic result; wherein the training data (TS) for various possible predefined seat occupancy states of the seat arrangement each have at least one training data record which represents respective values of N different characteristics of a total quantity G for the respective seat occupancy state, which individually or cumulatively form a radar point cloud (300) corresponding to the respective seat occupancy state characterize. Method (222) for training according to claim 1 , wherein the training of the machine learning model comprises: selecting a subset T of M features from the total set G, where M < N applies, and selecting based on the respective values of the N features of the total set G represented in the training data (TS). he follows; and fitting the machine learning model in such a way that it relies solely on the selected M features of the subset T for its dependency on features of the total set G after fitting. Method (222) for training according to claim 2 , wherein the training of the machine learning model further comprises training the already adjusted machine learning model on the basis of the training data (TS) with respect to the selected M features of the subset T. Method (222, 500) for training according to claim 2 or 3 , wherein the selection of the subset T of features from the total set G takes place iteratively, wherein: per iteration pass: starting from a starting set that contains a predetermined preselection of K of the N features of the total set G with 0 ≤ K ≤ M, each of the other NK Features of the total set G is initially assigned temporarily as a candidate feature of the starting set in order to form a test set P with K+1 features; the machine learning model is trained with values given by the training data (TS) per training data set for the K+1 features of the test set P and the associated seat occupancy status; the machine learning model trained in this way is validated on the basis of validation data (VS) which, in terms of its type, corresponds to the training data (TS) but is at least partially different from these and is thereby evaluated using at least one predetermined score; depending on the respective scores of the various candidate characteristics of the test sets P that occurred during the iteration the candidate feature is determined as a selection candidate; for the selection candidate, based on his score, it is checked whether he fulfills an admission criterion defined in relation thereto; and if this is the case, the selection candidate is included as a new element in the subset T and a new iteration begins without considering the now selected selection candidate as a candidate feature, while otherwise the iteration and thus the definition of the subset T is terminated; wherein the starting set for the first iteration process is predetermined, while the respective starting set of each subsequent iteration run is determined by the starting set of the iteration run immediately preceding it plus its selection candidate. Method (222, 500) for training according to claim 4 , wherein: the machine learning model comprises an ensemble with a plurality L of mutually similar individual machine learning models; the training of the machine learning model per test set comprises a corresponding training of the L individual machine learning models, the training of the L individual machine learning models differing in that for at least two of the L individual machine learning models a subset of the training data (TS) is used in each case, the subsets for the at least two individual machine learning models differing from one another; each of the L individual machine learning models trained in this way is validated on the basis of the type of validation data (VS) that corresponds to the training data (TS) but is at least partially different from these and is thereby evaluated using at least one individual score; for the resulting distribution of the L individual scores of the individual machine learning models, the respective value for at least one statistical measure characterizing the distribution is determined; and among all selection candidates of the test sets, a single selection candidate is selected as a candidate for the subset T depending on the one or more values of the at least one measure respectively assigned to it. Method (222, 500) for training according to claim 5 , where a mean value of the distribution is determined as a characteristic statistical measure of the distribution of the individual scores and that selection candidate is selected as a candidate for the subset T whose mean value represents the best value among all mean values of the selection candidates according to a definition of the score. Method (222, 500) for training according to claim 5 or 6 , where the inclusion criterion is or will be defined as a function of a confidence interval. Method (222, 600) for training according to claim 2 or 3 , wherein selecting the features of the subset T from the total set G comprises: grouping features of the total set G into different groups such that the features of each group are mutually correlated with respect to the training data (TS) and a certain correlation measure; and selecting one feature from each group as a representative of the group, the subset T being formed only from representatives. Method (222, 600) for training according to claim 8 , further comprising: training an ensemble of different individual machine learning models of the same type on the basis of training vectors extracted from the training data (TS) and having only representatives as features; The training of the individual machine learning models differs in that a subset of the training data (TS) is used for at least two of the individual machine learning models and the subsets differ from one another. Method (222, 600) for training according to claim 9 , where: when training the individual machine learning models as part of the training, a selection is made as to which of the representatives are to be included as a feature in the respective individual machine learning model, and these recorded features are weighted such that within the framework of the training of the individual machine learning models, each initial inclusion of a feature in one of these machine learning models that has not yet been included by any of the individual machine learning models is linked to a weighting that makes such inclusion more difficult than the inclusion of a feature already used ; and only those of the representatives are included in the subset T that were included as a feature in more than x% of the individual machine learning models, where x is a fixed or variably adjustable parameter with 30 ≤ x ≤ 70. Method (222, 500, 600) for training according to one of the preceding claims, wherein different training sets from the training data (TS) depending on the seat occupancy state assigned to them in each case a predetermined weighting of the seat occupancy states are taken into account to different degrees when training the machine learning model. Method (200) for automatically detecting a seat occupancy state of a seating arrangement (105) with at least one seat, the method (200) having: receiving or generating measurement data representing a radar point cloud (300) obtained on the basis of a radar scan of a spatial region (325a-e) surrounding at least sections of the seating arrangement (105) and having a plurality of radar points (310, 315); for each feature from a defined, non-empty set of features for characterizing properties of radar point clouds, determining a value assigned to the respective feature as a function of the radar point cloud represented by the measurement data (PC); Determining a seat occupancy state of the seating arrangement (105) using a trained machine learning model, the determined values of the characteristics being made available to the machine learning model (MD) as input data in order to generate a seat occupancy state of the seating arrangement ( 105) to obtain characteristic result; and Outputting information defined depending on the result. Method (200) according to claim 12 , wherein: the seat arrangement (105) has a plurality of seats (105a-e) for which a seat occupancy state is to be determined individually or cumulatively as part of the method; the set of radar points (310, 315) of the radar point cloud (300) by means of clustering depending on the respective spatial position of the radar points (310, 315) in relation to the seats (105a-e) into several subsets of the radar points ( 310, 315) containing cluster is subdivided in order to individually assign to each of the seats (105a-e) one of the clusters closest to it; the respective values of the features are determined for each of the seats (105a-e) individually on the basis of the cluster of radar points (310, 315) assigned to the respective seat (105a-e); the determination of the seat occupancy status of the seating arrangement (105) comprises individually determining a respective individual seat occupancy status for each of the seats (105a-e) using the trained machine learning model, the clusters assigned for the respective seat (105a-e). certain respective values of the features are made available as input data to the machine learning model in order to obtain a result characterizing a seat occupancy state of the respective seat as its output; and the information to be output is defined depending on the respective individual results for the different seats (105a-e). Method (200) according to Claim 13 , the radar point cloud (300) being segmented into a number of clusters in that each of the seats (105a-e) is assigned a subset of the radar points (310, 315) as a cluster depending on their respective position in such a way that the radar points (310, 315 ) of the cluster in a defined, closed spatial area (325a-e) in the vicinity of the seat. Method (200) according to any one of Claims 12 until 14 , wherein the outputting of the information comprises driving a signal source depending on the information in order to cause the signal source to output a defined signal depending on the driving. Method (200) according to claim 15 , wherein the signal source is controlled as a function of the information so that it emits a signal when the information results from a result according to which at least one seat of the seating arrangement (105) is occupied and/or a selected predetermined seat occupancy state is present. Method (200) according to Claim 16 , further comprising: detecting a seat belt fastening state of at least one seat of the seating arrangement (105) or receiving seat belt information characterizing this seat belt fastening state; the signal source being controlled as a function of the seat belt information and the information from the result in such a way that it emits a seat belt notice signal if, according to the information, at least one seat in the seating arrangement (105) is occupied and/or a selected, predetermined seat occupancy state is present and indicates seat belt information, that the associated seat belt of the seat is not fastened. Method (200) according to a Claims 12 until 17 , wherein the individual radar points (310, 315) of the radar point cloud (300) are each represented by one or more of the following data: - A position of the respective radar point in three-dimensional space; - A Doppler shift value of the radar signal to the respective radar point; - A signal-to-noise ratio value of the radar signal to the respective radar point. Method (200) according to any one of Claims 12 until 18 , wherein the machine learning (MD) model using the method for training according to any of Claims 1 until 11 is or will be trained. A method (222) for training according to any one of Claims 1 until 11 or method (200) according to claim 19 , the set of features used within the framework of the machine learning model containing at least one feature that falls into one of the following feature categories: - Features that indicate a spatial location of a cluster of radar points (310, 315) in the spatial environment of a seat describe the seating arrangement (105); - Features that describe the number of radar points (310, 315) of the radar point cloud (300) or a defined section thereof; - Features that describe for a specific cluster of radar points (310, 315) in the spatial environment of a seat of the seating arrangement (105), which seat of the seating arrangement (105) is assigned to the cluster; - features that describe Doppler shift values and/or a signal-to-noise ratio value of the radar signal to a respective radar point (310, 315) of the radar point cloud (300); - Features that describe a quantitative comparison of two or more clusters of radar points (310, 315) of the radar point cloud (300); - Features that describe an orientation and/or spatial extent of a cluster of radar points (310, 315) of the radar point cloud (300); - Features describing the radar point cloud (300) as a whole. Method (222) for training or method (200) according to any one of the preceding claims, wherein the result for the seating arrangement (105) or a respective seat (105a-e) thereof is or becomes defined as having a class according to a predetermined Classification of possible seat occupancy options indicates. Method (222) for training or method (200) according to Claim 21 , wherein the classification is or will be predetermined in such a way that it contains at least two of the following classes: - Seat arrangement (105) or the respective seat (105a-e) is unoccupied; - Seat arrangement (105) or the respective seat (105a-e) is occupied; - Seat arrangement (105) or the respective seat (105a-e) is occupied by one person; - Seat arrangement (105) or the respective seat (105a-e) is occupied by a person with a size above a size threshold; - Seat arrangement (105) or the respective seat (105a-e) is occupied by a person with a size below a size threshold; - Seat arrangement (105) or the respective seat (105a-e) is occupied with one thing; - Seat arrangement (105) or the respective seat (105a-e) is occupied by a child seat. Method (222) for training or method (200) according to any one of the preceding claims, wherein the machine learning model (MD) is based on at least one decision tree. System (115) for automatically detecting a seat occupancy state of a seat arrangement (105) with at least one seat, the system (115) having a data processing device which is configured to use the method (222) for training according to one of Claims 1 until 11 and/or the method (200) according to any one of Claims 12 until 20 to execute. Vehicle (100), comprising: a seating arrangement (105) with at least one seat (105a-e); a radar sensor (110) for at least partial radar scanning of the seating arrangement (105); and a system (115). Claim 24 for automatically recognizing a seat occupancy state of the seating arrangement (105) as a function of an at least partial radar scan of the seating arrangement (105) carried out by the radar sensor (110). Computer program or computer program product, comprising instructions, which when executed on the data processing device of the system Claim 24 , the system (115) cause at least one of the methods (200, 222) according to one of Claims 1 until 23 to execute.

Images