DE102023003669A1 - Method for detecting a seat belt slack in a vehicle - Google Patents

Abstract

The invention relates to a method for detecting belt slack in a seat belt (2) running on an occupant (3) in a vehicle (1). According to the invention, it is determined whether the occupant (3) is wearing a jacket as clothing while the vehicle (1) is being driven based on captured image data from an interior sensor system (6) of the vehicle (1), if it is determined that the occupant (3) wears a jacket, based on data stored in a database (DB) on the clothing worn by occupants (3) of vehicles (1) of a vehicle fleet, weather conditions and belt extension lengths in relation to the clothing worn by an occupant (3). a change in the belt extension length is determined based on the clothing worn by the occupant (3), - using a machine-trained model (M1) based on the data from the database (DB), a belt extension length for the occupant (3) of the vehicle that exceeds a predetermined threshold value (1) the vehicle fleet is predicted and - if the predicted belt extension length exceeds a predetermined threshold value, a visual and / or acoustic and / or haptic warning is issued in the vehicle (1).

Description

The invention relates to a method for detecting a belt slack in a seat belt running on an occupant in a vehicle.

From the DE 10 2019 134 141 B4 is known a seat belt condition determination method for use with a vehicle having a vehicle camera or multiple vehicle cameras and a seat belt sensor or multiple seat belt sensors. The method includes the steps of: collecting camera image data with the vehicle camera(s); collecting seat belt data with the seat belt sensor(s); and performing a seat belt classification process, the seat belt classification process receiving the camera image data as input and providing a seat belt classification as output, the seat belt classification process classifying an image from the camera image data into further classifications representative of a condition of the seat belt. The step of performing further includes performing the seat belt classification process that includes a passenger posture estimation process, the passenger posture estimation process receiving preprocessed camera image data as input and providing a passenger posture estimate as output, the passenger posture estimation process a position of or multiple passenger body parts in an image in the preprocessed camera image data, the passenger pose estimation process further comprising using a neural network to estimate the position of the body parts in the image.

The invention is based on the object of specifying a method for detecting a belt slack in a seat belt running on an occupant in a vehicle.

The object is achieved according to the invention by a method which has the features specified in claim 1.

Advantageous embodiments of the invention are the subject of the subclaims.

• - anhand erfasster Bilddaten einer Innenraumkamera des Fahrzeuges ermittelt wird, ob der Insasse eine Jacke als Kleidung während eines Fahrbetriebes des Fahrzeuges trägt,
• - wenn ermittelt wird, dass der Insasse eine Jacke trägt, anhand von in einer Datenbank gespeicherten Daten zu der von Insassen von Fahrzeugen einer Fahrzeugflotte getragenen Kleidung, Wetterbedingungen und Gurtauszugslängen in Bezug auf die jeweils von einem Insassen getragene Kleidung eine Änderung der Gurtauszugslänge aufgrund der jeweils von dem Insassen getragenen Kleidung ermittelt wird,
• - mittels eines basierend auf den Daten der Datenbank maschinell trainierten Modells eine einen vorgegebenen Schwellwert überschreitende Gurtauszugslänge für den Insassen des Fahrzeuges der Fahrzeugflotte prognostiziert wird und
• - wenn die prognostizierte Gurtauszugslänge einen vorgegebenen Schwellwert überschreitet, ein optischer und/oder akustischer und/oder haptischer Warnhinweis in dem Fahrzeug ausgegeben wird.

According to the invention, a method for detecting a belt slack in a seat belt running on an occupant in a vehicle provides that

• - based on captured image data from an interior camera of the vehicle, it is determined whether the occupant is wearing a jacket as clothing while the vehicle is being driven,
• - if it is determined that the occupant is wearing a jacket, based on data stored in a database on the clothing worn by occupants of vehicles in a vehicle fleet, weather conditions and belt extension lengths in relation to the clothing worn by an occupant, a change in the belt extension length due to the respective clothing worn by the occupant is determined,
• - using a machine-trained model based on the data from the database, a belt extension length that exceeds a predetermined threshold value is predicted for the occupant of the vehicle in the vehicle fleet and
• - If the predicted belt extension length exceeds a predetermined threshold value, a visual and/or acoustic and/or haptic warning is issued in the vehicle.

By using the method, the safety of occupants of the vehicle can be increased, since the visual and/or acoustic and/or haptic warning is issued in the vehicle when it is recognized that the seat belt is not comparatively tight on the occupant's body and thus a There is no belt.

In particular, such a belt slackness occurs when the occupant wears clothing that is too loose and/or heavily padded, especially jackets.

Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

• 1 schematisch ein Fahrzeug und eine Vorrichtung zum Erkennen einer Gurtlose eines an einem Insassen verlaufenden Sicherheitsgurtes in dem Fahrzeug,
• 2A schematisch eine Vorrichtung zum Ermitteln einer Gurtauszugslänge eines Sicherheitsgurtes und einen Sicherheitsgurt,
• 2B schematisch ein teilweise verdecktes Muster des Sicherheitsgurtes,
• 2C schematisch das Muster in zweidimensionaler Projektion im dreidimensionalen Raum,
• 3A schematische einen Ansatz zum Erkennen einer Gurtlose,
• 3B schematisch einen alternativen Ansatz des Verfahrens zum Erkennen einer Gurtlose,
• 4A schematisch einen Verfahrensablauf zur Datenermittlung zur Durchführung des Verfahrens,
• 4B schematisch einen Verfahrensablauf zu dem in 3A gezeigten Ansatz und
• 4C schematisch einen Verfahrensablauf zu dem in 3B gezeigten Ansatz.

Show:

• 1 schematically a vehicle and a device for detecting a belt slack in a seat belt running on an occupant in the vehicle,
• 2A schematically a device for determining a belt extension length of a seat belt and a seat belt,
• 2 B schematically a partially hidden pattern of the seat belt,
• 2C schematically the pattern in two-dimensional projection in three-dimensional space,
• 3A schematic an approach to detecting a belt slack,
• 3B schematically an alternative approach to the method for detecting a belt slack,
• 4A schematically a process flow for determining data to carry out the procedure,
• 4B schematically a process flow for the in 3A shown approach and
• 4C schematically a process flow for the in 3B shown approach.

Corresponding parts are provided with the same reference numbers in all figures.

1 shows a vehicle 1 and a device for detecting a seat belt slack in the 2A until 2C , 3A and 3B shown seat belt 2, which is in the 3 and 4 Inmate 3 shown as an example and in sections.

2A shows a device for determining a belt extension length of a seat belt 2 and a seat belt 2 with a pattern M, 2 B shows the seat belt 2 with an object O and in which partially covers the pattern M 2C the pattern M on the seat belt 2 is shown in a two-dimensional projection in three-dimensional space.

3A shows an approach to detecting seat belt slack and in 3B An alternative approach to detecting belt slack is shown.

The 4A until 4C each show a process sequence, where in 4A a procedure for determining data for carrying out a method for determining belt slack, in 4B a procedure for the in 3A shown approach and in 4C a procedure for the in 3B shown approach is shown.

In the 1 The device shown comprises a control unit 4 with a database DB, an external sensor system 5, an interior sensor system 6, a central computer unit 7 coupled to the vehicle 1 in terms of data technology, a data transmission unit 8 and an optical and/or acoustic and/or haptic output unit 9. The control unit 4 is connected to the external sensor system 5, the interior sensor system 6, the data transmission unit 8 and the output unit 9.

The vehicle 1 belongs to a vehicle fleet, for example a vehicle manufacturer, with all vehicles 1 of this vehicle fleet being linked to the central computer unit 7 in terms of data technology.

In general, it is known that if the seat belt 2 does not fit tightly to the body of an occupant 3 of a vehicle 1, the seat belt 2 has a belt slack, so that the occupant 3 has a greater forward displacement path available in the event of an accident of the vehicle 1 and thus a risk of injury for the occupant 3 can increase significantly. In addition, due to the lack of belts, a relatively high delay can affect the occupant 3. Such a belt slack is caused in particular by comparatively wide and/or thickly lined clothing, for example a winter jacket, so that the seat belt 2 does not lie tightly against the body of the occupant 3.

A method for detecting a seat belt slack in a seat belt 2 running on an occupant 3 in a vehicle 1 is described below.

In a first method step S1 of an in 4A In the process sequence shown, which begins with start S and ends with stop St, an occupant 3, i.e. a driver of the vehicle 1 or generally a user of the vehicle 1, is used using a variety of methods, for example based on captured image data from the interior sensor system 6, and/or machine vision methods. for example, facial recognition, and/or identified using a personalized vehicle key and/or using a personal identification number. Data from the occupant 3 recorded at least while the vehicle 1 is driving can thus be clearly assigned to the occupant. Personal information and/or characteristics as well as individual vehicle settings can, for example, be stored or stored in a personal user profile. The occupant 3 can also be a child traveling in the vehicle 1.

In a second method step S2 of in 4A The procedure shown shows whether a user profile is available for the occupant 3 of the vehicle 1. In particular, with regard to the procedural process for decisions to be made, a yes is marked with a Y and a no with an N.

If it is determined in a third method step S3 that there is no user profile stored in the vehicle 1 for the occupant 3, a user profile is created.

In a fourth method step S4, clothing of the occupant 3 is analyzed based on detected signals from the interior sensor system 6, in particular based on captured image data from an interior camera. For this purpose, the image data is evaluated by means of the control unit 4, in particular by means of computer-aided vision.

Using computer-aided vision, it is possible to recognize objects O in the vehicle 1 based on the captured image data and to locate a recognized object O and assign it to an object class. For this purpose, at least one model M1 of machine learning, for example a deep neuro nal network, with labeled data, in particular training data, trained during development. It is therefore possible to recognize, locate and categorize clothing of the occupant 3, i.e. to classify what type of clothing it is. In particular, it can be recognized whether the clothing worn by the occupant 3 is a summer or winter jacket. Furthermore, finer gradations are possible, so that, for example, a jacket model from a jacket manufacturer brand can be recognized. In addition, individual jackets or identical jacket models from a manufacturer brand can be recognized based on feature properties represented in such a model M1, for example in a so-called latent space of a deep neural network, if these representative feature properties are stored locally in a control unit 4 of the vehicle 1 Database DB and / or a database DB stored in the central computer unit 7 are stored.

In addition, a position of the clothing is determined relative to the occupant 3, so that it can be determined whether a jacket is being worn by the occupant 3 or not, for example has been placed on a passenger seat. In addition, it can be recognized whether the jacket that the occupant 3 wears while the vehicle 1 is driving is worn closed or open.

Furthermore, environmental information can be obtained by various vehicle sensors of the external sensor system 5 of the vehicle 1 and/or derived based on recorded sensor information. In this case, environmental information can be measured based on recorded signals from a wind sensor, a temperature sensor, a rain sensor on a windshield, based on light intensity measurements, etc. and/or from measurements from other sensors of the outdoor sensor system 5, for example a lidar-based sensor, a radar-based sensor, an ultrasound-based sensor, an outdoor camera etc., can be derived. For example, reflection properties in an image captured by the radar-based sensor, a deteriorated signal-to-noise ratio (SNR) of the lidar-based sensor, or image artifacts in captured image data from the outdoor camera can be used to infer rain and estimate the amount of precipitation.

Furthermore, a seating position of the occupant 3, that is to say an individual seat adjustment, can be determined based on captured image data from an interior sensor or several interior sensors of the interior sensor system 6, based on current actuator settings of an electrically adjustable vehicle seat and/or based on settings of an individual seat adjustment profile, based on a so-called memory function. be determined.

In a fifth method step S5, a belt extension length of the fastened seat belt 2 is determined, the belt extension length being able to be determined using various methods. Using a displacement sensor, the belt extension length of the safety item 2 can be determined for each vehicle seat. In particular, an occupant 3 can be recognized, as described above, so that data stored on the vehicle side can be assigned to individual occupants 3.

For example, the belt extension length of the seat belt 2 will be determined based on detected signals from the interior sensor system 6, in particular an optical sensor, for example an interior camera, and a pattern M on the seat belt 2. As an optical pattern M, for example, ordered geometric shapes, as in the 2A until 2C is shown, or randomly pronounced point clouds are used, as long as elements of the seat belt 2 have unique patterns M and an assignment of pattern M to position on the seat belt 2 is determined.

In one embodiment, the pattern M can be pronounced using a dye that fluoresces under ultraviolet light, a so-called fluorescent dye, so that the pattern M is not perceptible to the human eye under natural conditions. A UV light source 10, for example in the form of a light-emitting diode, makes it possible to excite the patterns M at freely definable times and to detect an emitted light and thus the patterns M by means of the interior sensor system 6, in particular an interior camera, and to transmit it to the control unit 4 , as in 2A is shown.

The belt extension length of the seat belt 2 can be determined algorithmically in the control unit 4 by analyzing the pattern M in the captured image data of the interior camera of the interior sensor system 6 and identifying the visible, unique pattern M. A section of the pattern M at the upper end of the seat belt 2 results in the belt extension length of the seat belt 2 via a clear assignment to the position on the seat belt 2. Using another sensor of the interior sensor system 6, for example another interior camera, the pattern M can be localized directly in three-dimensional space be, so that a trajectory of the seat belt 2, as in 2C is shown can be determined.

Furthermore, a model M1 can be trained using machine learning methods, which automatically recognizes and localizes the pattern M in the control unit 4, for example using a folded neural network. The model M1 allows patterns M which are not in a plane parallel to an image plane, that is to say are tilted and/or twisted in three-dimensional space to a camera plane, to be recognized with relatively high accuracy and to be localized in the image plane. In particular, a model M1 can be trained in such a way that the extension length of the seat belt 2, which can also be referred to as the path length, can be derived with comparatively high accuracy from a totality of visible patterns M, since patterns M are partially hidden or not visible in the image plane the trajectory in three-dimensional space can be learned implicitly during training, as in 2 B and 2C is shown. It is also possible to distinguish correct from incorrect trajectories of the seat belt 2, that is, incorrect use of the seat belt 2, by means of a correspondingly trained model M1.

Alternatively or additionally, a relatively accurate displacement sensor for determining the belt extension length of the seat belt 2 applied can be implemented for each vehicle seat by counting revolutions of a belt retractor mechanism. The belt extension length of the seat belt 2 being put on is a function of a number of revolutions, with a change in the radius of a retractor spool due to the seat belt 2 being rolled up or unrolled being taken into account.

In an in 4A In the sixth method step S6 shown, data is collected for each occupant 3 and stored in the database DB stored on the vehicle and/or the database DB stored in the central computer unit 7. Each data record contains information about the belt extension length, the clothing worn by the occupant 3, for example a recognized sweater and/or jacket model and/or a manufacturer's brand of a sweater and/or jacket, latent space information, or similarly determined environmental information as well optional information such as a location, a time, a seat adjustment, a height of the occupant 3 etc., which information can be used to further refine the method.

Based on this data in the database DB, a change in the belt extension length or a statistical indication of the change in the belt extension length of the seat belt 2 for different clothing can be determined individually for each occupant 3. In particular, a large positive change in the belt extension length associated with certain clothing is associated with an increased risk of larger belt slack, whereby, for example, in simplified terms, the increase in the belt extension length can be equated to a size of the belt slack.

In the central computer unit 7, information relating to the change in the belt extension length of the seat belt 2 or the statistical information on the change in the belt extension length for different clothing and/or features of clothing is aggregated and stored in the database DB.

According to 3A an approach to detecting a seat belt slack in a seat belt 2 is shown and in 4B The process flow for this approach is shown.

This process also begins with Start S and ends with Stop St, whereby for decisions to be made, a yes is marked with a Y and a no with an N.

In a first method step S1, feature recognition and feature extraction ME take place in relation to the clothing worn by the occupant 3 of the vehicle 1. As in 3A shown, at least a distinction is made as to whether the occupant 3 is wearing tight-fitting clothing or loose and/or thick clothing. In a second method step S2, the recognized clothing and its characteristics are compared with the data stored in the database DB.

In a third method step S3, it is then determined whether the belt slack exceeds a predetermined threshold value due to the clothing worn and recognized by the occupant 3. If this is not the case, the procedure jumps to Stop St and the procedure for determining the belt slack is ended.

However, if it is determined that the belt slackness exceeds the predetermined threshold value due to the clothing, in a fourth method step S4 an optical and/or acoustic and/or haptic warning is issued by means of the output unit 9 in the vehicle 1. This threshold value is specified because the belt slack when the threshold value is reached is so great that there is an increased “out-of-position risk” for the occupant 3 who is wearing the seat belt 2, and the belt slack leads to greater delays and more serious injuries of the occupant 3 can lead.

With regard to this approach, it is possible to obtain statistical information about the average of an occupant 3 in a vehicle 1 based solely on the clothing characterized by the interior sensor system 6 to produce a change in the belt extension length across all occupants 3 of vehicles 1 in the vehicle fleet with the same clothing.

For example, an occupant 3 in a vehicle 1 of the vehicle fleet wears a down jacket with fur applications. Down jackets with fur applications of the same characteristic values were previously recorded in other vehicles 1 of the vehicle fleet and aggregated in the database DB of the central computer unit 7 to form an average change in the belt extension length when wearing such a down jacket.

If this average change in the belt extension length of the seat belt 2 being worn exceeds the predetermined threshold value, a warning message is issued in the vehicle 1 to the occupant 3 of this vehicle 1, indicating that it can be assumed that the belt slack exceeds the threshold value. This means that no measurement of the belt extension length is required, so that there need not be any sensors in the vehicle 1 to determine the belt extension length. Likewise, no reference value with regard to the belt extension length is necessary, for example if this cannot be determined because the occupant 3 is using the vehicle 1 for the first time.

in 3B A further approach to detecting a seat belt slack in a seat belt 2 is shown, the process flow for this approach being shown in 4C is shown.

This procedure also begins with Start S and ends with Stop St, whereby for appropriate decisions a yes is marked with a Y and a no with an N.

According to the further approach, the relative change in the belt extension length is predicted in a first method step S1 using a machine learning model M1. For this purpose, the data sets stored in the central computer unit 7 in the database DB are used to train the model M1. When training, in addition to signals detected by the interior sensor system 6, environmental information, information that characterizes the occupant 3, for example a body size, a body weight or the like, a seat adjustment, etc. can also be used as input variables in order to further refine the model M1. After training, it is possible to use the model M1 to make a prediction about the expected belt slack based on detected signals from the interior sensor system 6 and, if necessary, also based on the environmental information, the information that characterizes the occupant 3. With this approach, too, no measurement of a belt extension length is necessary, so that there need not be any sensors in the vehicle 1 to determine the belt extension length. Likewise, no reference value with regard to the belt extension length is necessary, for example if this cannot be determined because the occupant 3 is using the vehicle 1 for the first time.

In a second method step S2, it is then determined whether the predicted belt slack due to the clothing worn and recognized by the occupant 3 exceeds a predetermined threshold value. If this is not the case, the procedure jumps to Stop St and the procedure for determining the belt slack is ended.

If, on the other hand, it is determined that the belt slackness exceeds the predetermined threshold value due to the clothing, in a third method step S3 an optical and/or acoustic and/or haptic warning is issued by means of the output unit 9 in the vehicle 1. This threshold value is specified because the belt slack when the threshold value is reached is so great that there is an increased “out-of-position risk” for the occupant 3 who is wearing the seat belt 2, and the belt slack leads to greater delays and more serious injuries of the occupant 3 can lead.

What both approaches have in common is that a comparatively small proportion of the vehicles 1 in the vehicle fleet must be equipped with a sensor system for determining the belt extension length and/or a methodology for determining the belt extension length. As soon as a sufficiently large data set has been aggregated in the central database DB based on data from this proportion of vehicles 1 in the vehicle fleet, it is possible to detect a potential seatbelt slack in the other vehicles 1 in the vehicle fleet solely on the basis of recorded signals from the interior sensor system 6 and, if necessary, additional environmental information. To be able to predict information that characterizes the occupant 3, seat settings, etc.

In one possible embodiment, the corresponding data can already be collected during development and the database DB can be initialized with this data and/or an initial model M1 can be pre-trained and refined using vehicle fleet data.

In a further embodiment, the corresponding data can be collected during development and the database DB can be populated with this data and/or a model M1 can be pre-trained without sensors for determining the belt extension length having to be present in the vehicles 1 of the vehicle fleet.

The method is described in more detail in an application example. A driver as a passenger 3 of a comparatively expensive vehicle 1 places his child as a passenger 3 with a winter jacket in a child seat. Based on signals detected by the interior sensor system 6, the control unit 4 detects that the child is wearing a winter jacket. By comparing the characteristics of the winter jacket and the data stored in the DB database, a risk of an average belt slack of, for example, 10 cm for this winter jacket is reported. Since this belt slack exceeds the predetermined threshold value, a warning, for example by means of a warning symbol, is output in a display area of a display unit of an infotainment system, a so-called head unit of the vehicle 1, as an output unit 9, that a belt slack is generated due to the winter jacket and thus in the event of a collision Vehicle 1 poses a risk of injury to the child.

Reference symbol list

1

vehicle

2

Seatbelt

3

inmate

4

Control unit

5

Outdoor sensors

6

Interior sensors

7

central computer unit

8th

Data transmission unit

9

Output unit

10

UV light source

DB

Database

J

Yes

M

Pattern

M.E

Feature extraction

M1

Model

N

No

O

object

S

begin

St

stop

S1 to S6

Procedural step

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102019134141 B4 [0002]

Claims (3)

Method for detecting a belt slack in a seat belt (2) running on an occupant (3) in a vehicle (1), characterized in that - based on captured image data from an interior sensor system (6) of the vehicle (1), it is determined whether the occupant (3 ) wears a jacket as clothing while the vehicle (1) is driving, - if it is determined that the occupant (3) is wearing a jacket, based on data stored in a database (DB) on the occupants (3) of vehicles ( 1) clothing worn by a fleet of vehicles, weather conditions and belt extension lengths in relation to the clothing worn by an occupant (3), a change in the belt extension length is determined due to the clothing worn by the occupant (3), - using a method based on the data in the database (DB) machine-trained model (M1) a belt extension length exceeding a predetermined threshold value is predicted for the occupant (3) of the vehicle (1) of the vehicle fleet and - if the predicted belt extension length exceeds a predetermined threshold value, an optical and / or acoustic and / or Haptic warning message is issued in the vehicle (1). Procedure according to Claim 1 , characterized in that the belt extension length is determined based on a pattern (M) of the seat belt (2). Procedure according to Claim 1 or 2 , characterized in that the data recorded on the vehicle side is transmitted to a central computer unit (7) which is linked to the vehicles (1) of the vehicle fleet.

Images