GB2605754A

GB2605754A - A method for estimating the height of a person inside of a motor vehicle by a height estimation device

Info

Publication number: GB2605754A
Application number: GB2102323.9A
Authority: GB
Inventors: Dokania Shubham
Original assignee: Mercedes Benz Group AG
Current assignee: Mercedes Benz Group AG
Priority date: 2021-02-19
Filing date: 2021-02-19
Publication date: 2022-10-19
Also published as: GB202102323D0

Abstract

A method and device for estimating a height (Fig. 1, 26) of a person (Fig. 1, 12) inside of a motor vehicle (Fig. 1, 14) by detecting an arm (Fig. 1, 20, 22) of the person, determining a length of the arm, and estimating the height of the person depending on the length of the arm by using a mathematical model (Fig. 1, 28) by an electronic computing device (Fig. 1, 18), perhaps using a trained or neural network, perhaps a Gaussian process model. The lengths and angles of the arm, wrist (44), forearm (50), elbow (46), upper arm (52) and shoulder (48) may be detected and used in determining the length of the arm. The height estimate may determine adjustment of ergonomic devices, such as a seat (Fig. 2, 54), or a steering wheel, within the vehicle. The model, which may run as a computer program, may also determine an uncertainty value (Fig. 1, 42) for the height estimate.

Description

A METHOD FOR ESTIMATING THE HEIGHT OF A PERSON INSIDE OF A MOTOR

VEHICLE BY A HEIGHT ESTIMATION DEVICE

FIELD OF THE INVENTION

[0001] The invention relates to the field of automobiles. More specifically, the invention relates to a method for estimating a height of a person inside of a motor vehicle, a corresponding computer program product as well as a corresponding height estimating device.

BACKGROUND INFORMATION

[0002] Estimating human height based on visual input is already known in the state of the art and a challenging task, especially when the subject is seated inside the motor vehicle and only partially visible for a detection device of the motor vehicle. There is a need in the art to detect the height of the person in order to adjust an ergonomic functional device, like a seat or a steering wheel, in particular according to the most ergonomic seating condition, hence ensuring maximum comfort for driving.

[0003] US 1 0643085 B1 discloses a method for detecting body information on passengers of a vehicle based on humans' status recognition. The method includes steps of: a passenger body information-detecting device, (a) inputting an interior image of the vehicle into a face recognition network to detect faces of the passengers and output passenger feature information, and inputting the interior image into a body recognition network to detect bodies and output body-part length information; and (b) retrieving specific height mapping information by referring to a height mapping table of ratios of segment body portions of human groups to heights per the human groups, acquiring a specific height of the specific passenger, retrieving specific weight mapping information from a weight mapping table of correlations between the heights and weights per the human groups, and acquiring a weight of the specific passenger by referring to the specific height.

SUMMARY OF THE INVENTION

[0004] It is an object of the invention to provide a method, a computer program product as well as a corresponding height estimation device, by which a robust height estimation of a person inside of the motor vehicle can be realized.

[0005] This object is solved by a method, a computer program product as well as a height estimation device according to the independent claims. Advantageous forms of configuration are presented in the dependent claims.

[0006] One aspect of the invention relates to a method for estimating a height of a person inside of a motor vehicle by a height estimation device. At least one arm of the person is detected inside the motor vehicle by a detection device of the height estimation device. A length of the detected arm is determined by an electronic computing device of the height estimation device. The height of the person is estimated depending on the determined length of the arm by using a mathematical model by the electronic computing device.

[0007] Therefore the method provides a robust height estimation while being highly data efficient and reliable.

[0008] The detection device may be for example a camera or a thermal camera for detecting the arm of the person. In particular the invention makes use of the fact that the domain of the human anatomy across a variety of factors including age, gender, and geographical information or furthermore, comprises a strong correlation between the arm span and the height of an individual. In particular, a correlation between a single arm length can be used in order to estimate the height of the person. This correlation is used in the mathematical model.

[0009] Therefore by detecting and determining the length of the arm of the person, the electronic computing device is able to estimate the height of the person and may for example adjust an ergonomic functional device in the motor vehicle. Therefore, the person may perceive a higher comfort level in the motor vehicle while driving or sitting inside the car.

[0010] In an embodiment the arm length of the detected arm is determined by a trained network of the electronic computing device and/or the height of the person is estimated by a trained network of the electronic computing device. In particular, data collection is used, while using a model-in-the-loop-approach. In particular a joined location in the three-dimensional space for the arm is collected to train the network. This information is stabilized over time and used to compute the length of the inner arm. For collection, a subject only provides their actual height information and the rest of the information required for the height estimation is automatically collected by the network, completely minimizing the need for any manual labeling. Based on this data collected, the mathematical model uses in particular a Bayesian approach and filters the necessary information from the arm length and orientation information, and after optimization, this network is able to predict and approximate the height along with confidence intervals. In the deployed height estimation device based on the predictions of the mathematical model and the confidence scores, the mathematical model is able to judge its own uncertainty in the prediction and can filter out unlikely predictions. Furthermore, it uses a stabilization filter to remove any noise due to sensor errors.

[0011] In another embodiment, for determining the length of the arm a wrist of the person and a forearm of the person and an elbow of the person and an upper arm of the person and a shoulder of the person is detected by the detection device. In particular, the complete arm of the person is detected in a three-dimensional space. The arm length is determined by using this information of the trained network and then determines the arm length.

[0012] According to another embodiment the electronic computing device determines an angle between the forearm and the upper arm and the length is determined depending on the determined forearm, the upper arm and the angle. Therefore, a robust and precise determining of the arm length is provided.

[0013] In another embodiment, depending on the estimated height of the person an ergonomic functional device of the motor vehicle is adjusted. Therefore, a comfort level for the person may be raised inside the motor vehicle. In particular, the ergonomic functional device is automatically adjusted according to the estimated height of the person.

[0014] In another embodiment, a position of a seat as the ergonomic functional device and/or a position of a steering wheel as the ergonomic functional device is adjusted.

Therefore, a comfort level of a driver is automatically raised by adjusting the seat and/or the steering wheel. Furthermore, if the person is a passenger, the seat of the passenger may also be automatically adjusted in order to raise the comfort level of the passenger.

[0015] According to another embodiment the mathematical model is trained by using an approximate Gaussian process model. This approximate Gaussian process model is a variant of the Gaussian process. Due to the large complexity of the Gaussian process, it is not feasible to deploy this on hardware for large datasets. For this reason the approximate Gaussian process model is used, which also behaves as an infinite neural network for the computation.

[0016] In another embodiment, the electronic computing device additionally determines an uncertainty value for the estimation of the height. Therefore, the mathematical model is able to judge its own uncertainty in the prediction and can filter out unlikely predictions.

[0017] The afore-mentioned method is in particular a computer-implemented method. Therefore, another aspect of the invention relates to a computer program product comprising program code means for performing the method according to the preceding aspect. Furthermore the invention relates to a computer-readable storage medium comprising the computer program product.

[0018] Another aspect of the invention relates to a height estimation device for estimating a height of a person inside of a motor vehicle, comprising at least one detection device and one electronic computing device, wherein the height estimation device is configured to perform a method according to the preceding aspect. In particular, the method is performed by the height estimation device.

[0019] The electronic computing device comprises means for performing the method. In particular, the electronic computing device may comprise processors, circuits, for example integrated circuits, and further electronic means for performing the method.

[0020] Another further aspect of the invention relates to a motor vehicle comprising the height estimation device according to the preceding aspect.

[0021] Advantageous forms of configuration of the method are to be regarded as advantageous forms of the computer program product, the height estimation device and the motor vehicle. The height estimation device and the motor vehicle comprise means for performing the method.

[0022] Further advantages, features, and details of the invention derive from the following description of preferred embodiments as well as from the drawings. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figures and/or shown in the figures alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and/or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0024] The drawings show in: [0025] Fig. 1 a schematic block diagram according to an embodiment of the height estimation device; and [0026] Fig. 2 a schematic view of a person inside of the motor vehicle.

[0027] In the figures the same elements or elements having the same function are indicated by the same reference signs.

DETAILED DESCRIPTION

[0028] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0029] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0030] The terms "comprises', "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by "comprises" or "comprise" does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0031] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0032] Fig. 1 shows a schematic block diagram according to an embodiment of a height estimation device 10. The height estimation device 10 is for estimating a height 26 of a person 12 (Fig. 2) inside of a motor vehicle 14, which is schematically shown. The height estimation device 10 comprises at least one detection device 16, which may for example be configured as a camera or a temperature camera. Furthermore, the height estimation device 10 comprises at least one electronic computing device 18.

[0033] The height estimation device 10 is configured for estimating the height 26 of the person 12. At least one arm 20, 22 (Fig. 2) of the person 12 inside of the motor vehicle 14 is detected by the detection device 16 of the height estimation device 10. A length 24 of the detected arm 20, 22 is determined by the electronic computing device 18. The height 26 of the person 12 is estimated depending on the determined length 24 of the arm 20, 22 using a mathematical model 28 by the electronic computing device 18.

[0034] In particular, Fig. 1 shows, that in this embodiment by the detection device 16 a plurality of images 30 are provided. From the images 30 a joint localization 32 is performed. From the joint localization 32 a joint extraction 34 is performed. From the plurality of images 30 a network 36 is used to determine the three-dimensional joint localizations 38 from the joint extraction 34. The three-dimensional joint localizations 38 are filtered and stabilized, which is shown in the block 40.The arm length 24 and the filtered and stabilized three-dimensional joint localizations 38 are then forwarded to the mathematical model 28. The mathematical model 28 then computes the approximate height 26 and may also compute an uncertainty value 42 for the estimation of the height 26, [0035] In particular Fig. 1 shows the overall process for determining the height 26. In particular, the network 36 uses a model-in-the-loop approach, wherein the network 36 is trained. Therefore, in the training process the joint localization in the three-dimensional space for the person's hand, elbow and shoulder is collected. This information is stabilized over time and used to compute the length of the inner arm. For collection, the subject only provides the actual height information. The rest of the information that is required for height estimation is automatically collected by the height estimation device 10. Thereby the need for any manual labelling is completely minimized. The mathematical model 28 is based on this data collected, wherein the mathematical model 28 uses a Bayesian approach proposed here to filter the necessary information from the arm length 24 and orientation information, and after optimization, is able to predict and approximate the height 26 along with the confident intervals. In the height estimation device 10, which is for example deployed in the motor vehicle 14, based on the prediction of the mathematical model 28 and the confidence scores, the mathematical model 28 is able to judge its own uncertainty in the prediction and may filter out unlikely predictions.

[0036] The data collection for the network 36 may be based on an available joint localization deep learning module, wherein the location of the arm joints, which may be for example a hand 44 (Fig. 2), an elbow 46 (Fig. 2), and a shoulder 48 (Fig. 2) on the input image plane (U, V). Based on this plane location an input from a time-of-flight depth camera, the corresponding 3D location of each joint in the real world system (X, Y, Z) is determined. This allows for the determining of the arm length 24 over the surface of the arm 20, 22, and to predict the arm orientation angle 0 at the elbow 46. This is also in particular shown in Fig. 2.

[0037] Since the localization system is already available, the height estimation device 10 directly estimates the three-dimensional information for different human subjects who enter the motor vehicle 14, and only recalls their actual height 26 for training the network 36. The height estimation device 10 does not require the measurement of different body parts or a collection of images for further adaptations. The network 36 then stores all this information from each frame in a structured format which later may be used by the mathematical model 28 for training.

[0038] A machine learning method is used for the mathematical model 28, in particular to train the mathematical model 28. In particular, the machine learning method used is called an approximate Gaussian process model, which is a variant of the Gaussian process. Due to the large complexity of the Gaussian process, it is not feasible to deploy this process on hardware for large datasets, so that the approximate Gaussian process model, which also behaves as an infinite neural network, for the computation is used. The mathematical model 28 takes the entire structured data as collected and selects a set of inducing points which best describe the pattern information available in the observations. Based on these inducing points, and taking the sensor noise into account, the approximate Gaussian process method learns a function distribution over the space of arm lengths and full height. This function, once optimized, is able to infer an approximate height value given a forearm 50 (Fig. 2), an upper arm 52 (Fig. 2), in particular the length of the forearm 50 and the length of the upper arm 52, and the arm orientation as the angle at the elbow 46. The angle S at the elbow 46 helps to stabilize any three-dimensional corrections due to change in the arm surface. The mathematical model 28 also computes the uncertainty value 42 for its prediction.

[0039] The Gaussian process models a distribution of functions taken as a collection of random variables, any finite number of which have consistent joint Gaussian distributions. Inducing points are heuristically picked up from the dataset to best describe the information and model the Gaussian process kernel according to this method. In an embodiment, a composite kernel built from the addition of the RBF and a linear kernel is used. The mathematical model 26 is optimized for its parameters using the log-likelihood as below for 0, and the mean and covariance functions may be derived from the same easily. The variance of the prediction depicts the uncertainty value 42 and the mean is the approximate output: 1 1 71 lOg p(f (x) I 6 x) = --2 f (x)T K (0, x, f)-1 f(x) --2 log det(K(0, x, x.)) --2 log 27 [0040] At the same time of inference, for the post-processing step, the mathematical model 28 generates two values, in particular the height 26, and the uncertainty value 42. Based on the uncertainty value 42 or confidence, a threshold on the minimum amount of confidence is set up which ignores the frame where the mathematical model 28 is unsure about its predictions. Such a case may happen in the case of a bad frame, complex arm positioning, sensor noise, false positives in joint detection or furthermore, and this confidence estimate helps to avoid errors in the predictions due to such causes. On top of this, the arm length is also stabilized as well as the computed height 26 through the application of a simple and efficient low-pass filter, which may be for example a so-called 1 Euro filter. This helps to avoid any sudden jumps in the predictions that may have passed the previous uncertainty tests. This also makes sure that the changes in height 26 are smooth over frames to avoid sudden fluctuations due to cases such as fast movement, camera blurring, or sensor error.

[0041] In particular, the low-pass filter for stabilization of the three-dimensional values and the corresponding computing length is applied. Along with the Gaussian process model, also a linear model may be used for processing the forearm length and upper arm length. This prediction is more useful for taller people whose distribution is sparser. Based on the arm angle 0 computed, the parameters can be adjusted and the corresponding height values before the final post-processing is computed. A parameter is responsible to filter the frames and remove the prediction with a high degree of uncertainty.

[0042] For the final post-processing to generate the approximate height 26, all parameters for the arm lengths, and the angle 0 are used. The values for hyper-parameters may be used and can be adjusted on the fly during the testing. These parameters dictate the weightage and shift between the linear and Gaussian model outputs, and ensure a smooth transition for predictions. This mechanism ensures robustness and stability for all ranges of outputs, and has been seen to perform well even for the sparsely distributed tall height data.

[0043] In the post-processing steps, the parameters are of crucial importance. These parameters dictate the adjustment/switching between the linear and Gaussian models. Essentially, the set of values of the parameters from the data distribution, where a change in distribution is observed, may be set. This value is used to smoothen the sudden transition/jump of predictions at the thresholds. The parameter acts like a small band where a weighted average of linear and Gaussian models are used to morph the sudden toggle between the two models. Usually this parameter is set to be around 2-3 for a bandwidth of +/-cm of the parameter.

[0044] Starting out with these initial values of the parameters an eager search over the hyper-parameter space is used by these three parameters to optimize for best f it over accuracy and loss in the data distribution.

[0045] Fig. 2 shows a top view of the person 12 inside of the motor vehicle 14. With the detection device 16 a first arm 20 and a second arm 22 of the person 12 are detected. Fig. 2 shows, that the first arm 20 is straight and the second arm 22forms the angle e. Fig. 2 shows an example for localization of the arm orientation. In the three-dimensional image the wrist 44, the elbow 46, and the shoulder 48 are detected, and the angle 0 is computed over three-dimensional coordinates for the angle 8 between the corresponding points. This information is input to the core computational mathematical model 28 for processing.

[0046] In particular according to the estimated height 26 a position of an ergonomic functional device 54, which may be for example a seat of a steering wheel of the motor vehicle 14, is adjusted.

Reference Signs height estimation device 12 person 14 motor vehicle 16 detection device 18 electronic computing device first arm 22 second arm 24 length 26 height 28 mathematical model plurality of images 32 joint localization 34 joint extraction 36 network 38 3D joint localization block 42 uncertainty value 44 wrist 46 elbow 48 shoulder forearm 52 upper arm 54 ergonomic functional device angle

Claims

CLAIMS1. A method for estimating a height (26) of a person (12) inside of a motor vehicle (14) by a height estimation device (10), the method comprising the steps of: - detecting at least one arm (20, 22) of the person (12) inside of the motor vehicle (14) by a detection device (16) of the height estimation device (10); - determining a length (24) of the detected arm (20, 22) by an electronic computing device (18) of the height estimation device (10); and - estimating the height (26) of the person (12) depending on the determined length (24) of the arm (20, 22) by using a mathematical model (28) by the electronic computing device (18).
2. The method according to claim 1, characterized in that the arm length (24) of the detected arm (20, 22) is determined by a trained network (36) of the electronic computing device (18) and/or the height (26) of the person (12) is estimated by a trained network, in particular the trained mathematical model (28), of the electronic computing device (18).
3. The method according to claim 1 or 2, characterized in that for determining the length (24) of the arm (20, 22) a wrist (44) of the person (12), a forearm (50) of the person (12), an elbow (46) of the person (12), an upper arm (50) of the person (12), and a shoulder (48) of the person (12) are detected by the detection device (16).
4. The method according to claim 3, characterized in that the electronic computing device (18) determines an angle (0) between the forearm (50) and the upper arm (52) and the length (24) is determined depending on the determined forearm (50), the upper arm (52), and the angle (9).
5. The method according to any one of claims 1 to 4, characterized in that depending on the estimated height (26) of the person (12) an ergonomic functional device (54) of the motor vehicle (14) is adjusted.
6. The method according to claim 5, characterized in that a position of a seat as the ergonomic functional device (54) and/or a position of a steering wheel as the ergonomic functional device (54) is adjusted.
7. The method according to any one of claims 1 to 6, characterized in that the mathematical model (28) is trained by using an approximate Gaussian process model.
8. The method according to any one of claims 1 to 7, characterized in that the electronic computing device (18) additionally determines an uncertainty value (42) for the estimation of the height (28).
9. A computer program product comprising program code means for performing a method according to any one of claims 1 to 8.
10. A height estimation device (10) for estimating a height (28) of a person (12) inside of a motor vehicle (14), the height estimation device (10) comprising at least one detection device (16) and one electronic computing device (18), wherein the height estimation device (10) is configured to perform a method according to any one of claims 1 to 8.