DE102014225039A1 - Method and apparatus for providing sparse Gaussian process models for calculation in an engine control unit - Google Patents

Abstract

The invention relates to a method for generating a number M of sparse Gaussian process models with the aim of model evaluation in a hardware-based model calculation unit (43), comprising the following steps:
Providing (S1) training data points X and associated output values Y ⁽¹⁾ , ..., Y ^(M) of a plurality of outputs y ⁽¹⁾ , ..., y ^(M) ;
Forming a common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ), where X' is virtual pad data points and H ⁽¹⁾ , ..., H ^(M) correspond to model parameters for each of the Gaussian process models to be created;
- Optimize (S3) the common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ) by a set of the virtual pad data points and the model parameters H ⁽¹⁾ , ..., H ^(M) for each of the Gaussian process models to be created.

Description

Technical area

The invention relates generally to engine control devices in which functional models are implemented as data-based functional models. In particular, the present invention relates to methods for determining sparse Gaussian process models from provided node data.

State of the art

For the implementation of functional models in control units, in particular engine control units for internal combustion engines, the use of data-based functional models is provided. Frequently, parameter-free data-based function models are used, since these are generated without specific specifications from training data, ie. H. a set of training data points that can be created.

An example of a data-based function model is the so-called Gaussian process model, which is based on the Gaussian process regression. Gaussian Process Regression is a versatile method for data-based modeling of complex physical systems. The underlying regression analysis is based on usually large amounts of training data, so it makes sense to use approximate solutions that can be evaluated more efficiently.

For the Gaussian process model there is the possibility of a sparse Gaussian process regression, in which only a representative amount of support point data is used to create the data-based function model. For this purpose, the support point data must be selected or derived in a suitable manner from the training data. With the sparse Gaussian process regression, the training points are replaced by a smaller number of virtual interpolation points, thereby simplifying the calculation and saving memory and computing time in the prediction.

The pamphlets E. Snelson et al., Sparse Gaussian Processes Using Pseudoinputs, 2006 Neural Information Processing Systems 18 (NIPS) and Csató, Lehel; Opper, Manfred, "Sparse On-Line Gaussian Processes"; Neural Computation 14: pp. 641-668, 2002 , disclose a method for determining support point data for a sparse Gaussian process model.

Other related methods are known from Smola, AJ, Schölkopf, W., "Sparse Greedy Gaussian Process Regression", Advances in Neural Information Processing Systems 13, pp. 619-625, 2001 , and Seeger, M., Williams, CK, Lawrence, ND, "Fast Forward Selection to Speed Up Sparse Gaussian Process Regression," Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics, 2003 ,

Furthermore, control components with a main calculation unit and a separate model calculation unit for calculating data-based function models in a control unit are known from the prior art. For example, the document discloses DE 10 2010 028 259 A1 a control unit with an additional logic circuit as a model calculation unit, which is designed to calculate exponential functions in order to support the performance of Bayes regression methods, which are required in particular for the calculation of Gaussian process models.

Overall, the model calculation unit is designed to carry out mathematical processes for calculating the data-based function model based on parameters and support points or training data. In particular, the functions of the model calculation unit for efficiently calculating exponential and sum functions are implemented purely in hardware, so that it is possible to calculate Gaussian process models with a higher computing speed than can be done in the software-controlled mainframe.

Disclosure of the invention

According to the invention, a method for determining a plurality of sparse Gaussian process models according to claim 1 as well as a model calculation unit, a control unit and a computer program according to the independent claims are provided.

Further embodiments are specified in the dependent claims.

• – Bereitstellen von Trainingsdatenpunkten X und zugeordnete Ausgangswerte Y⁽¹⁾, ..., Y^(M) von mehreren (Anzahl M) Ausgangsgrößen y⁽¹⁾, ..., y^(M);
• – Bilden einer gemeinsamen Marginal-Likelihood-Funktion p(Y⁽¹⁾, ..., Y^(M)|X, X', H⁽¹⁾, ..., H^(M)), wobei X' virtuellen Stützstellendatenpunkten und H⁽¹⁾, ..., H^(M) Modellparametern für jedes der zu erstellenden Gauß-Prozess-Modelle entsprechen;
• – Optimieren der gemeinsamen Marginal Likelihood-Funktion p(Y⁽¹⁾, ..., Y^(M)|X, X', H⁽¹⁾, ..., H^(M)), um eine Menge der virtuellen Stützstellendatenpunkte und die Modellparameter H⁽¹⁾, ..., H^(M) für jedes der zu erstellenden Gauß-Prozess-Modelle zu erhalten.

According to a first aspect, there is provided a method of creating a number M of sparse Gaussian process models for execution in a purely hardware-based model calculation unit, comprising the following steps:

• Providing training data points X and associated output values Y ⁽¹⁾ , ..., Y ^(M) of a plurality (number M) of output quantities y ⁽¹⁾ , ..., y ^(M) ;
• Forming a common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ), where X' is virtual pad data points and H ⁽¹⁾ , ..., H ^(M) correspond to model parameters for each of the Gaussian process models to be created;
• - Optimizing the common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ) to obtain a set of the virtual landmark data points and the model parameters H ⁽¹⁾ , ..., H ^(M) for each of the Gaussian process models to be created.

The above method provides a way to easily create multiple sparse Gaussian process models based on a number of identical virtual tributary data points, respectively.

Sparse Gaussian process models are much more memory-efficient than conventional Gaussian process models, since only K << N support point data points have to be stored (M: number of virtual support point data points, N: number of training data points). Often a quarter of the node data points or less is sufficient. Thus, the use of the virtual support point data points can often save up to 80% of the memory requirement since the original training data points do not have to be stored. Consequently, more data-based functional models can be stored in a physical model calculation unit. In addition, the evaluation of the individual, smaller Gaussian process models can be done faster.

Frequently, however, the training data points come from a test bench measurement in which more than one output variable is measured with the same training data points. If one wishes to model several output variables, according to the prior art, a data-based functional model is learned for each output variable. Using conventional Gaussian processes, it is noticeable that the parameters of the individual function models partly agree: in the case of non-normalized data, the support point data points are for the most part identical for all individual function models. This can be exploited to save space on the hardware unit by storing the support point data only once.

However, this is not possible when sparse Gaussian process models are used. In this case, namely, the virtual support point data points are different for each individual model. Although saving in each of the individual models determined compared to the conventional approach of the Gaussian process models storage space, with multiple Gaussian process models, however, the total storage space requirements of the respective virtual support point data points may be greater than the unique storage space requirement of the underlying training data points, which in conventional Gaussian process models can be used to calculate multiple output quantities.

For example, the memory footprint of six individual function models, which were reduced to 20% of the memory footprint of a comparable non-sparse function model with the underlying training data using the sparse Gaussian process model, is 120%. Since the underlying training data for the computation of the sparse Gaussian process models is typically stored only once, the storage space requirement can be higher when using several sparse Gaussian process models, since these each use different support point data sets.

The above method therefore provides for combining the memory advantages of sparse Gaussian process models with the possibility of multiple use of support point data points. The essential idea is not to optimize the virtual support points per model, but together, so that all individual models share the same virtual training data. This achieves a reduction of the common storage space necessary for all models.

Further, for optimizing the common marginal likelihood function, p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ ,..., H ^(M) ) can be used for linearly independent outputs the expression p (Y ⁽¹⁾ | X, X ', H ⁽¹⁾ ) ··· p (Y ^(M) | X, X', H ^(M) ) be optimized.

Further, for optimizing the common marginal likelihood function, p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ ,..., H ^(M) ) can be used for linearly dependent outputs the expression p (Y ⁽¹⁾ | X, X ', H ⁽¹⁾ ) · p (Y ⁽²⁾ | Y ⁽¹⁾ , X, X', H ⁽¹⁾ , H ⁽²⁾ ) ···. .. * p (Y ^(M) | Y ⁽¹⁾ , ..., Y ^(M-1) , X, X ', H ⁽¹⁾ , ..., H ^(M) ) be optimized.

It can be provided that the training data points are obtained by measurement on a test bench.

Furthermore, the common marginal likelihood function can plausibility the model parameters H ⁽¹⁾ , ..., H ^(M) for each of the Gaussian process models to be created for the output values Y ⁽¹⁾ , ..., Y ^{(M ) of} the output quantities y ⁽¹⁾ , ..., y ^(M) , whereby the virtual support point data points X 'and the model parameters H ⁽¹⁾ , ..., H ^(M) are optimized by maximizing the plausibility.

According to another aspect, a model calculation unit is for performing a calculation of sparse Gaussian process models provided, wherein the sparse Gaussian process models are determined based on the common virtual support point data points X '.

According to a further aspect, a control device with a software-controlled main calculation unit and the above purely hardware-based model calculation unit is provided.

Brief description of the drawings

Embodiments are explained below with reference to the accompanying drawings. Show it:

1 a schematic representation of an overall system for determining a sparse Gaussian process model and the engine control unit, on which the sparse Gaussian process model is implemented; and

2 a flowchart for illustrating a method for determining sparse Gaussian process models with identical virtual support point data points.

Description of embodiments

1 shows an arrangement 1 with a modeling system 2 based on z. B. in a (not shown) test bench recorded training data for determining a data-based function model, in particular a Gaussian process model, in the situation. The training data provides training data points of one or more input variables as well as one or more output quantities representing the behavior of a physical system 3 , such as an internal combustion engine, describe.

The use of non-parametric, data-based function models is based on a Bayes regression method. The basics of Bayesian regression are, for example, in CE Rasmussen et al., Gaussian Processes for Machine Learning, MIT Press 2006 , described. Bayesian regression is a data-based method based on a model. To create the model, measurement points of training data and associated output data of an output variable are required. The creation of the model is based on the use of support point data, which correspond to the training data in whole or in part or are generated from these. Furthermore, abstract hyperparameters are determined which parameterize the space of the model functions and effectively weight the influence of the individual measurement points of the training data on the later model prediction.

The abstract hyperparameters are determined by an optimization method. One possibility for such an optimization method is an optimization of a marginal likelihood p (Y | H, X). The marginal likelihood p (Y | H, X) describes the plausibility of the model parameters H, given the measured y-values of the training data, represented as vector Y and the x-values of the training data, represented as matrix X. In model training, p (Y | H, X) is maximized by searching for suitable hyperparameters which lead to a course of the model function determined by the hyperparameters and the training data and which map the training data as accurately as possible. To simplify the calculation, the logarithm of p (Y | H, X) is often maximized because the logarithm does not change the monotony of the plausibility function.

The calculation of the Gaussian process model is carried out according to the following calculation rule. The input values x~ _d for a test point x (input variable vector) are first normalized and centered, according to the following formula:

In this case, m _x the mean value function with respect to an average value corresponding to the input values of the nodes data, s _x the variance of the input values of the nodes data, and d the index for the dimension D of the test point x.

As a result of creating the non-parametric, data-based function model you get:

The thus obtained model value v is normalized by means of an initial normalization, according to the formula: ṽ = vs _y + m _y .

Here, v correspond to a normalized model value (output value) at a normalized test point x (input variable vector of dimension D), ṽ a (non-normalized) model value (output value) at a (non-normalized) test point (input variable vector of dimension D), x _{i of} a sample point Support point data, N the number of nodes of the support point data, D the dimension of the input data / training data / support point data space, and l _d and σ _f the model parameters H from the model training. The vector Q _y is a quantity calculated from the model parameters and the training data. Furthermore, m _y correspond to the mean value function with respect to an average value of the output values Support point data and s _{y of} the variance of the output values of the support point data.

The modeling system 2 further executes a method of processing the acquired training data to provide the data-based function model using hyperparameters and node data representing a subset or derived amount of the training data. In this way, a so-called sparse Gaussian process model can be created.

These support point data and hyperparameters are then transferred to a controller 4 transferred and stored there. The control unit 4 is related to a physical system 3 , z. B. an internal combustion engine, which is operated using the data-based function model.

1 further shows a schematic representation of a hardware architecture for an integrated control block, z. B. in the form of a microcontroller in which integrated in a main computing unit 42 and a model calculation unit 43 are provided for the purely hardware-based calculation of a data-based function model. The hyperparameters and support point data determined, for example, in a previous optimization are stored in a memory unit 41 saved. The main calculator 42 , the storage unit 41 and the model calculation unit 43 are via an internal communication connection 44 , such as As a system bus, with each other.

The intended as a microcontroller main computing unit 42 is designed to calculate function values of the provided data-based function model using a software-determined algorithm. To speed up the calculation and to relieve the microcontroller 42 is provided, the model calculation unit 43 to use. The model calculation unit 43 is fully hardware-based and is only capable of performing a particular calculation rule based essentially on repeated computations of additions and multiplications and an exponential function. Basically, the model calculation unit 43 that is, essentially hardwired and therefore not designed to provide software code as in the main computing unit 42 perform.

Alternatively, a solution is possible in which the model calculation unit 43 to provide a constrained, highly specialized instruction set for computing the data-based function model. In the model calculation unit 43 However, in no embodiment is a processor provided. This enables a resource-optimized implementation of such a model calculation unit 43 or a surface-optimized design in integrated construction. In one possible implementation, the model calculation unit 43 as FPGA in particular in a unit in the control unit 4 be educated.

In such a control unit 4 In addition to conventional Gaussian process models, sparse Gaussian process models can also be calculated. Due to the fact that, in the case of sparse Gaussian process models, the amount of support point data is significantly lower than in the case of conventional Gaussian process models, the storage capacity to be provided can make the memory unit 41 for storing the support point data or it may be several records of training data from several sparse Gaussian process models in the memory unit 41 get saved.

wobei k T / x, Q_y ∈ R^N gilt. Hervorzuheben ist, dass durch k T / x der Kovarianzvektor zwischen dem Abfragepunkt x und den Stützstellendatenpunkten dargestellt wird. Diese berechnet sich durch den „squared exponential“-Kernel als

A conventional Gaussian process regression uses the given support point data points / training data points to compute the covariance matrix. You get the model prediction in the form

in which k T / x, Q _y ∈ R ^N applies. It should be emphasized that through k T / x the covariance vector between the query point x and the support point data points is displayed. This is calculated by the "squared exponential" kernel as

With sparse Gaussian process models, the essential idea is to replace the given support point data formed by the "real" support point data points with "virtual", ie artificially generated support point data points. K artificial support point data points are generated and suitably positioned by an optimizer so that the model prediction of a sparse Gaussian process model with the virtual support point data points is as accurate as possible with the Gaussian process model with the original support point data points. By integrating out the artificial y data, it is possible to have K virtual X positions x _i to optimize.

The model prediction for the sparse Gaussian process model is given by y = k T / * Q -1 / MK _MN (Λ - σ 2 / nI) ^-1 Y where k T / * ∈ R ^M , Q _M ∈ R ^{M × M} , K _MN ∈ R ^{M × N} , Λ is an N-dimensional diagonal matrix and Y is the vector of the y-values of the original support point data points.

It is k T / * again, the covariance vector, but this time calculated between the query point x and the M-dimensional vector of the virtual support point data points x _i , However, the vector multiplied by it as scalar product is now by the expression Q * / y = Q -1 / MK _MN (Λ - σ 2 / nI) ^-1 Y given.

wenn geeignete Werte für den Parametervektor Q * / y sowie die virtuellen Stützstellendatenpunkte verwendet werden.So you get the same form as for the prediction of conventional Gaussian processes:

if suitable values for the parameter vector Q * / y as well as the virtual support point data points are used.

In 2 FIG. 4 is a flowchart illustrating a method for providing a plurality of sparse Gaussian process models with hyperparameters and virtual node data that are identical for all Gaussian process models, shown schematically.

During the training of a sparse Gaussian process model, the model parameters H are set for each of the individual function models, as well as the location of a number M of common virtual support point data points X '. This is done, for example, by optimizing the marginal likelihood p (Y | H, X, X '). The marginal likelihood p (Y | H, X, X ') describes the plausibility of the model parameters H and the virtual pad data points X', where the measured y values of the training data, represented as vector Y, and the x values (training data points) Training data, shown as matrix X, are given.

In order for all outputs, i. For each function model, to perform the marginal likelihood optimization separately, which would result in different sets of virtual vertex data points X 'and corresponding model parameters, the individual optimization problems are now combined, e.g. by setting up a common marginal likelihood function for all function models and optimizing them in a manner known per se.

In step S1, measured values Y ⁽¹⁾ , Y ⁽²⁾ for a plurality of output quantities y ⁽¹⁾ , y ⁽²⁾ for a set of training data X are determined or predetermined.

In step S2 it is checked whether the output quantities y, for each of which a function model in the form of a sparse Gaussian process model is to be created, are linearly dependent or not. If the output variables are not linearly dependent (alternative: no), the following common marginal likelihood function is optimized for two output variables in step S3: p (Y ⁽¹⁾ , Y ⁽²⁾ | X, X ', H ⁽¹⁾ , H ⁽²⁾ ) = p (Y ⁽¹⁾ | X, X', H ⁽¹⁾ ) · p (Y ^{( 2)} | X, X ', H ⁽²⁾ )

For M output quantities, the following applies accordingly: p (Y ⁽¹⁾ | X, X ', H ⁽¹⁾ ) · p (Y ⁽²⁾ | X, X', H ⁽²⁾ ) ··· p (Y ^(M) | X, X) ', H ^(M) ) where Y ⁽ⁱ⁾ corresponds to the measured output values of the i-th model, X corresponds to the original training data, which is identical for all outputs y, and H ⁽ⁱ⁾ corresponds to the model parameters of the ith function model. Here, p (Y ⁽ⁱ⁾ | X, X ', H ⁽ⁱ⁾ ) corresponds to the likelihood function of the i-th conventional sparse Gaussian process model.

As a result of the optimization, one obtains a set of virtual support point data X 'and the model parameters H ⁽ⁱ⁾ for each of the M function models.

If the output quantities are not linearly dependent (alternative: yes), the following common marginal likelihood function is optimized in step S4:

For two dependent outputs: p (Y ⁽¹⁾ , Y ⁽²⁾ | X, X ', H ⁽¹⁾ , H ⁽²⁾ ) = p (Y ⁽¹⁾ | X, X', H ⁽¹⁾ ) · p (Y ^{( 2)} | Y ⁽¹⁾ , X, X ', H ⁽¹⁾ , H ⁽²⁾ )

For M dependent output quantities, the following applies accordingly: p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ) = p (Y ⁽¹⁾ | X, X', H ^{( 1)} ) · p (Y ⁽²⁾ | Y ⁽¹⁾ , X, X ', H ⁽¹⁾ , H ⁽²⁾ ) ··· · p (Y ^(M) | Y ⁽¹⁾ , ..., Y ^(M-1) , X, X ', H ⁽¹⁾ , ..., H ^(M) )

These likelihood functions can be analogous to those from the document Boyle & Frean: Dependent Gaussian Processes, Advances in Neural Information Processing Systems 17, p 217-224, 2005 be optimized known methods.

As a result of the optimization, one obtains a set of virtual support point data X 'and the model parameters H ⁽ⁱ⁾ for each of the M function models.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102010028259 A1 [0007]

Cited non-patent literature

• E. Snelson et al., "Sparse Gaussian Processes Using Pseudoinputs", 2006 Neural Information Processing Systems 18 (NIPS) [0005]
• Csató, Lehel; Opper, Manfred, "Sparse On-Line Gaussian Processes"; Neural Computation 14: pp. 641-668, 2002 [0005]
• Smola, AJ, Schölkopf, W., "Sparse Greedy Gaussian Process Regression", Advances in Neural Information Processing Systems 13, p. 619-625, 2001 [0006]
• Seeger, M., Williams, CK, Lawrence, ND, "Fast Forward Selection to Speed up Sparse Gaussian Process Regression," Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics, 2003. [0006]
• CE Rasmussen et al., Gaussian Processes for Machine Learning, MIT Press 2006 [0028]
• Boyle & Frean: Dependent Gaussian Processes, Advances in Neural Information Processing Systems 17, p 217-224, 2005 [0056]

Claims (10)

Method for creating a number M of sparse Gaussian process models with the aim of model evaluation in a hardware-based model calculation unit ( 43 ), comprising the following steps: - providing (S1) training data points X and associated output values Y ⁽¹⁾ , ..., Y ^(M) of a plurality of output quantities y ⁽¹⁾ , ..., y ^(M) ; Forming a common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ), where X' is virtual pad data points and H ⁽¹⁾ , ..., H ^(M) correspond to model parameters for each of the Gaussian process models to be created; - optimizing (S3) the common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ) by one To get the set of virtual vertex data points and the model parameters H ⁽¹⁾ , ..., H ^(M) for each of the Gaussian process models to be created. The method of claim 1, wherein for optimizing the common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^(M) ) for linearly independent output variables the expression p (Y ⁽¹⁾ | X, X ', H ⁽¹⁾ ) ··· p (Y ^(M) | X, X', H ^(M) ) is optimized. The method of claim 1 or 2, wherein for optimizing the common marginal likelihood function p (Y ⁽¹⁾ , ..., Y ^(M) | X, X ', H ⁽¹⁾ , ..., H ^{( M)} ) for linearly dependent output quantities the expression p (Y ⁽¹⁾ | X, X ', H ⁽¹⁾ ) · ... · p (Y ^(M) | Y ⁽¹⁾ , ..., Y ^(M-1) , X, X', H ⁽¹⁾ , ..., H ^(M) ) is optimized. Method according to one of claims 1 to 3, wherein the training data points are obtained by measurement on a test bench. Method according to one of claims 1 to 4, wherein the common marginal likelihood function has a plausibility of the model parameters H ⁽¹⁾ , ..., H ^(M) for each of the Gaussian process models for the output values Y ^{(1 )} , ..., Y ^(M) indicates the output quantities y ⁽¹⁾ ,..., Y ^(M) , the virtual support point data points X 'and the model parameters H ⁽¹⁾ , ..., H ^(M) Maximizing the plausibility can be optimized. Model calculation unit ( 43 ) for performing a sparse Gaussian process model calculation, wherein the sparse Gaussian process models are determined based on the common virtual tributary data points X '. Control unit ( 4 ) with a software-controlled mainframe ( 42 ) and a hardware-based model calculation unit ( 43 ) according to claim 6. Computer program which is set up to carry out all the steps of a method according to one of Claims 1 to 5. A machine-readable storage medium on which a computer program according to claim 8 is stored. Electronic control unit ( 4 ), which is adapted to carry out all the steps of a method according to one of claims 1 to 5.

Images