KR20230035942A - Surrogate Filter Operating method and data assimilation method using thereof - Google Patents

Abstract

An embodiment relates to an operation method of a replacement filter, and a data assimilation method using the same. According to the embodiment, the operation method of the replacement filter comprises: a step of calculating a state prediction value before correction based on an InPCE3 model; a step of correcting a current observation value from the state prediction value based on an InPCE4 model; a step of updating an ensemble parameter of a replacement filter; a step of determining the observation value by the updated ensemble parameter based on the InPCE4 model; and a step of updating the state prediction value based on the determined observation value.

Description

Surrogate Filter Operating method and data assimilation method using it

The embodiment relates to a method of operating a replacement filter and a method of assimilating data using the same.

Data assimilation refers to the process of appropriately estimating the meteorological conditions from observational data such as meteorological conditions or flow rates, water levels, and floods at a specific point in time or during a certain period of time. The estimated meteorological condition, that is, the analysis field, is input as the initial condition of the numerical forecasting model and is used to predict future weather, flow rate, water level, flood, etc.

In performing data assimilation using the ensemble Kalman filter, the ensemble Kalman filter uses a method of estimating the background error covariance using ensembles.

In the data assimilation system, the initial N ensemble members generate forecasts corresponding to the N ensemble members from the numerical model, and analysis corresponding to the N ensemble members from the given observations using the background error covariance obtained from the obtained forecasts. After acquiring the field, the obtained analysis field is used as the initial field again, and the forecast is obtained from the model M, and the next data assimilation is performed.

Previously developed data assimilation techniques are essential for real-time flood prediction, but they involve iterative model execution for ensembles. Accordingly, when the time required to execute each model is long, real-time forecasting is generally impossible due to repetitive execution.

Regarding data assimilation, Korean Patent Registration No. 10-1758234 discloses a data assimilation method and device.

The invention according to the embodiment aims to provide a model capable of efficient and highly accurate real-time forecasting by proposing a Surrogate Kalman Filter having the same accuracy as the existing data assimilation model.

In detail, a new alternative filter is provided using the polynomial chaos expansion (PCE) theory for the entire or internal process of the existing ensemble Kalman filter, and the disadvantages of the sequential experimental design (SED) are compensated. In order to do this, we propose a dual optimization method, Sequential experimental design-Polynomial degree (SED-PD), to solve the problem of selecting the value of the polynomial order randomly or by trial and error in the existing optimization system, even when the accuracy does not decrease monotonically. We want to provide a way to ensure convergence.

Through the invention according to the embodiment, it is possible to provide a model capable of efficient and highly accurate real-time forecasting by proposing a surrogate Kalman filter having the same accuracy as the existing data assimilation model.

In detail, a new alternative filter is provided using the polynomial chaos expansion (PCE) theory for the entire or internal process of the existing ensemble Kalman filter, and the disadvantages of the sequential experimental design (SED) are compensated. In order to do this, we propose a dual optimization method, Sequential experimental design-Polynomial degree (SED-PD), to solve the problem of selecting the value of the polynomial order randomly or by trial and error in the existing optimization system, even when the accuracy does not decrease monotonically. A method for ensuring convergence can be provided.

1 is a diagram for explaining configurations of replacement filters proposed in an embodiment.
2 is a flowchart for explaining an optimization method for building a PCE in an embodiment.
In an embodiment, it is a diagram showing stopping iteration of a loop according to a convergence criterion.
4 is a diagram showing which criterion is used in an embodiment.
5 is a diagram comparing parameter operation results of an existing dual ensemble Kalman filter and four Dual SuKFs in an embodiment.
6 is a diagram comparing the accuracy of a conventional ensemble Kalman filter, a dual ensemble Kalman filter, and eight replacement filters in an embodiment.
FIG. 7 is a diagram of predictive comparison of flow by a data assimilation experiment of an existing ensemble Kalman filter, a dual ensemble Kalman filter, and 8 replacement filters in an embodiment.
8 is a diagram comparing the accuracy of the event of FIG. 7 in an embodiment.
Figure 9 compares the relative accuracies of eight alternative filters in an example.
10 is a diagram for explaining relative differences with respect to metrics having different golden times in an embodiment.
11 is a diagram showing a result of comparing efficiencies between conventional ensemble Kalman filters and replacement filters in an embodiment.
12 is a diagram for explaining a difference between an existing SED technique and a proposed SED-PD technique in an embodiment.
13 is a flowchart illustrating a data assimilation method performed in an apparatus for data assimilation according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes can be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents or substitutes to the embodiments are included within the scope of rights.

Terms used in the examples are used only for descriptive purposes and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

In addition, in the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

In addition, in describing the components of the embodiment, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element may be directly connected or connected to the other element, but there may be another element between the elements. It should be understood that may be "connected", "coupled" or "connected".

Components included in one embodiment and components including common functions will be described using the same names in other embodiments. Unless stated to the contrary, descriptions described in one embodiment may be applied to other embodiments, and detailed descriptions will be omitted to the extent of overlap.

The invention according to the embodiment may be applied to a single or dual ensemble Kalman filter. A single or dual Ensemble Kalman Filter (EnKF) is characterized by updating the model state in the time dimension, and the difference between the two filters is whether model parameters are updated or not.

In an embodiment, the state transition equation is as Equation 1 below.

[Equation 1]

Here, x ^- _t is the i-th state prediction value predicted at time t, and x ⁺ _-1 means the i-th state prediction value updated at time t-1. The size of x _t is [N _DA Х N _s ], where N _DA is an ensemble member and N _s is a model state. θ ⁺ _-1 is the ith ensemble parameter updated at time t-1 and θ has the size of [N _DA Х N _p ], where N _p is the number of model parameters. u _t-1 is the ith ensemble force occurring at time t-1, and it can be generated according to a log-normal distribution having the same mean as the actual force u _t-1 and the covariance E ^u _t-1 .

In an embodiment, E ^u _t-1 may be assumed using a relative error between the physical force (RE _u ) and u _-1 (ie, E ^u _t-1 = RE _u Х u _t-1 ). The size of u _t-1 is [N _DA Х N _I ], where N _I is the physical force. Any mathematical model can be applied to f(.) for calculating the model state. w ⁱ _t-1 representing the uncertainty of the model structure means the ith model error at time t-1.

Assuming that the model parameters also follow a random walk (random walk), the ith ensemble parameter at time t can be processed by adding Gaussian noise τ ⁱ _t-1 with covariance E ^θ _-1 to θ _-1 .

[Equation 2]

In Equation 2 above, the covariance E ^θ _-1 can be assumed by multiplying the relative error of the parameter (RE _θ ) and the parameter (θ ^{i +} _t-1 ) of the ensemble parameter distribution at time t-1.

At the predicted state estimate of i at time t, Equation 3 below can calculate a function for the model state estimate, parameter, and physical force,

[Equation 3]

Here, h(.) is a propagator for calculating the output of the model and may be a hydrologic model including a nonlinear relationship.

According to Equation 4 below, the ith ensemble parameter at time t of Equation 2 can be corrected using the Kalman gain K ^θ _t and the ith observation value y ^obs.i _t at time t.

[Equation 4]

[Equation 4]

y ^obs.i _t is an observation perturbed by measurement noise that follows a normal distribution with a mean equal to the predefined covariance and mean of the actual observation, and that observation is based on the relative error of the observation (RE _obs ) and the observation (y ^obs _t ). (ie, E ^obs _t = RE _obs × y ^obs _t ).

Using this updated ensemble parameter θ ^{i +} _t , the state prediction value can be updated once again with an equation similar to Equation 3.

[Equation 5]

In addition, it can be seen through Equation 6 that x ^{i +} _t in the ith ensemble model can be corrected using the Kalman gain K ^x _t and perturbed observations at time t.

[Equation 6]

All of Equations 1 to 6 are related to the dual ensemble Kalman filter, and in the case of a single ensemble Kalman filter, all equations except for Equations 2, 4, and 5 may be equally applied. Here, the noise parameter of Equation 2 may not be considered, Equation 4 is assumed to be θ ^{i +} _t = θ ^{i -} _t , and y ^{i +} _t = y ^{i -} _t of Equation 5. Can be simplified and considered there is.

Combining the above six equations, the final form of the ensemble Kalman filter can be expressed as:

[Equation 7]

[Equation 8]

In an embodiment of the invention, eight types of replacement filters are proposed to replace the Ensemble Kalman Filter.

Here, eight surrogate filters based on PCE (Polynomial Chaos Expansion) theory can be presented to replace the Ensemble Kalman Filter. The replacement filter proposed through the embodiment can achieve high accuracy with low computational cost and apply the same basic assumptions as the implementation of the ensemble Kalman filter. The difference between the 8 filters can be explained through FIG. 1 .

1 is a diagram for explaining configurations of replacement filters proposed in an embodiment.

In the embodiment, the process of generating 8 different replacement filters to be proposed is shown, and the internal processes of the existing ensemble Kalman filter are 4 PCE models (PCE-I, PCE-II, PCE-III, PCE-IV) can be replaced with

The first criterion for distinguishing the eight replacement filters of the embodiment is to determine which process to replace in the system of ensemble Kalman filters. Full replacement filters or partial replacement filters with different replacement structures can be proposed. A filter that replaces the whole can be provided by constructing a single replacement filter that can replace the process of the entire ensemble Kalman filter.

Given all the input variables on the right side of Equations 7 and 8 above, the output variables on the left side can be calculated through a substitution filter.

In another embodiment, several replacement filters can be constructed to replace specific processes of the Ensemble Kalman Filter.

Theoretically, it is possible to construct alternative filters for any process by distinguishing all processes in the ensemble Kalman filter. The embodiment proposes a method of building two independent filters for only two of these processes. This process is the most unfavorable in terms of computational efficiency, but corresponds to a process that greatly affects accuracy. The process may correspond to the process of calculating the ensemble state and stream flow propagated over time through the propagation factors f(.) and h(.) in Equations 1, 3 and 5. The process of the ensemble Kalman filter other than these two processes can be applied in the same way as before. This is hereinafter referred to as a Surrogate Partial Filter (SuPF).

In an embodiment, in applying replacement filters, it can be checked if the PCE changes over time.

One of the existing types of PCE is time-variant PCE (VaPCE). It can be continuously reconstructed based on new information about forces and flows in all predictive processes of data assimilation.

The collection of training sets is simpler than other types (i.e. u _t and yt are real data) because the precipitation forces and stream flow are known during the data assimilation process. Substitution filters created using time-variant PCE can be named time-varying substitution filters (VaSuF, namely VaSuWF and VaSuPF). Other types of PCEs are built only once during the calibration period and can be used later during the forecast period. Since this time-invariant PCE (InPCE) does not need to be reconstructed during real-time prediction, applicability and efficiency can be maximized.

InPCE needs to be made to better represent the behavior of the original model under a wide range of conditions. PCE models trained on limited data sets in the past have shown good performance on other events similar to the calibration set, but for events different from those in the training series, it is difficult to construct an alternative model that mimics the original model. These problems can be solved by learning using as much data as possible, but for some (unmeasured) domains, large amounts of data cannot be obtained.

Therefore, in the embodiment, a new procedure for collecting training sets may be proposed to build a time-invariant PCE that does not require measurement data. In an embodiment, all input variables of the filter (model) are expressed in Equations 7 and 8, i.e. model state, model parameters, rainfall measurements and observed emissions are uncertain and vary within a certain range. Instead of arranging them in a predetermined way, input values can be generated randomly through a sampling process. A Latin Hypercube (LHS) sampling technique may be used for this. The process can make the time-invariant PCE fit for as many input conditions as possible. Hereinafter, the substitution filters generated by the time-invariant PCE may be referred to as time-invariant substitution filters (InSuF, ie, InSuWF and InSuPF).

The criterion for applying the filter relates to the amount of information required to generate the PCE, that is, whether to select training information for a single time step or for the entire period. This may include setting the extent to which the generated PCE can replace the original model. That is, the PCE generated using a wider range of training data can replace the results of the original filter under a wider range of simulation conditions.

The description according to the embodiment relates to a single substitution filter and a dual substitution filter.

Another criterion for applying a replacement filter may be based on the goal of data assimilation. It can be determined whether to update only the state vector (corresponding to the single ensemble Kalman filter in Equation 7) or to update the model parameters and state vector (corresponding to the dual ensemble Kalman filter in Equation 8).

Substitute filters for single and dual ensemble Kalman filters are referred to as single substitute filters (e.g. InSuWF, InSuPF, VaSuWF and VaSuPF) and dual substitute filters (double SuF, i.e. double InSuWF, double InSuPF, double VaSuWF and double VaSuPF), respectively. can be called

The mathematical representations of these eight replacement filters are discussed below for clarity. This may include an indication of both the replacement filter and the corresponding PCE. The latter may specifically consist of four time-varying PCEs (VaPCE1, VaPCE2, VaPCE3, and VaPCE4) and four time-invariant PCEs (InPCE1, InPCE2, InPCE3, and InPCE4).

Hereinafter, eight replacement filters will be described in detail.

As shown in Figure 1, the 8 substitution filters proposed in the embodiment are 2 × 2 × 2 sub-cases, that is, the degree of substituting the internal process (partial or whole), the method of learning an alternative model (invariant or variant), and data assimilation It can be configured according to the framework for a total of 8 types according to the target (single or dual) to perform.

First, a single time-varying full replacement filter (VaSuWF) has the same equation as Equation 7, and the entire process set of ensemble Kalman filters can be replaced with a first time-varying PCE (named VaPCE1) where VaSuWF = VaPCE1.

[Equation A1]

[Equation A2]

Second, the dual time-varying full replacement filter (Dual VaSuWF) may have the same mathematical form as Equation 8. The entire process set of dual ensemble Kalman filters can be replaced by a second time-varying PCE (VaPCE2), which can be referred to here as Dual VaSuWF = VaPCE2.

[Equation A3]

[Equation A4]

Thirdly, the single time-varying partial substitution filter (VaSuPF) has the same mathematical form as Equation 7, but the process of calculating x ^i- _t in the latter ensemble Kalman filter is replaced by the third time-varying PCE (VaPCE3) and the process of calculating y ⁱ _t is replaced by the fourth time-varying PCE (VaPCE4), where VaSuPF may include VaPCE3 and VaPCE4.

[Equation A5]

[Equation A6]

[Equation A7]

Fourth, the dual time-varying partial substitution filter (Dual VaSuPF) has the same mathematical form as Equation 8, but the process of calculating x ^{i -} _t in the latter dual ensemble Kalman filter is the third time-varying PCE (VaPCE3 ) can be replaced with The two processes of calculating y ⁱ _t can be replaced with the same fourth time-varying PCE (VaPCE4) as in Equation A7.

[Equation A8]

The other four time-invariant substitution filters can be set up similarly to the time-varying substitution filters, except using time-varying PCE (InPCE).

Fifth, the single time-invariant full substitution filter (Single InSuWF) has the same mathematical form as Equation 7, and the entire process of the ensemble Kalman filter can be replaced with the first time-invariant PCE (InPCE1) where InSuWF = InPCE1.

[Equation A9]

[Equation A10]

Sixth, the dual time-invariant full substitution filter (Dual InSuWF) may have the same mathematical form as Equation 8. The entire process set of Dual EnKF can be replaced by a second time-invariant PCE (InPCE2), where Dual InSuWF = InPCE2.

[Equation A11]

[Equation A12]

Seventh, the single time-invariant partial replacement filter (InSuPF) has the same mathematical form as Equation 7, and the process of calculating x ^{i -} _t in the ensemble Kalman filter is replaced by a third time-invariant PCE (InPCE3) The process of calculating y ⁱ _t can be replaced with the fourth time-invariant PCE.

[Equation A13]

[Equation A14]

[Equation A15]

Eighth, the dual time-invariant partial substitution filter (Dual InSuPF) has the same mathematical form as Equation 8, and the process of calculating x ^{i -} _t in the dual ensemble Kalman filter is the third time-invariant PCE ( InPCE3), and the two processes of calculating y ⁱ _t can be replaced with the same fourth time-invariant PCE (InPCE4) in Equation A15.

[Equation A16]

In an embodiment, PCE may be used to develop the replacement filter described above and handle the load of the process. PCE can represent a stochastic quantity as a convergent polynomial series of random parameters.

Considering the M-th PCE replacement filter, given the input X ⁱ _t and the output Y ⁱ _t M of equations A2, A4, A6, A7, A10, A12, A14 and A15, the relationship between X ⁱ _t and Y ⁱ _t can be explained as follows.

[Equation 9]

In Equation 9, the number of inputs X ⁱ _t and output Y ⁱ _t (each configuration of the PCE output is referred to as a quantity of interest, QoI) is NX and NY, respectively, which may appear differently depending on the 14 PCEs mentioned above . Specifically, for the four PCEs there are NS+NP+2NI+1, NS+NP+2NI+1, NS+NP+NI and NS+NP+NI and PCE4 for PCE1, PCE2, PCE3, respectively, regardless of the value of NX. The corresponding values of NY are NS + 1, NS + NP + 1, NS and 1, respectively. One can then try to approximate M with the Mth PCE expressed as an appropriate polynomial expansion. It can be extended to the construction of a polynomial fundamental term Ψ _a ( X ⁱ _t ), which is a multivariate orthogonal polynomial for the distribution of input X ⁱ _t . ε _α is the deterministic polynomial expansion coefficient for multiple indices α = {α1, ... , α _NX }, which simplifies the notation and can represent the degree of the polynomial term. Three widely used methods for estimating the PCE coefficient ε _α are Gaussian orthogonality, Bayesian compressive sensing, and sparse regression.

In an embodiment, the constructed PCE may include two hyperparameters: the size (experimental design x) N of the training data set and the degree p of the polynomial. It is important to set the values of the hyperparameters before estimating the PCE coefficients, which directly affect the behavior of the original filter. In general, these settings are determined through trial and error, and to solve this problem, in the embodiment, a method is proposed that includes a procedure for identifying 15 ultimate N and p values by continuously increasing the values until the convergence criterion is met. This method represents an extension of Sequential experimental design (SED).

In the general SED approach, the enhancement of the experimental design(x) stops when the accuracy metric reaches the target error, and in some cases it does not satisfy the essential assumption that the accuracy decreases monotonically, so in the embodiment, both N and p are enriched to A polynomial degree (SED-PD) scheme can be provided that extends the existing method so that the stopping (convergence) criterion can be sequentially met.

SED-PD according to the embodiment consists of two subsequent iteration cycles for determining the optimal values of N and p, where the sub-loop of p is affected by repetition of N, and the description of the SED-PD scheme is shown in FIG. 2 can be referenced.

2 is a flowchart for explaining an optimization method for building a PCE in an embodiment.

The embodiment is configured in a manner including two iterative loops to optimize hyperparameters N and p at the same time, and also to prevent non-convergence during optimization by presenting four criteria.

The inner loop is for optimizing the value of p, and the outer loop can be used to determine values for optimizing N and p, including the inner loop. N ^* means an increment that determines the amount of samples added for each iteration loop.

At step 201, the experimental design size N may be initialized to a workable number.

In an embodiment, selecting an initial value of N that is too small will take a long time to converge, and conversely, if the initial value is too large, the value may not converge at all. Therefore, it is important to choose an appropriate number for the initial value of N. In general, the optimal size of the experimental design can be determined by the complexity of the original model and the available computational cost.

At step 202, according to the initialized N, the experimental design x is sampled according to the LHS technique, and a corresponding response can be calculated.

In step 203, another hyperparameter, polynomial degree p, may be initialized. Here, p=1 may be initialized.

At step 204, a PCE candidate for each QoI may be built based on the given values of N and p.

At step 205, accuracy may be evaluated.

In an embodiment, a statistic may be introduced to quantify the difference between the results of the original filter and the replacement filter, determine the degree of convergence, and evaluate the performance of the replacement filter, and a statistic for QoI corresponding to each iteration loop may be introduced. . A method for determining the accuracy of each QoI may refer to Equation 10 below.

[Equation 10]

Here, h _k is the k-th diagonal term of matrix F(F ^T ) ^-1 F ^T , and matrix F is an information matrix. Embodiments are designed to quantify exactly how the surrogate model behaves by calculating the deviations between these model outputs compared to the original model, and can detect overfitting.

In step 206, it may be checked whether p meets the criterion.

The period of the inner loop refers to steps 204 to 206, and allows to determine the product t given a value of N. If the convergence (stop) criterion for p is not met, the inner loop can be run again for a new p incremented by 1 (p increased by 1). This loop may be repeated until a convergence criterion based on accuracy is met.

In step 207, it may be checked whether N meets the criterion.

When the inner loop ends, the outer loop starts, which includes steps 202-207. The outer loop can include the previous inner loop to determine the optimal values of the two hyperparameters. A similar determination can be made as to whether the other criterion for N is met for a fixed value of p determined in the inner loop.

If the criterion is not met, the new N is incremented by N* (i.e., l _N is increased by 1), returning to step 202, and a new sample of size N _* is added to the existing samples in the experimental design. can Steps 202 through 207 can then be repeated until the criterion for N is met. Each time a new value of N is chosen and the outer loop runs, p can be determined anew within the inner loop.

Both criteria must be met to complete the SED-PD loop, and optimal values of N and p can be determined for each QoI.

Hereinafter, convergence criteria for stopping the SED-PD loop will be described.

In an embodiment, multiple convergence criteria based on an error metric may be proposed to stop the iteration of the loop. This can be linked to an optimization problem to maximize the accuracy of PCE while minimizing the computational cost required for PCE construction.

3 is a diagram showing stopping the iteration of a loop according to a convergence criterion in an embodiment.

According to the sequential procedure of SED-PD, the convergence criterion can be applied twice to the selection of N and p, respectively. Four criteria are proposed for this, and if at least one of the four criteria is satisfied, the loop for determining N and p can be stopped.

In an embodiment, the origin of each line represents an iteration from which a value of p or N is to be determined.

^QoI _LOO 값이 충분히 작은 목표 오차에 도달하는 것이다. 해당 값은 미리 정해질 수 있다. 최적의 값은

^QoI.l _LOO 값이 iteration l(l = {lp, lN})(그림 3 참조)(하한 임계값보다 작음)에서 p 및 N의 값을 선택하여 구체적으로 결정될 수 있고, 수학식 11과 같이 나타낼 수 있다.The first criterion (criterion 1) is,

^{The QoI} _LOO value reaches a sufficiently small target error. The corresponding value may be determined in advance. the optimal value is

^{The QoI.l} _LOO value can be specifically determined by selecting the values of p and N in iteration l (l = {lp, lN}) (see Figure 3) (less than the lower limit threshold), expressed as in Equation 11 can

[Equation 11]

The first criterion is only valid if errors are expected to decrease monotonically. However, since the complexity of the model is affected by many sources of uncertainty, the first criterion may not be satisfied within a finite number of iterations. Also, excessively increasing the number of N and p does not always provide a better surrogate model, and a second criterion can be proposed to avoid this overfitting phenomenon. The second criterion may refer to Equation 12.

[Equation 12]

^QoI.l-2 _LOO,

^QoI.l-1 _LOO 및

^QoI.l _LOO는 각각 연속되는 반복에서 계산된 오차 추정치이다. 반복이 매우 일찍 중단되거나, 최적의 값이 여전히 큰 경우를 방지하기 위해 보호 수단으로 또 다른 임계값인

^upper _th를 도입하고 예컨대,

^upper _th를 0.1로 설정할 수 있다. 최적의 값은

^QoI _LOO가 가장 적은 반복으로 p 또는 N 값으로 결정될 수 있다. 도 3의 실시예에 의하면, l-2에서 p 또는 N 값이 선택될 수 있다.here,

^QoI.l-2 _LOO ,

^QoI.l-1 _LOO and

^QoI.l _LOO is the error estimate computed at each successive iteration. Another threshold value, as a safeguard, to prevent iterations from stopping too early, or when the optimal value is still large.

Introduce ^upper _th and e.g.

You can set ^upper _th to 0.1. the optimal value is

^QoI _LOO can be determined as p or N value with the fewest iterations. According to the embodiment of FIG. 3 , a value of p or N may be selected in 1 -2.

In an embodiment, if the error is not small and hardly decreases, the process may not converge. You can account for this problem by calculating the degree of reduction, or slope. This may be determined according to the slope of Equation 13 below, which is calculated using the error estimated in successive iterations by the second criterion. The third criterion can be expressed by Equation 13 below.

[Equation 13]

^slope _th는 0.05로 기울기가 거의 없는 상태의 경우가 적용될 수 있다. p 또는 N

^QoI _LOO가 가장 작을 때 반복에서 결정될 수 있다. 예를 들어, 도 3에서 - 1에서 p 또는 N 값이 선택될 수 있다.here,

As ^slope _th is 0.05, the case of almost no slope can be applied. p or N

It can be determined in an iteration when ^{the QoI} _LOO is the smallest. For example, a value of p or N at −1 in FIG. 3 may be selected.

If convergence is not achieved even by the criteria described above, the criteria of Equation 14 below may be applied.

[Equation 14]

^QoI _LOO가 가장 작을 때 반복에서 결정될 수 있다.In an embodiment, the values of p and N may be limited to avoid infinite repetition of SED-PD. Here, the maximum polynomial order p _max may be applied. The maximum number of experimental designs N _max may be set to (p _max + 1) ^NX . p or N

It can be determined in an iteration when ^{the QoI} _LOO is the smallest.

4 is a diagram showing which criterion is used in an embodiment.

The number of iterations for each QoI (y-axis) when optimizing the criterion and hyperparameters used for stopping the loop according to the embodiment is shown, and p (large box on the left of each subplot) and N (each subplot) large box on the right of the plot).

The subplots in (a) to (h) may correspond to configuring different PCEs using the SED-PD method. One of the four stopping criteria is indicated by the color of each cell for p and N. The number of iterations l _p and l _N to optimize p and N are indicated inside each cell.

Referring to the colors shown in the embodiment, only the first criterion is not valid, and the loop can be stopped according to the second to fourth criteria.

5 is a diagram comparing parameter operation results of an existing dual ensemble Kalman filter and four Dual SuKFs in an embodiment.

Figure 5 (a) shows the operation of Dual EnKF, Figure 5 (b) Dual InSuWF, Figure 5 (c) Dual InSuPF, Figure 5 (d) Dual VaSuWF, and Figure 5 (e) shows the operation of Dual VaSuPF.

As shown, as a result of comparing the posterior distributions of the 9 parameters, it can be seen that among the 4 replacement filters, the remaining 3 replacement filters except Dual InSuWF well reproduce the operation of the existing Kalman filter.

6 is a diagram comparing the accuracy of an existing ensemble Kalman filter, a dual ensemble Kalman filter, and 8 alternative filters in an embodiment.

Figure 6 (a) is the result of comparing predictions for 500 ensemble river flows using 90% confidence intervals, and Figures 6 (b) to (d) show the results of conventional ensemble Kalman filters and This is the result of comparing the accuracy of a total of 10 alternative filters. FIG. 6(b) relates to NSE, FIG. 6(c) to PE, and FIG. 6(d) to CRPS.

As shown, the results of Dual EnKF, Dual VaSuWF, Dual VaSuPF, and Dual InSuPF are more accurate than the existing filters, and the accuracy of Dual VaSuWF, Dual VaSuPF, and Dual InSuPF is similar to that of the existing Dual EnKF. there is.

FIG. 7 is a diagram showing prediction comparison of flow by a data assimilation experiment of an existing ensemble Kalman filter, a dual ensemble Kalman filter, and 8 replacement filters in an embodiment.

8 is a diagram comparing the accuracy of the event of FIG. 7 in an embodiment.

In the actual data assimilation experiment for Event 2 at the top and Event 3 at the bottom of FIG. 7 , predictions of 500 ensemble flows using 90% confidence intervals for each of the 10 filters can be compared.

8(a) to 8(c) compare the accuracy matrices for 10 filters with an ensemble size of 500 in an actual data assimilation experiment for event 2 and FIG. 8(d) to FIG. 8(f) for event 3 shows a result. 8(a) and 8(d) relate to NSE, FIGS. 8(b) and 8(e) relate to PE, and FIGS. 8(c) and 8(f) relate to CRPS.

It can be seen that the embodiment shows results similar to those in the drawing of FIG. 6 .

Figure 9 compares the relative accuracies of eight alternative filters in an example.

Relative differences can be compared for the five evaluation metrics on the x-axis for event 2 in FIG. 9 (a) and event 3 in FIG. 9 (b). These comparison pairs can be determined based on the three criteria of alternative filters.

Among them, the positive or negative values of the points comparing InSuWF and InSuPF indicate that the prediction results of the dual filter, partial filter, and time-invariant filter are relatively more accurate than the results calculated by the full filter, time-varying filter, and single filter, respectively. there is.

10 is a diagram for explaining relative differences with respect to metrics having different golden times in an embodiment.

FIG. 10(a) shows five evaluation metrics with different lead times (golden time) of 1 to 6 hours for event 2 and FIG. 10(b) for event 3.

The values of each subplot can be compared against the best of 60 (10 filters x 6 lead times) closest to the ideal value for each evaluation metric. Negative values of the metric indicate the degree of performance degradation compared to the best value.

Securing the lead time is a very important issue in disaster forecasting problems such as floods, but generally, the accuracy of the forecast decreases as the lead time increases.

In an embodiment, in the case of provided Dual VaSuWF, Dual InSuPF, and Dual VaSuPF, even if lead time increases, prediction accuracy may be maintained.

11 is a diagram showing a result of comparing efficiencies between conventional ensemble Kalman filters and replacement filters in an embodiment.

The result according to the embodiment is the result for event 2. As a result of comparing the efficiencies of the existing ensemble Kalman filters and the proposed alternative filters, it can be confirmed that the calculation speed of Dual InSuPF is about 500 times faster than that of the existing Dual EnKF. You can check out this good thing.

12 is a diagram for explaining a difference between an existing SED technique and a proposed SED-PD technique in an embodiment.

In the embodiment, in SED, the polynomial order p is sequentially fixed to a value of 1 to 7, and iterations for N can be set to be performed up to 1000 in the case of VAPCE1 and 100 in the case of InPCE.

In an embodiment, in the subplots Fig. 12(b) and Fig. 12(d) enlarged in SED-PD, dotted lines separate repetitions of the outer loop, respectively, and solid lines separate repetitions of the inner loop. For VAPCE1, we show the results at t=50 for event 2.

It is shown that when the optimization is performed using the existing SED technique, convergence is not reached.

13 is a flowchart illustrating a data assimilation method performed in an apparatus for data assimilation according to an embodiment.

Data assimilation can be performed using the replacement filter described above with reference to FIGS. 1 to 12 .

In step 1310, the device identifies the type of object for data assimilation.

In an embodiment, the type of data assimilation object can be checked to determine one of the eight filters described above.

For example, you can check the parameters of data assimilation (single or dual), check the degree of imputation of the process (partial or whole), and consider how the substitute model was learned (invariant or variant).

In step 1320, the device determines a replacement filter in which at least a part of the ensemble Kalman filter is replaced according to the type.

In embodiments, replacement filters may be determined according to the type identified. A method of determining a replacement filter according to an embodiment may refer to the method of FIG. 1 .

In step 1330, the device performs data assimilation of the object by applying a replacement filter.

In the embodiment, by using one of the eight types of replacement filters determined according to the degree of replacement of the internal process, the method of learning the replacement model, and the subject of data assimilation, compared to the existing dual ensemble Kalman filter, equivalent or It can provide a data assimilation method with high accuracy and fast computational performance.

In an embodiment, an SED-PD technique equipped with a dual optimization system may be provided. During the construction of PCE, four intermediate criteria are required to solve the two problems that occur in existing SEDs. Through this, various criteria can be introduced to ensure the convergence of the optimization process. It also solves the problem of having to choose the value of the polynomial order ad-hoc or trial-and-error in the optimization process.

In an embodiment, various frameworks for constructing replacement filters may be presented. In the past, whole (replacing the entire process of the original filter) and variant (requiring reconstruction at each time step) approaches have been commonly used, but these approaches can degrade prediction performance in terms of accuracy and efficiency. We reduce the problem of creating an entire dimension into one PCE through partial replacement filters by proposing partial replacement (replaces only part of the process of the original filter) and time-invariant replacement (valid for the entire period) schemes that outperform existing approaches. We can turn to the problem of generating multiple PCEs with a single dimension, and as a result, the number of dimensions can be drastically reduced. Additionally, real-time predictions can be made possible through a time-invariant approach that allows replacement filters to be created during the non-flood season.

Among the alternative filters provided in the embodiment, these three filters, Dual VaSuWF, Dual VaSuPF and Dual InSuPF, have reduced predictive performance with the lead time of all filters, but do not degrade with longer lead times.

In particular, a dual time-invariant replacement filter (i.e., Dual InSuPF) can successfully replace the existing ensemble Kalman filter and can provide good performance in terms of efficiency and accuracy. Ultimately, in the method of predicting floods in real time, it not only provides accurate prediction results equivalent to existing filters, but also significantly reduces the computational burden in larger ensemble predictions.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (17)

In the operation method of the replacement filter for data assimilation,
Calculating a state prediction value before correction based on the InPCE3 model;
based on the InPCE4 model, correcting a current observation from the state estimate;
updating ensemble parameters of the replacement filter;
determining an observation based on the updated ensemble parameter based on the InPCE4 model; and
Updating the state estimate based on the determined observation.
including,
How substitution filters work.
According to claim 1,
The step of calculating the predicted state before the correction,
Calculating the state prediction value before the correction based on the equation below
including,
How substitution filters work.
Equation:

-x ^- _t is the i-th state prediction predicted at time t, x ⁱ⁺ _t-1 is the i-th state prediction updated at time t-1, and θ ⁱ⁺ _t-1 is the i-th state prediction updated at time t-1 is the ensemble parameter, where u ⁱ _t-1 is the ith actual physical force at time t-1, and w ⁱ _t-1 is the ith model error at time t-1-
According to claim 1,
The step of correcting the current observation from the state estimate,
Correcting the observation based on the equation below
including,
How substitution filters work.
Equation:

identifying the type of object for data assimilation;
determining a replacement filter in which at least a part of the ensemble Kalman filter is replaced, according to the type; and
Performing data assimilation of the target by applying the replacement filter
including,
Material assimilation method.
According to claim 4,
The step of confirming the type of data assimilation,
checking the variables of the data assimilation;
checking the replacement degree of the data assimilation process; and
Confirming the learning method of the alternative model of the data assimilation
including,
Material assimilation method.
According to claim 5,
The step of checking the variables of the data assimilation is
Checking whether the variable is a state variable or a parameter
including,
Material assimilation method.
According to claim 5,
The step of confirming the learning method of the alternative model of data assimilation,
Confirming whether the alternative model is a time-varying learning method that is reconstructed for each time unit or a time-invariant learning method that is effective at all times
including,
Material assimilation method.
According to claim 5,
The step of confirming the degree of replacement of the data assimilation process,
Confirming the degree to which the entire data assimilation process or a part of the data assimilation process is replaced
including,
Material assimilation method.
According to claim 1,
Determining a replacement filter in which at least a part of the ensemble Kalman filter is replaced,
Determining at least one replacement filter to be applied to the ensemble Kalman filter among eight replacement filters provided according to the data assimilation type
including,
Material assimilation method.
A computer program stored in a computer readable medium to be combined with hardware to execute the method of any one of claims 1 to 9.
In the apparatus for data assimilation,
one or more processors;
Memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors;
said program,
identifying the type of object for data assimilation;
determining a replacement filter in which at least a part of the ensemble Kalman filter is replaced, according to the type; and
Performing data assimilation of the target by applying the replacement filter
including,
Device.
According to claim 11,
The step of confirming the type of data assimilation,
checking the variables of the data assimilation;
checking the replacement degree of the data assimilation process; and
Confirming the learning method of the alternative model of the data assimilation
including,
Device.
According to claim 12,
The step of checking the variables of the data assimilation is
Checking whether the variable is a state variable or a parameter
including,
Device.
According to claim 12,
The step of confirming the learning method of the alternative model of data assimilation,
Confirming whether the alternative model is a time-varying learning method that is reconstructed for each time unit or a time-invariant learning method that is effective at all times
including,
Device.
According to claim 12,
The step of confirming the degree of replacement of the data assimilation process,
Confirming the degree to which the entire data assimilation process or a part of the data assimilation process is replaced
including,
Device.
According to claim 11,
Determining a replacement filter in which at least a part of the ensemble Kalman filter is replaced,
Determining at least one replacement filter to be applied to the ensemble Kalman filter among eight replacement filters provided according to the data assimilation type
including,
Device.
In the replacement filter for data assimilation,
one or more processors;
Memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors;
said program,
Calculating a state prediction value before correction based on the InPCE3 model;
Based on the InPCE4 model, correcting the current observation from the state estimate;
updating ensemble parameters of the replacement filter;
determining an observation based on the updated ensemble parameter based on the InPCE4 model; and
Updating the state estimate based on the determined observation.
including,
alternative filter.

Images