WO2024071377A1

WO2024071377A1 - Information processing device, information processing method, and program

Info

Publication number: WO2024071377A1
Application number: PCT/JP2023/035615
Authority: WO
Inventors: 領大西; 勇輝安田
Original assignee: 国立大学法人東京工業大学
Priority date: 2022-09-29
Filing date: 2023-09-29
Publication date: 2024-04-04
Also published as: JP2024049645A

Abstract

Provided is an information processing device capable of accurately and efficiently predicting the state of an environment. A structure conversion unit (30) converts the structure of observation data, obtained by observing a temporal/spatial state, into observation data of a lattice data structure. A potential time/space mapping unit (40) performs mapping from a first actual time/space to a potential time/space, with respect to prediction data as well as the observation data that has been converted into the lattice data. A non-linear conversion unit (50) performs non-linear conversion on the observation data and the prediction data that have been subjected to the mapping, in the potential time/space. A high-resolution analysis data acquisition unit (60) performs mapping from the potential time/space to a second actual time/space with respect to the observation data and the prediction data that have been subjected to the non-linear conversion, and acquires high-resolution analysis data.

Description

Information processing device, information processing method, and program

This disclosure relates to an information processing device, an information processing method, and a program.

For example, in the fields of weather forecasting or oceanographic forecasting, there is a demand for accurate and efficient prediction of environmental conditions through simulations, etc. Similarly, there is a demand for efficient prediction of environmental conditions within devices. Methods for improving prediction accuracy include increasing resolution (super-resolution) and data assimilation. Related to this technology, Non-Patent Document 1 discloses a technology called super-resolution data assimilation (SRDA). Furthermore, Non-Patent Documents 2 to 5 disclose technologies related to the present disclosure.

The technology in Non-Patent Document 1 simply combines a super-resolution technique with a data assimilation technique. Therefore, in Non-Patent Document 1, data assimilation is performed by ensemble calculation. Here, in ensemble calculation, it is necessary to simulate various similar situations. Therefore, with the technology in Non-Patent Document 1, there is a risk that the calculation costs will increase if an attempt is made to make accurate predictions. Therefore, with the technology in Non-Patent Document 1, there is a risk that it will not be possible to make accurate and efficient predictions.

The present disclosure aims to provide an information processing device, information processing method, and program that can accurately and efficiently predict the state of the environment.

The information processing device according to the present disclosure has a structure conversion unit that converts the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure that indicates numerical values defined on lattice points arranged at a predetermined interval in space-time; a latent space-time mapping unit that maps the observation data converted into lattice data and prediction data, which is lattice data in space-time obtained by simulation and includes at least the time of the observation data and a time before the observation data, from a first real time-space to a latent space-time having a smaller number of elements than the first real time-space; a nonlinear transformation unit that performs a nonlinear transformation on the observation data and the prediction data that have been mapped in the latent space-time; and a high-resolution analysis data acquisition unit that acquires high-resolution analysis data that is lattice data in space-time and has a higher resolution in space-time than the prediction data by mapping the observation data and the prediction data that have been nonlinearly transformed from the latent space to a second real time-space that has a larger number of elements than the latent space-time and has a higher resolution than the first real time-space, and data assimilation of the observation data and the prediction data is performed by the latent space-time mapping unit and the nonlinear transformation unit.

　The information processing method according to the present disclosure converts the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure showing numerical values defined on lattice points arranged at a predetermined interval in space-time, maps the observation data converted into lattice data and predicted data, which is lattice data in space-time obtained by simulation and includes at least the time of the observation data and a time before the observation data, from a first real-time space to a latent-time space having fewer elements than the first real-time space, performs a nonlinear transformation on the mapped observation data and the predicted data in the latent-time space, and maps the nonlinearly transformed observation data and the predicted data from the latent-time space to a second real-time space having a larger number of elements than the latent-time space and a higher resolution than the first real-time space, thereby obtaining high-resolution analysis data that is lattice data in space-time and has a higher resolution in space-time than the predicted data, and performs mapping from the first real-time space to the latent-time space, and performs a nonlinear transformation on the observation data and the predicted data in the latent-time space, thereby performing data assimilation between the observation data and the predicted data.

The program disclosed herein includes a process for converting the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values defined on lattice points arranged at a predetermined interval in space-time; a process for mapping the observation data converted into lattice data and predicted data, which is lattice data in space-time obtained by simulation and is at least for the time of the observation data and a time including the past of the observation data, from a first real time-space to a latent time-space having a smaller number of elements than the first real time-space; and a process for mapping the mapped observation data and predicted data in the latent time-space. and a process of mapping the nonlinearly transformed observation data and the predicted data from the latent space-time to a second real space-time that has a greater number of elements than the latent space-time and a higher resolution than the first real space-time, thereby obtaining high-resolution analysis data that is lattice data in space-time and has a higher resolution in space-time than the predicted data. The process of mapping from the first real space-time to the latent space-time and the process of performing a nonlinear transformation on the observation data and the predicted data in the latent space-time perform data assimilation between the observation data and the predicted data.

The present disclosure provides an information processing device, information processing method, and program that can accurately and efficiently predict the state of the environment.

FIG. 13 is a diagram for explaining calculation resolution and grid data. FIG. 13 is a diagram for explaining calculation resolution and grid data. FIG. 13 is a diagram for explaining a super-resolution simulation method. 1 is a diagram showing a configuration of an information processing device according to an embodiment of the present invention; 4 is a flowchart showing an information processing method executed by the information processing device according to the present embodiment. 1 is a diagram illustrating a configuration of an information processing device according to a first embodiment. FIG. 1 is a diagram for explaining a technique according to a comparative example. FIG. 2 is a diagram for explaining super-resolution and data assimilation according to the first embodiment. 11 is a diagram comparing experimental results according to the first embodiment with experimental results according to a comparative example. FIG. 1 is a diagram for explaining a method of learning components according to the first embodiment using a variational Bayes method. FIG. FIG. 1 is a block diagram illustrating an example of the hardware configuration of a calculation processing device capable of realizing an apparatus and a system according to each embodiment.

(Overview of the embodiment)
Prior to describing the embodiment, an outline of the embodiment will be described. Note that, although the embodiment will be described below, the invention according to the claims is not limited to the following embodiment. Also, not all of the combinations of features described in the embodiment are necessarily essential to the solution of the invention. Also, in the following description, indexes (such as English letters) used are not necessarily common throughout this specification.

For example, in the field of weather forecasting (weather forecast simulation), micrometeorological forecasts are sometimes applied. Micrometeorological forecasts refer to the weather near the ground up to an altitude of about 100 m, which is heavily influenced by artificial structures and human activities. Micrometeorological forecasts provide simulation results with a resolution of about 100 to 1000 times higher than that of general weather forecasts. Micrometeorological forecasts are mainly provided for urban areas, but their application is not limited to cities. Because of their ultra-high resolution, micrometeorological forecasts can incorporate flows past buildings and heat exhaust from buildings, which are not considered in normal weather forecasts. In other words, micrometeorological forecasts can simulate atmospheric flows that are closer to reality. In the near future, micrometeorological forecasts will likely obtain observational data from sensors, cameras, drones, smartphones, etc. placed in urban environments, and use these observational data to make predictions. In order to make such predictions accurately, it is necessary to increase the resolution of the calculations.

Figures 1 and 2 are diagrams for explaining calculation resolution and grid data. Figure 1 shows a three-dimensional calculation mesh G1. The three-dimensional calculation mesh G1 is represented by a grid corresponding to a three-dimensional space defined by a three-dimensional coordinate space of the X-axis, Y-axis, and Z-axis. The shorter the grid spacing, the higher the resolution of the calculation in the spatial direction. Conversely, the longer the grid spacing, the lower the resolution of the calculation in the spatial direction.

FIG. 2 shows a four-dimensional computation mesh G2. The four-dimensional computation mesh G2 is configured such that the three-dimensional computation mesh G1 is arranged in the time direction (shown by the T axis), that is, in a time series. Here, the shorter the time interval (sampling period; the interval between _T1 and _T2 in FIG. 2), the higher the resolution of the computation in the time direction. Conversely, the longer the time interval, the lower the resolution of the computation in the time direction. In this embodiment, the "space-time" is described as a four-dimensional space-time defined by three-dimensional space and one-dimensional time, but the dimension of the space-time is not limited to four dimensions.

Here, data indicating the numerical values of physical quantities, etc. in the four-dimensional computation mesh G2 can be regarded as lattice data in four-dimensional space-time. The lattice data indicates the numerical values of physical quantities (velocity, etc.) defined on lattice points arranged at a predetermined interval in space-time. That is, at each point in three-dimensional space (three-dimensional computation mesh G1), there is a numerical value (physical quantity, etc.) indicating the state of that point, and as shown in the four-dimensional computation mesh G2, the numerical value of each point changes in the one-dimensional time direction. The change in the time direction of each point in this three-dimensional space is indicated by lattice data. In this case, the lattice data can be expressed as a four-dimensional data array (numerical array) indicating the numerical values of the physical quantities, etc. The number of elements in the data array is called the number of elements. Also, a data array can be provided for each physical quantity. Also, the lattice data can be called structured data. Also, when lattice data is handled in a neural network described later, the lattice data may indicate physical quantities that humans can understand, or may indicate numerical values that humans cannot understand. In other words, the lattice data has a structure indicating numerical values defined on lattice points arranged at a predetermined interval in space-time.

On the other hand, when micrometeorological forecast simulations are performed with a high calculation resolution, the amount of data to be handled increases, resulting in huge calculation costs. Therefore, it is possible to obtain high-resolution (HR: High Resolution) calculation results by performing super-resolution (SR: Super Resolution) on low-resolution (LR: Low Resolution) simulation results.

Figure 3 is a diagram for explaining the super-resolution simulation method. The super-resolution simulation system uses a super-resolution device to perform super-resolution on the low-resolution prediction results obtained by performing a low-resolution simulation. This results in a high-resolution prediction result. The super-resolution device learns through deep learning (neural network) using the high-resolution results obtained in advance from a high-resolution simulation. In other words, the super-resolution device learns by performing supervised learning using a large amount of high-resolution results as training data in advance. With this configuration, during operation, high-resolution prediction results can be obtained by performing a low-resolution simulation, thereby reducing calculation costs.

In addition, in order to improve the accuracy of the prediction simulation, it is possible to perform data assimilation between the observation data and the prediction results. In the ensemble Kalman filter, which is one of the data assimilation methods, a large number of similar situations are simulated by ensemble calculation, and the error is estimated from the variance of the prediction results. Then, based on the magnitude of this error, the degree to which the prediction results should be brought closer to the observation data is determined. By performing data assimilation, the prediction results can be brought closer to the observation data, making it possible to improve the accuracy of the prediction results. Here, as described above, the technology in Non-Patent Document 1 simply combines the super-resolution method and the data assimilation method. In other words, in Non-Patent Document 1, super-resolution and data assimilation are performed independently. In such a method, it is necessary to perform data assimilation by ensemble calculation. Therefore, the calculation cost increases.

In contrast, in this embodiment, as described below, super-resolution and data assimilation are performed simultaneously using time-series data of grid data. This makes it unnecessary to perform ensemble calculations. Therefore, in this embodiment, it becomes possible to perform predictions with high accuracy and efficiency.

FIG. 4 is a diagram showing the configuration of an information processing device 10 according to this embodiment. The information processing device 10 is, for example, a computer. The information processing device 10 has a simulation unit 20, an observation data acquisition unit 22, a prediction data acquisition unit 24, a structural transformation unit 30, a latent space-time mapping unit 40, a nonlinear transformation unit 50, a high-resolution analysis data acquisition unit 60, and a low-resolution analysis data calculation unit 70. These components can be realized by a hardware configuration described later. The functions of these components will be described later.

FIG. 5 is a flowchart showing an information processing method executed by the information processing device 10 according to this embodiment. The simulation unit 20 performs a simulation of the state of the environment (step S20). Specifically, the simulation unit 20 performs a low-resolution simulation as described above. More specifically, the simulation unit 20 performs a low-resolution simulation in the time direction and the spatial direction.

The observation data acquisition unit 22 acquires one or more types of observation data (step S22). The observation data is data obtained by observing a state in time and space. The observation data can be acquired, for example, from sensors and cameras placed in the environment, drones, smartphones, etc. Here, the structure of the observation data does not need to be a lattice data structure. Furthermore, if the structure of the observation data is a lattice data structure, the resolution is arbitrary and may be low resolution or high resolution. Details of the observation data will be described later.

The prediction data acquisition unit 24 acquires the prediction data (step S24). Specifically, the prediction data acquisition unit 24 acquires the prediction data, which is the result of the simulation by the simulation unit 20. Here, the prediction data is time series data indicating the change in state over time. The prediction data is also lattice data in space-time obtained by the simulation. The prediction data is prediction data for a time (time series) that includes at least the time of the observation data and a time prior to that time. Here, the "time of the observation data" includes the latest time (reference time) in all the observation data. Details of the prediction data will be described later.

The structure conversion unit 30 converts the structure of the observation data (step S30). Specifically, the structure conversion unit 30 converts the structure of the observation data into observation data with a lattice data structure. More specifically, the structure conversion unit 30 converts the structure of the observation data, which is data obtained by observing a state in space-time, into observation data with a lattice data structure that indicates numerical values defined on lattice points arranged at predetermined intervals in space-time. The function of the structure conversion unit 30 will be described in detail later.

The latent space-time mapping unit 40 maps the observation data and prediction data into latent space-time (step S40). Specifically, the latent space-time mapping unit 40 maps the observation data converted into lattice data in the process of S30 and the prediction data obtained in the process of S24 from the first real space-time into latent space-time. As described below, data assimilation between the observation data and the prediction data is performed by the latent space-time mapping unit 40 (processing of S40).

Here, the latent space-time is a space-time with fewer elements than the first real space-time. In other words, the latent space-time is a space-time with lower resolution than the first real space-time. Therefore, the number of elements of the data array in the latent space-time is fewer than the number of elements of the data array in the first real space-time. The data in the latent space-time can be composed of a numerical array that compresses the time and space information for the observed data and the predicted data. Note that the latent space-time can also be said to be a latent space that includes the concept of time (time series). Furthermore, dimensions are not distinguished in the latent space-time. In other words, the latent space does not distinguish between time and space, nor does it distinguish between the dimensions of three-dimensional space. Furthermore, the first real space-time is a space-time corresponding to the environment in which the predicted data is obtained.

The latent space-time mapping unit 40 can also obtain data in which observation data (observation data converted into lattice data) is mapped into latent space-time, and data in which prediction data is mapped (projected) into latent space-time, by mapping to latent space-time as described above. The latent space-time mapping unit 40 may use the observation data converted into lattice data to map prediction data into latent space-time. Therefore, "data in which prediction data is mapped into latent space-time" may also include data obtained by mapping observation data into latent space-time. On the other hand, when mapping observation data, the latent space-time mapping unit 40 maps the observation data alone into latent space-time.

By mapping the predicted data using the observation data converted into lattice data, data assimilation between the observation data and the predicted data is performed. In other words, when the observation data converted into lattice data and the predicted data are mapped into the latent space-time, they may be mixed (fused). In other words, in the latent space-time, the observation data converted into lattice data is incorporated into the predicted data. The functions of the latent space-time mapping unit 40 will be described in more detail later.

The nonlinear transformation unit 50 performs a nonlinear transformation in the latent space-time (step S50). Specifically, the nonlinear transformation unit 50 performs a nonlinear transformation on the observation data and the predicted data that have been mapped in the latent space-time. More specifically, the nonlinear transformation unit 50 may fuse the observation data and the predicted data that have been mapped to the latent space-time to obtain data in the latent space-time (latent space-time data; fused data). Thus, the nonlinear transformation unit 50 (processing of S50) performs data assimilation between the observation data and the predicted data. The nonlinear transformation unit 50 may also repeat the nonlinear transformation to make the distribution of the values of the latent space-time data mapped to the latent space-time discontinuous. The nonlinear transformation unit 50 may also make the distribution of the values of the latent space-time data mapped to the latent space-time by the nonlinear transformation complex or simple. In this case, the nonlinear transformation unit 50 may perform a nonlinear transformation so that super-resolution is appropriately performed. Note that the nonlinear transformation unit 50 may not change the number of elements when changing the distribution of the values. Note that super-resolution can be performed by increasing the number of elements. The functions of the nonlinear conversion unit 50 will be described in more detail later.

The high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data (step S60). Specifically, the high-resolution analysis data acquisition unit 60 maps the observation data and prediction data that have been subjected to nonlinear transformation from the latent time space to the second real time space. In this way, the high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data. In other words, the high-resolution analysis data is analysis data that has been subjected to super-resolution in the time direction and the space direction.

Here, the second real-time space is a real-time space having a larger number of elements than the latent time space and a higher resolution than the first real-time space. Therefore, the second real-time space can be said to be a high-resolution space (HR space). The high-resolution analysis data (HR analysis data) is lattice data in time and space. The high-resolution analysis data is data with a higher resolution in time and space than the prediction data. The high-resolution analysis data may be time series data in a time (time series) including the past and future of the time of the observation data. That is, when the input prediction data exists in the future with respect to the time of the observation data (reference time), the high-resolution analysis data is analysis data including a similar future time. That is, the prediction data may be prediction data in a time including the time of the observation data and the past and future of that time. In this case, the high-resolution analysis data acquisition unit 60 may acquire high-resolution analysis data in a time including the past and future of the time of the observation data. A more detailed function of the high-resolution analysis data acquisition unit 60 will be described later.

The low-resolution analysis data calculation unit 70 calculates the low-resolution analysis data (step S70). Specifically, the low-resolution analysis data calculation unit 70 calculates the low-resolution analysis data using the high-resolution analysis data. Here, the low-resolution analysis data (LR analysis data) is analysis data with a lower resolution in time and space than the high-resolution analysis data. More specifically, the low-resolution analysis data calculation unit 70 calculates the low-resolution analysis data by performing arithmetic operations (mathematical methods) such as algebraic interpolation on the high-resolution analysis data. A more detailed description of the functions of the low-resolution analysis data calculation unit 70 will be given later.

The simulation unit 20 performs a simulation for the next timing using the low-resolution analysis data as input (S20). This causes the processing flow shown in FIG. 5 to be repeated.

As described above, the information processing device 10 according to this embodiment is configured to acquire high-resolution analysis data by performing data assimilation and super-resolution on observation data in time and space and low-resolution prediction data in time and space. The information processing device 10 is also configured to perform processing in latent time and space when acquiring high-resolution analysis data. Therefore, it becomes possible to accurately and efficiently predict the state of the environment.

In addition, the information processing device 10 according to this embodiment is configured to calculate low-resolution analysis data. This makes it possible to continue performing low-resolution simulations. This makes it possible to perform simulations efficiently.

The nonlinear transformation unit 50 may also perform super-resolution in the time direction by transforming the data array of the data mapped to the latent time space. In this case, the high-resolution analysis data acquisition unit 60 may acquire high-resolution analysis data by performing super-resolution in the spatial direction independently for each time in the time direction on the data that has been super-resolved in the time direction in the latent time space. This allows for even more efficient processing. Details will be described later.

Furthermore, the structure transformation unit 30, the latent space-time mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be realized by a trained model trained by a machine learning algorithm. In this case, the structure transformation unit 30, the latent space-time mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be realized by a trained model trained by supervised learning using data with a higher spatiotemporal resolution than the predicted data as teacher data. Alternatively, the structure transformation unit 30, the latent space-time mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be realized by a trained model trained by unsupervised learning so as to reduce the loss function. This makes it possible to efficiently acquire high-resolution analysis data using observation data that is not lattice data and low-resolution predicted data. Details will be described later.

(Embodiment 1)
Hereinafter, the embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In addition, in each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

<Information processing device>
6 is a diagram showing a configuration of an information processing device 100 according to the first embodiment. The information processing device 100 is, for example, a computer. The information processing device 100 includes a learning processing unit 110, a simulation unit 120, an observation data acquisition unit 122, a prediction data acquisition unit 124, a structure conversion unit 130, a latent space-time mapping unit 140, a nonlinear conversion unit 150, a high-resolution analysis data acquisition unit 160, and a low-resolution analysis data calculation unit 170. These components can be realized by a hardware configuration described later.

The learning processing unit 110 performs machine learning such as neural networks on the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160. This allows the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 to realize their respective functions as trained models.

Here, when performing machine learning, the learning processing unit 110 may learn the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 together, rather than learning them separately. Specifically, the learning processing unit 110 learns the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a continuous manner using an end-to-end deep learning method. In other words, the learning processing unit 110 performs machine learning by regarding the functions of the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as layers of a neural network. In other words, the learning processing unit 110 performs machine learning by regarding the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as one neural network. Note that pre-learning may be performed separately in each of the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160.

The learning processing unit 110 may perform supervised learning or unsupervised learning. When performing supervised learning, the learning processing unit 110 may perform learning using, for example, highly accurate and high resolution time series weather data as teacher data. Details of the processing of the learning processing unit 110 will be described later. Note that the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high resolution analysis data acquisition unit 160 are not limited to being realized as a trained model trained by machine learning by the learning processing unit 110.

The simulation unit 120 corresponds to the simulation unit 20 described above. The simulation unit 120 performs a low-resolution simulation in the time direction and the space direction. The simulation unit 120 generates low-resolution prediction data (simulation data) using the input initial state.

Low-resolution prediction data (LR prediction simulation data) is digital data defined on lattice points in space-time. In other words, the prediction data is lattice data defined in space-time. In other words, the prediction data has a data array defined by a lattice structure. In other words, the prediction data, which is lattice data in space-time, has a data array for each lattice point at a predetermined interval in the spatial direction and the time direction. In the four-dimensional calculation mesh G2 shown in FIG. 2, the lattice points are arranged in the time axis direction, and in the three-dimensional calculation mesh G1 in the four-dimensional calculation mesh G2, the lattice points are arranged in the three spatial axes directions. The lattice data of the prediction data has a numerical value at each of these lattice points. The intervals between the lattice points may be equal or unequal. Note that since the simulation unit 120 performs a low-resolution simulation, the lattice data of the obtained prediction data has longer intervals in the time direction and the space direction on average compared to the high-resolution lattice data.

For example, it is assumed that the prediction data is defined by a data array A _hijk in a four-dimensional space-time. The subscript h corresponds to the time direction (t direction), the subscript i corresponds to the X-axis direction in the three-dimensional space, the subscript j corresponds to the Y-axis direction in the three-dimensional space, and the subscript k corresponds to the Z-axis direction in the three-dimensional space. In this case, the subscripts h, i, j, and k of the data array A _hijk take integer values. One of the sets of integer values (h, i, j, k) corresponds to one of the grid points of the four-dimensional calculation mesh G2. This makes it possible to uniquely specify the numerical value A _hijk on the grid point. In this way, the prediction data is defined by a data array A _hijk having integer subscripts for each dimensional direction in the four-dimensional space.

Here, in the data array A _hijk , it is assumed that there are H elements in the time direction. In other words, the subscript h takes H values. Similarly, it is assumed that there are I elements in the X-axis direction, that is, the subscript i takes I values. Furthermore, it is assumed that there are J elements in the Y-axis direction, that is, the subscript j takes J values. Furthermore, it is assumed that there are K elements in the Z-axis direction, that is, the subscript k takes K values. In this case, the number of elements in the data array A _hijk is H*I*J*K. If H=I=J=K=10, the number of elements in the data array A _hijk is ¹⁰⁴ .

The predicted data includes data for past times in the time series. The predicted data may also include data for future times in the time series. As described above, the predicted data is time data that includes the time of the observed data (reference time) described below.

The forecast data includes data indicating physical variables (physical quantities) necessary to predict the state of the atmosphere or other environment based on physical equations. In the case of a weather forecast simulation, the above-mentioned physical equations are, for example, the Navier-Stokes equations of fluids or thermodynamic equations. Furthermore, the physical quantities are, for example, air speed, pressure, temperature, water vapor mixing ratio, cloud particle number density, etc. For each of these physical quantities, the above-mentioned four-dimensional data array can be provided. It can be said that the data array represents the "field" of each physical quantity in physics.

It should also be noted that in this embodiment, the prediction data (LR simulation results) are obtained from a single scenario. In other words, the prediction data is a single (single) simulation result obtained when a simulation (physical simulation) is performed from a unique initial state. In contrast, in the data assimilation described in Non-Patent Document 1, simulations are performed for a variety of similar situations in order to perform ensemble calculations. In other words, in Non-Patent Document 1, a prediction simulation is performed using multiple scenarios.

The predicted data acquisition unit 124 corresponds to the above-mentioned predicted data acquisition unit 24. The predicted data acquisition unit 124 acquires the above-mentioned predicted data (prediction result of low-resolution physical simulation) from the simulation unit 120. The predicted data is expressed by the following formula (1).

... (1)

Here, x _t ^L is a vector field consisting of all physical quantities of the predicted data. That is, x is a vector field indicating a set of values (vectors) at each lattice point in three-dimensional space at a certain time t. And, formula (1) indicates a set of vector fields of all physical quantities from time t = 0 to time t = n. In other words, formula (1) indicates all data arrays (numerical arrays) of four-dimensional space-time related to the predicted data. Also, the subscript L indicates low resolution in the spatial direction. Also, t indicates the timestamp of the predicted data. t indicates a timestamp with a long time interval. That is, t indicates low resolution in the time direction (i.e., a long time interval). That is, formula (1) represents data with a small number of lattices (small number of elements; low resolution) in four-dimensional space-time.

The observation data acquisition unit 122 corresponds to the above-mentioned observation data acquisition unit 22. The observation data acquisition unit 122 acquires observation data from sensors and cameras, drones, smartphones, etc. arranged in the environment, similar to the observation data acquisition unit 22. The observation data is expressed by the following formula (2).

... (2)

Equation (2) represents a set of digital data consisting of a numerical value (observation value) o that indicates a certain state at time τ. Here, τ represents the timestamp of the observation data. The time interval of τ does not have to be equal. Furthermore, o may or may not represent a physical quantity. In other words, o may represent the numerical value of a non-physical quantity.

Note that the observation data may be lattice data. Alternatively, the observation data does not have to be lattice data. In other words, the observation data may be unstructured data (non-lattice data). In unstructured observation data, observation values are associated with time information and spatial information, but there is no regularity in the intervals in time and space. In other words, unstructured observation data does not have a spatial mesh structure. Therefore, the observation data may indicate observation values in random time and space.

The observation data may include various miscellaneous data of different qualities. The observation data may be image data, sound data, point data, or log data. The observation data may be a set of values of physical quantities representing the state of the atmosphere, or digital data from which these physical quantities can be estimated. For example, the observation data may indicate AMeDAS observation values. In this case, the observation values may indicate the temperature, humidity, wind speed, etc. of various locations. The observation data may also indicate the radiance of an object (such as a building). This allows the temperature at a position near the object and at the time of observation to be estimated. The observation data may also indicate an acceleration log of an aircraft floating in the air, such as a drone. This allows the wind speed or turbulence dissipation rate at the position of the aircraft and at the time of observation to be estimated. The observation data may also indicate an image of the sky. This allows the cloud cover or precipitation at the position and time of the image to be estimated. The observation data may also indicate the sales of cold desserts (ice cream, sorbet, popsicle, shaved ice, etc.) at a certain location (such as a convenience store). This allows the local temperature of the area to be estimated. In other words, it can be assumed that the higher the sales of frozen desserts, the higher the temperature in the area.

Furthermore, when a simulation is performed with sufficient accuracy, the simulation results can be said to represent the real state well. Therefore, the results of a simulation with sufficient accuracy can be considered as observational results that represent reality well, and can be considered as observational data. Depending on the settings, it is possible to perform "experiments close to reality" that incorporate the shape of a building and even the exhaust heat emitted by the building, such as a micrometeorological simulation. Such simulations have sufficiently small errors that they can be considered as observational results that represent reality well, that is, as observational data.

Furthermore, the spatial and temporal resolution at which the observational data is observed may be high resolution or low resolution. When the observational data represents satellite data or radar data, it is obtained by line (one-dimensional) observation or surface (two-dimensional) observation. When the observational data is data obtained by a Doppler lidar (LiDAR: Light Detection And Ranging), it is obtained by three-dimensional observation. In these cases, the spatial resolution (and the time interval of the observation) can determine whether the resolution of the observational data is low resolution or high resolution.

Also, when the observation data corresponds to AMeDAS data, it is obtained by point (zero-dimensional) observation. Even in this case, the resolution can be defined as follows. That is, even in the case of point observation, there is a representative scale that indicates the scale of the state that the observation value represents. When the representative scale is large, the observation data can be considered to be low-resolution observation data. When the representative scale is small, the observation data can be considered to be high-resolution observation data. For example, in a normal meteorological model, observation values with a coarse spatial resolution that represents 1 km to 10 km horizontally are observed. Therefore, strict conditions are imposed on the observation, such as being in a place with no obstructions nearby, on grass, not exposed to direct sunlight, and not being affected by artificial exhaust heat. Such observation data can be considered to be low-resolution observation data. On the other hand, in a micrometeorological model, observation values with a fine spatial resolution that represents 1 m to 5 m horizontally, which is affected by artificial exhaust heat, can be observed. In a micrometeorological model, simulations can be performed taking into account the impact of such observation data on the atmosphere. Such observation data can be considered to be high-resolution observation data.

Note that high-resolution observation data and low-resolution prediction data have significantly different spatial and temporal resolutions. Therefore, when performing data assimilation, it is necessary to average the high-resolution observation data in the spatiotemporal direction to match the low-resolution prediction data. Therefore, it should be noted that it is usually difficult to directly perform data assimilation on high-resolution observation data and low-resolution prediction data. In contrast, in this embodiment, data assimilation can be performed on any observation data and low-resolution prediction data.

The structure conversion unit 130 corresponds to the structure conversion unit 30 described above. The structure conversion unit 130 has a function as a structurizer. The structure conversion unit 130 may be realized by an existing structurizer. The structure conversion unit 130 converts the observation data, which is unstructured data (non-lattice data), into observation data with a lattice data structure. In other words, the structure conversion unit 130 converts the observation data into grid data. The structure conversion unit 130 converts the observation data into data on each lattice point of the lattice data in high-resolution space-time. In other words, the structure conversion unit 130 converts the observation value into a physical quantity defined on a lattice. Note that the lattice data obtained by converting the observation data may correspond to lattice data having a larger number of elements (i.e., high resolution) than the lattice data of the prediction data.

If a function representing the structurizer realized by the structure conversion unit 130 is s(), the function of the structure conversion unit 130 is expressed by the following formula (3). The left side of formula (3) corresponds to the output data of the structure conversion unit 130. Note that the observation data input to the structure conversion unit 130 (structurizer) may be lattice data. Furthermore, the observation data input to the structure conversion unit 130 may indicate a physical quantity or a numerical value other than a physical quantity.

...(3)

Here, o _T ^H is a vector field consisting of all the observation values of the observation data. That is, o is a vector field indicating a set (vector) of values at each lattice point in three-dimensional space at a certain time T. The left side of formula (3) indicates a set of vector fields of all the observation values (physical quantities, etc.) from time T = 0 to time T = N. In other words, the left side of formula (3) indicates all the data arrays (numerical arrays) in four-dimensional space-time corresponding to the observation data (physical quantities, etc.). The subscript H indicates high resolution in the spatial direction. Furthermore, T indicates a timestamp of the observation data. T indicates a timestamp with a short time interval. In other words, T indicates high resolution in the time direction (i.e., a short time interval). In other words, the left side of formula (3) indicates that the data has a large number of lattices (a large number of elements; high resolution) in four-dimensional space-time.

The structurizer of the structure conversion unit 130 may be realized by a trained model trained by machine learning, such as a neural network. In this case, the structure conversion unit 130 may be realized by a linear projection operator, a fully connected layer, or a graph convolution network (the same applies to the latent space-time mapping unit 140, the nonlinear conversion unit 150, and the high-resolution analysis data acquisition unit 160 described later). In this case, the structure conversion unit 130 is trained by the learning processing unit 110. For example, the structurizer may be realized by the technology shown in Non-Patent Document 2. As described above, the structure conversion unit 130, the latent space-time mapping unit 140, the nonlinear conversion unit 150, and the high-resolution analysis data acquisition unit 160 are trained in a continuous manner by the end-to-end deep learning method. Then, the structure conversion unit 130 (structurizer) is trained to convert the observation data into appropriate grid data for performing the super-resolution data assimilation according to this embodiment.

Furthermore, the structurizer does not have to be realized by a trained model trained by machine learning. In this case, the structure conversion unit 130 may read the time and position information of the observation data, and project the observation data onto the lattice point that is closest to the read time and position information among the lattice points arranged at a predetermined interval in space-time. In this case, the structure conversion unit 130 may also substitute missing values for the lattice points onto which the observation data is not projected.

Note that even if the observation data input to the structure conversion unit 130 is data indicating a non-physical quantity, the structure conversion unit 130 may output lattice data indicating the numerical values of physical quantities, etc. corresponding to the observation data. For example, if the observation data input to the structure conversion unit 130 is data indicating the sales of frozen desserts at each point at each time, the structure conversion unit 130 may output lattice data indicating the temperature at each point at each time. Furthermore, the structure conversion unit 130 may input a variety of multiple observation data. In this case, the structure conversion unit 130 may output lattice data indicating the numerical values of physical quantities, etc. corresponding to the multiple observation data. For example, if the observation data input to the structure conversion unit 130 is the acceleration of a drone and the sales of frozen desserts at each point at each time, the structure conversion unit 130 may output lattice data indicating the wind speed and temperature at each point at each time. Furthermore, the structure conversion unit 130 is not limited to outputting lattice data indicating physical quantities that can be understood by humans, corresponding to the input observation data. The structure conversion unit 130 may output lattice data corresponding to a numerical array that cannot be understood by humans (i.e., can only be understood by a neural network). In other words, the structure conversion unit 130 outputs lattice data (numerical array) indicating numerical values defined on lattice points arranged at a predetermined interval in space-time. Furthermore, when the observation data is image data (camera data), the structure conversion unit 130 (or another component) may perform processing using object recognition or pixel segmentation as preprocessing. Furthermore, for this preprocessing, a pre-trained neural network or the like may be used. Furthermore, the structure conversion unit 130 may perform the above-mentioned structural conversion processing on the data that has been subjected to such preprocessing.

As described above, the structural transformation unit 130 receives as input observation data, which is non-lattice data or lattice data and indicates the observed values of physical or non-physical quantities. The structural transformation unit 130 then outputs lattice data of a four-dimensional space-time data array (numerical array) relating to physical quantities (temperature, wind speed, etc.) that indicate the state of the environment (e.g., atmospheric conditions). With this configuration, the structural transformation unit 130 can perform non-linear transformation of observation data of non-physical quantities or non-lattice data that cannot be handled by existing data assimilation methods, and convert it into observation data of a numerical array that can be assimilated.

The latent space-time mapping unit 140 corresponds to the latent space-time mapping unit 40 described above. The latent space-time mapping unit 140 has a function as an encoder. The latent space-time mapping unit 140 may be realized by an existing encoder. The latent space-time mapping unit 140 maps the observed data and predicted data in the first real space-time to latent space-time. For each time, the latent space-time mapping unit 140 converts the observed data and predicted data structured in four-dimensional space-time into data in latent space-time. In other words, in the latent space-time mapping unit 140, the output for each time is calculated independently for the input for each time.

If the functions representing the encoder realized by the latent space-time mapping unit 140 are e _x () and e _o (), the function of the latent space-time mapping unit 140 is expressed by the following formulas (4) and (5). Here, e _x () is a function for mapping predicted data x to latent space-time. _{e o} () is a function for mapping observed data o to latent space-time. As a result, at each time t, observed data o and predicted data x are each mapped to latent space-time.

...(4)

...(5)

As a result, the latent space-time mapping unit 140 obtains data of the latent space-time represented by the following formula (6): Formula (6) represents a set of data of the latent space-time at each time from time t=0 to t=n.

...(6)

Here, formula (4) indicates that the predicted data x _t ^L is mapped to the latent space-time using the observation data _{o t} converted to lattice data, thereby obtaining the mapping data p _t . The mapping data p _t indicates a numerical array in the latent space-time corresponding to the low-resolution predicted data. Here, the latent space-time mapping unit 140 may perform pre-processing and main processing on the observation data. For example, as the pre-processing, the latent space-time mapping unit 140 may convert the input observation data o _T ^H into observation data o _t having a structure that matches the lattice structure in the space-time of the predicted data. o _t is obtained by coarsening the time or space step of the observation data o _T converted to lattice data to match the lattice structure in the space-time of the low-resolution predicted data. Then, for example, as the main processing, the latent space-time mapping unit 140 may obtain the mapping data p _t corresponding to the predicted data by simultaneous data assimilation that reflects the converted o _t in the low-resolution predicted data.

Furthermore, equation (5) indicates that mapping data _qt can be obtained by mapping the observation data _ot converted into lattice data to latent space-time. The mapping data _qt indicates a numerical array in latent space-time corresponding to the observation data. When mapping the observation data, the latent space-time mapping unit 40 maps the observation data alone to latent space-time.

As can be seen from the subscript t in equations (4) to (6), in the processing of the latent space-time mapping unit 140, the resolution in the time direction (time step t) inside the latent space-time mapping unit 140 is matched to the resolution of the predicted data. On the other hand, the resolution in the space direction is smaller than the resolution of the predicted data. In other words, as described above, the number of elements in the numerical array of the mapping data in latent space-time is smaller than the number of elements in the numerical array of the predicted data. Therefore, the latent space-time mapping unit 140 reduces the number of elements in the numerical arrays of the observed data and predicted data (dimensional compression in topological space).

Furthermore, equations (4) to (6) show that the observed data and the predicted data are fused in latent space-time. In other words, equation (6) shows data (fused data) in which the observed data and the predicted data are fused in latent space-time. Therefore, it can be said that data assimilation is performed on the observed data and the predicted data by the latent space-time mapping unit 140.

The latent space-time mapping unit 140 (encoder) may be realized by a trained model trained by machine learning, such as a neural network. For example, the latent space-time mapping unit 140 may use convolution and pooling to perform nonlinear transformation while reducing the number of elements. The neural network may also be a neural network that reflects physical symmetry. A convolutional neural network may be adopted to reflect spatial translational symmetry. A group convolutional neural network may be adopted to reflect spatial rotational symmetry. A vision transformer or a graph convolutional neural network may be adopted to reflect relabeling symmetry. This makes it possible to perform transformation that takes physical symmetry into account, unlike existing data assimilation methods. The same applies to the decoder of the high-resolution analysis data acquisition unit 160, which will be described later.

As described above, the latent space-time mapping unit 140 receives as input low-resolution predicted data, which is a set of data arrays in four-dimensional space-time, and observed data converted into lattice data. The latent space-time mapping unit 140 then outputs mapping data in latent space-time with a small number of elements. With this configuration, the latent space-time mapping unit 140 can improve the processing efficiency of the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160. In other words, since the number of elements in the numerical array is reduced, the amount of data to be processed is reduced. This reduces the amount of processing required for computational resources.

The nonlinear conversion unit 150 corresponds to the nonlinear conversion unit 50 described above. The nonlinear conversion unit 150 has a function as a time series converter. The nonlinear conversion unit 150 may be realized by an existing time series converter. The nonlinear conversion unit 150 performs nonlinear conversion on the observation data and prediction data that have been mapped in the latent time space. More specifically, the nonlinear conversion unit 150 may perform nonlinear conversion on the time series to generate data in which the observation data and the prediction data mapped to the latent time space are fused. Then, the nonlinear conversion unit 150 according to the first embodiment may perform super-resolution in the time direction on the fused data. Here, as described above, the number of elements is reduced in the latent time space compared to the real time space. Therefore, the amount of data to be processed is suppressed, and super-resolution can be efficiently performed in the time direction.

If the function representing the time series converter realized by the nonlinear conversion unit 150 is F(), the function of the nonlinear conversion unit 150 is expressed by the following formula (7). The left side of formula (7) corresponds to the output data of the nonlinear conversion unit 150. As described above, the subscript T indicates high resolution in the time direction. In addition, r _T indicates data (fusion data) in which the observation data and the prediction data mapped to the latent space-time at time T are fused by data assimilation. The left side of formula (7) indicates a set of fusion data from time T = 0 to time T = N. In other words, the left side of formula (7) indicates a data array (numerical array) in a four-dimensional space-time corresponding to the fusion data.

...(7)

As described above, equation (7) indicates that the observed data and the predicted data are fused in the latent space-time. In other words, the left side of equation (7) indicates data (fused data) in which the observed data and the predicted data are fused in the latent space-time. Therefore, it can be said that data assimilation is performed on the observed data and the predicted data by the nonlinear transformation unit 150.

The time series converter of the nonlinear conversion unit 150 may be realized by a trained model trained by machine learning, such as a neural network. In this case, the nonlinear conversion unit 150 may be realized by a neural network that uses an attention mechanism. In addition, the time series converter of the nonlinear conversion unit 150 may perform nonlinear conversion, for example, using a technology called Transformer described in Non-Patent Document 3.

The nonlinear transformation unit 150 may also implicitly calculate the prediction error using information in the time-space direction. That is, the nonlinear transformation unit 150 may calculate the error inside the neural network from the time change of the spatial pattern. In other words, the nonlinear transformation unit 150 may calculate the prediction error by matching the time-space pattern and determining the magnitude of the error relative to the ground truth data. The nonlinear transformation unit 150 may also be realized by a neural network in which data assimilation is performed between the predicted data and the observed data using the prediction error as a weight. In this way, data assimilation can be performed between the predicted data and the observed data through efficient calculations that do not require an ensemble.

Furthermore, the time series transformer of the nonlinear transformer 150 may repeatedly execute a linear transform (e.g., an affine transform) and a nonlinear transform using ReLU (Rectified Linear Unit) or the like. In the process of executing this process, the nonlinear transformer 150 may transform the data array so as to make the time step finer, for example, using the technique of Non-Patent Document 4. The nonlinear transformer 150 may divide a plurality of elements in the latent space-time into two, elements in the time direction and elements in the space direction, and increase the number of elements in the time direction. For example, assume that there are H' elements in the time direction, I' elements in the X-axis direction, J' elements in the Y-axis direction, and K' elements in the Z-axis direction. In this case, the nonlinear transformer 150 transforms the four-dimensional array H'×I'×J'×K' into a two-dimensional array H'×M. Note that M is the number of elements in the space direction, and M=I'×J'×K'. The nonlinear transformer 150 may then transform this array into an array of 2H'×M/2. This doubles the number of elements in the time direction, and halves the time step. Therefore, super-resolution is performed in the time direction on the data mapped to the latent time space. In this way, the nonlinear transformation unit 150 may perform super-resolution in the time direction by transforming the data array on the data mapped to the latent time space.

As described above, the nonlinear transformation unit 150 receives as input a set of numerical arrays (data arrays) corresponding to observed data and predicted data in latent time space. The nonlinear transformation unit 150 may then output time series data in the latent time space that has been super-resolved in the time direction. The time-direction resolution of this time series data in the latent time space that has been super-resolved in the time direction is higher than the time-direction resolution of the predicted data. As a result, the nonlinear transformation unit 150 performs super-resolution in the time direction in a latent time space with a small number of elements (i.e., low-dimensional in topological space), thereby reducing the amount of calculations and therefore the processing costs. This makes it possible to improve computational efficiency.

Furthermore, super-resolution in the time direction is performed by the time series transformer of the nonlinear transformation unit 150. As described above, since the number of elements in the numerical array is small in the latent time space, the amount of memory required is reduced, and long time series can be handled. This makes it possible to efficiently perform super-resolution in the time direction. Furthermore, the processing of the nonlinear transformation unit 150 makes it possible to perform data assimilation of observed data and predicted data while performing super-resolution in the time direction. Therefore, it becomes possible to efficiently perform data assimilation and super-resolution in the time direction.

The high-resolution analysis data acquisition unit 160 corresponds to the high-resolution analysis data acquisition unit 60 described above. The high-resolution analysis data acquisition unit 160 has a function as a decoder. The high-resolution analysis data acquisition unit 160 may be realized by an existing decoder. The high-resolution analysis data acquisition unit 160 maps the observation data and prediction data that have been subjected to nonlinear transformation from the latent time space to the second real time space. The high-resolution analysis data acquisition unit 160 maps the data obtained by the processing of the nonlinear transformation unit 150 for each time from the latent time space to the second time space. As a result, the high-resolution analysis data acquisition unit 160 performs super-resolution in the spatial direction for the data obtained by the processing of the nonlinear transformation unit 150. As a result, the high-resolution analysis data acquisition unit 160 converts the data in the latent time space into high-resolution analysis data for each time. In other words, in the high-resolution analysis data acquisition unit 160, the output for each time is calculated independently for the input for each time.

In this way, the high-resolution analysis data acquisition unit 160 acquires the high-resolution analysis data Da1. As described above, the high-resolution analysis data Da1 is data with a higher resolution in time and space than the predicted data. Furthermore, the high-resolution analysis data Da1 may be analysis data in a time (time series) that includes the past and future of the time of the observed data, depending on the time range of the input predicted data. Therefore, the high-resolution analysis data Da1 can be analysis data that has been extrapolated in the time direction (time extrapolation) with respect to the time of the observed data (reference time).

If a function representing the decoder realized by the high-resolution analysis data acquisition unit 160 is d(), the function of the high-resolution analysis data acquisition unit 160 is expressed by the following formula (8). d() is a function for mapping fusion data _rT relating to time T from latent space-time to the second real space-time. As a result, at each time T, fusion data in which observation data and predicted data are fused by data assimilation is mapped to the second real space-time with high resolution. As a result, super-resolution in the spatial direction is performed on the fusion data that has been super-resolved in the time direction by the nonlinear transformation unit 150.

...(8)

As a result, the high-resolution analysis data acquisition unit 160 obtains data of the second real time space represented by the following formula (9). Formula (9) shows a set of high-resolution analysis data y at each time from time T=0 to T=N. Formula (9) corresponds to high-resolution analysis data Da1, which is output data from the high-resolution analysis data acquisition unit 160. The high-resolution analysis data shown in formula (9) is time-series data on a high-resolution grid. The high-resolution analysis data Da1 may include past, present, and future data.

... (9)

Here, y _T ^H is a vector field consisting of all physical quantities of the high-resolution analysis data. That is, y is a vector field indicating a set of values (vectors) at each lattice point in three-dimensional space at a certain time T. And, formula (9) indicates a set of vector fields of all physical quantities from time T=0 to time T=N. In other words, formula (9) indicates all data arrays (numerical arrays) in four-dimensional space-time related to the high-resolution analysis data. Also, the subscript H indicates high resolution in the spatial direction. Also, T indicates a timestamp of the high-resolution analysis data. T indicates a timestamp with a short time interval. That is, T indicates high resolution in the time direction (that is, a short time interval). That is, formula (9) indicates that the data has a large number of lattices (a large number of elements; high resolution) in four-dimensional space-time.

The decoder of the high-resolution analysis data acquisition unit 160 may be realized by a trained model trained by machine learning, such as a neural network. For example, the neural network related to the high-resolution analysis data acquisition unit 160 may be a neural net that reflects physical symmetry. This makes it possible to perform conversion that takes physical symmetry into account, unlike existing data assimilation methods. The high-resolution analysis data acquisition unit 160 may also generate high-resolution analysis data that has been extrapolated (time extrapolated) in the time direction relative to the time of the observation data (reference time) using a neural network. In other words, the neural network of the high-resolution analysis data acquisition unit 160 can be trained to output high-resolution analysis data that has been time extrapolated relative to the reference time, using time series data that has a higher resolution and accuracy in space-time than the predicted data as teacher data.

The decoder of the high-resolution analysis data acquisition unit 160 may also repeatedly execute linear transformation and nonlinear transformation (e.g., ReLU) to increase the spatial resolution. In the process of executing this process, the high-resolution analysis data acquisition unit 160 may transform the data array (numerical array) so as to increase the spatial resolution, for example, using a technique called Pixel Shuffle, which is shown in Non-Patent Document 5. In this case, for example, the fusion data is a numerical array in which the number of elements in the time direction is n and the number of elements in the spatial direction is m. In this case, the high-resolution analysis data acquisition unit 160 may transform the n×m array into an n/2×2m array. This doubles the number of elements in the spatial direction (m→2m), so that the spatial resolution doubles.

As described above, the high-resolution analysis data acquisition unit 160 receives as input a numerical array (data array) that is time-series data in latent time-space corresponding to the fusion data. The high-resolution analysis data acquisition unit 160 then outputs a four-dimensional numerical array that is time-series data on a high-resolution grid in the second real time-space with a large number of elements. With this configuration, the high-resolution analysis data acquisition unit 160 can efficiently perform super-resolution in the spatial direction in real time-space on the fusion data that has been subjected to data assimilation and super-resolution in the time direction.

In other words, the high-resolution analysis data acquisition unit 160 processes the numerical array (fusion data) in the latent space-time at each time independently. As a result, in the processing in the high-resolution analysis data acquisition unit 160, super-resolution in the spatial direction can be performed without referring to information in the time direction. Therefore, in the processing in the high-resolution analysis data acquisition unit 160, the necessary computational resources (memory amount, etc.) are saved, and it is possible to efficiently perform super-resolution in three-dimensional space. Therefore, it is possible to efficiently obtain high-resolution analysis data that has been super-resolved in the time direction and the spatial direction. In addition, when the prediction data is prediction data at the time of the observation data and a time including the past and future of the said time, the high-resolution analysis data acquisition unit 160 can acquire high-resolution analysis data at a time including the past and future of the time of the observation data. As a result, it is possible to provide high-resolution analysis data and future prediction information in which time extrapolation has been performed at the same time. Therefore, it is possible to provide services with higher added value.

Furthermore, by obtaining high-resolution analysis data with super-resolution in the time and space directions, it is possible to supplement low-resolution prediction simulations as follows. In other words, in practice, the resolution of prediction simulations such as weather and oceanographic predictions is often insufficient in the time and space directions. For example, in aquaculture, when the water temperature rises, fish become more active, so if the fish are fed too much, they may die due to lack of oxygen. Therefore, aquaculture farmers may want to know the timing of water temperature rise in order to adjust the amount of feed. However, in the case of current ocean forecasts, the resolution in the plane direction is relatively low, about 10 km. A prediction simulation with this level of resolution may not be able to accurately predict the water temperature near the fish farmer's fish farm. In order to accurately predict the water temperature in each fish farmer's fish farm, a fine resolution (high resolution) of at least 1 km, and preferably about 100 m, is required. In contrast, this embodiment can obtain high-resolution analysis data with super-resolution in both the time and spatial directions, making it possible to provide accurate information to businesses that want pinpoint weather and ocean condition forecasts.

The low-resolution analysis data calculation unit 170 corresponds to the low-resolution analysis data calculation unit 70 described above. The low-resolution analysis data calculation unit 170 calculates low-resolution analysis data Da2 using the high-resolution analysis data Da1. The resolution of the low-resolution analysis data Da2 may correspond to the resolution of the prediction data. The low-resolution analysis data Da2 is lattice data having a data array with a lower resolution than the high-resolution analysis data Da1. The low-resolution analysis data Da2 may also be snapshot data at a certain time.

Specifically, the low-resolution analysis data calculation unit 170 uses a predetermined function f(y) and inputs the above-mentioned high-resolution analysis data y to the function f to calculate the low-resolution analysis data Da2. The function f may be, for example, a function representing algebraic interpolation. The function f may be a function that performs an algebraic interpolation operation such as linear interpolation. The function f may also be a function defined by the linear interpolation method, the bicubic method, or the Lanczos method. For example, the low-resolution analysis data calculation unit 170 can calculate the low-resolution analysis data Da2 from the high-resolution analysis data Da1 by resizing (reducing the resolution) the high-resolution analysis data. Specifically, the low-resolution analysis data calculation unit 170 locally interpolates values corresponding to each lattice point of the lattice data using a polynomial to enlarge or reduce the lattice data. In this way, the low-resolution analysis data Da2 is calculated.

As described above, the low-resolution analysis data Da2 is input to the simulation unit 120 and used to perform a predictive simulation of the next timing. The low-resolution analysis data Da2 may also be used to numerically solve physical equations to obtain future predictions. This allows the time evolution of the state of the predictive simulation to be performed. In this case, the low-resolution analysis data Da2 may indicate the state at the current time.

The information processing device 100 according to the first embodiment is configured to perform data assimilation and super-resolution in a latent space-time with a reduced number of elements, as described above. This allows the number of elements to be handled in the calculation, i.e., the amount of data, to be reduced, and data assimilation and super-resolution can be performed efficiently. Therefore, it is possible to efficiently assimilate observation data that indicates non-physical quantities and observation data that is irregular in the time and space directions to the prediction data. Also, as described above, when the learning processing unit 110 learns the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a continuous manner using an end-to-end learning method, a large amount of computational resources is required. Therefore, by performing processing in the latent space-time as described above, the amount of data to be processed can be reduced, and such learning can be performed efficiently.

In addition, the information processing device 100 according to the first embodiment is configured to perform super-resolution in the time direction in the nonlinear conversion unit 150 and super-resolution in the space direction in the high-resolution analysis data acquisition unit 160. This makes it possible to efficiently perform super-resolution in the time direction and the space direction. That is, in order to simultaneously perform super-resolution in both the time direction and the space direction, a large amount of memory and calculation time are required. In particular, in the learning stage, in order to back-propagate the error, it is necessary to store the calculation graph and the gradient value. Therefore, a huge amount of memory and calculation time are required. In contrast, in the information processing device 100 according to the first embodiment, the nonlinear conversion unit 150 efficiently refers to the dimension in the time direction in order to perform super-resolution in the time direction on a latent time space with a small number of elements. On the other hand, the high-resolution analysis data acquisition unit 160 processes snapshots at each time, so there is no need to consider the dimension in the time direction. Furthermore, since snapshots at each time are processed, these snapshots can be processed simultaneously. Therefore, it is possible to suppress the required calculation resources.

<Comparison with Comparative Examples>
Next, a comparison example according to Non-Patent Document 1 will be compared with the technique according to the first embodiment.
As described above, the technique of Non-Patent Document 1 simply combines a super-resolution technique and a data assimilation technique. In other words, in Non-Patent Document 1, super-resolution and data assimilation are performed independently.

FIG. 7 is a diagram for explaining a technique according to a comparative example. FIG. 7 explains a technique according to Non-Patent Document 1 as a comparative example. In the comparative example, as shown by the white circle dots, physical simulations are performed for various situations by ensemble calculation. Then, as shown by the black circle dots, it is assumed that there is observation data at time t1. In this case, as shown by the white triangular dots, super-resolution is performed using the physical simulation results at that time t1, and high-resolution prediction is performed. This super-resolution is performed independently at time t1 for the ensemble calculation results, i.e., for each situation. Then, at that time t1, data assimilation is performed on the high-resolution ensemble prediction results and the observation data, and high-resolution prediction data in which the observation data has been assimilated is obtained as the final output, as shown by the black triangular dots. In this way, the comparative example uses ensemble calculation. Also, in the comparative example, super-resolution at a certain moment is performed independently of data assimilation at the same moment. Then, data assimilation is performed in a high-resolution space.

FIG. 8 is a diagram for explaining super-resolution and data assimilation according to the first embodiment. In the first embodiment, a single scenario is used. That is, a physical simulation is performed from a unique initial state, and a unique simulation result is obtained in the time series, as shown by the white dots. In FIG. 8, physical simulation results (predicted data) are obtained, which are time series data, corresponding to times t1, t2, t3, and t4. Then, instead of performing super-resolution at a certain time, observation data shown by black dots and predicted data, which is time series data, are input, and super-resolution and data assimilation are performed simultaneously in a latent time space with a reduced number of elements. As a result, high-resolution data, which is time series data, is obtained, as shown by black triangular dots. Also, as shown in FIG. 8, the high-resolution data has a higher resolution in the time direction than the physical simulation. Thus, in the first embodiment, unlike the comparative example, ensemble calculation is not performed in the simulation. Also, in the first embodiment, unlike the comparative example, super-resolution and data assimilation are performed simultaneously in the latent time space. Also, in the first embodiment, unlike the comparative example, super-resolution and data assimilation are performed using time series information.

Figure 9 is a diagram comparing the experimental results of the first embodiment with those of the comparative example. Figure 9 is a diagram showing the time series of the mean absolute error (MAE) of the vorticity field ω. Graph A shows the case where neither data assimilation nor super-resolution is performed. Graph B shows the case of the comparative example. Graph C shows the case of the first embodiment. As shown in Figure 9, in the case of the first embodiment, the error is minimized. Therefore, the technology of the first embodiment has made it possible to achieve highly accurate predictions.

Furthermore, in the experimental results for the comparative example, the calculation time per experiment was 320 seconds. In contrast, in the experimental results for the first embodiment, the calculation time per experiment was 61 seconds. In this way, the technology for the first embodiment has achieved a significant reduction in calculation time compared to the comparative example.

<Learning Method>
Next, a learning method by the learning processing unit 110 of the information processing device 100 according to the first embodiment will be described. As described above, the learning processing unit 110 learns the structure conversion unit 130, the latent space-time mapping unit 140, the nonlinear conversion unit 150, and the high-resolution analysis data acquisition unit 160 in a continuous manner by the end-to-end learning method. In other words, the learning processing unit 110 may learn the structure conversion unit 130, the latent space-time mapping unit 140, the nonlinear conversion unit 150, and the high-resolution analysis data acquisition unit 160 as one neural network having these as layers. In other words, the learning processing unit 110 learns the structure conversion unit 130, the latent space-time mapping unit 140, the nonlinear conversion unit 150, and the high-resolution analysis data acquisition unit 160 collectively so that the observation data and the prediction data are input and appropriate high-resolution analysis data is output.

First, the first learning method will be described. The first learning method is supervised learning. The teacher data (correct answer data) is, for example, highly accurate and high-resolution data. In the case of a system that performs weather forecasting, the teacher data is, for example, highly accurate and high-resolution meteorological data. In other words, the teacher data is highly accurate and high-resolution time series data of physical variables of the atmosphere. The teacher data is, for example, time series data (four-dimensional numerical array) of a velocity field, a temperature field, a density field, etc. Also, the teacher data may be a result of a micrometeorological simulation with ultra-high resolution. The learning processing unit 110 receives the above-mentioned predicted data and observed data as input, and updates the parameters (weights, etc.) of the neural network in the gradient direction of the error between the teacher data and the final output (high-resolution analysis data) by the error backpropagation method using the above-mentioned teacher data. The learning processing unit 110 repeats such processing to learn the neural network that constitutes the structure conversion unit 130, the latent space-time mapping unit 140, the nonlinear conversion unit 150, and the high-resolution analysis data acquisition unit 160.

Next, the second learning method will be described. The second learning method is unsupervised learning. In the above-mentioned supervised learning, for example, highly accurate and high-resolution time series data of atmospheric physical variables is required as the supervised data. However, it may be difficult to obtain such data. In contrast, in unsupervised learning, such supervised data is not required. Unsupervised learning may be performed, for example, by the variational Bayes method. Also, unsupervised learning may be performed by adversarial learning. Below, the learning method using the variational Bayes method will be described.

The variational Bayes method is a type of approximation method that approximates the true probability distribution p with a simpler probability distribution q. The parameters of q are then estimated by minimizing the KL divergence or similar. The variational Bayes method is a probabilistic model that treats the true physical variables of the state of the environment, such as the atmosphere, as the hidden state, and gives the observed value or low-resolution predicted value of the input based on this hidden state.

As one example of the implementation of the variational Bayes method, a lower bound of the log likelihood ln(p(o|x)) is introduced and this lower bound is maximized. The log likelihood ln(p(o|x)) can be transformed into the following formula (10) using Jensen's inequality. Here, o corresponds to the observed data and x corresponds to the low-resolution predicted data. In the middle of transforming formula (10), a hidden variable y is introduced. This hidden variable corresponds to the high-resolution analysis data Da1 shown in formula (9). As a result, a loss function that enables the hidden variable y to be estimated from the observed data o and the low-resolution prediction x can be derived.

...(10)

In formula (10), the observed value o corresponds to low-resolution or high-resolution observed data. o corresponds to the above formula (2) (or the left side of formula (3)). The hidden variable y corresponds to the high-resolution analysis data Da1. The observed value o may be highly accurate predicted data. The variable x corresponds to the input low-resolution predicted data (formula (1)). This probability model is composed of a recognition model and a generation model. The recognition model (the part that calculates the hidden variables) corresponds to the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 (encoder-decoder model) in the information processing device 100. The generation model will be described later.

The fourth equation (the rightmost side; the final form of the transformation) in equation (10) corresponds to the variational lower bound (VLB), which is the lower bound of the log-likelihood. The first term E _q [p(o|x, y)] of the variational lower bound indicates the reconstruction error and corresponds to the log-likelihood of the observed data o. The second term KL(q(y|x, o)|p(y|x)) of the variational lower bound indicates the KL divergence. The KL divergence is an index corresponding to the distance between distribution q and distribution p.

The learning processing unit 110 performs learning by updating the model parameters (neural network parameters) using the backpropagation method and the gradient descent method so as to maximize (increase) the variational lower bound shown in the fourth equation of equation (10). This variational lower bound corresponds to the training error. Maximizing the variational lower bound corresponds to minimizing the loss function in machine learning. In other words, increasing the variational lower bound corresponds to decreasing the loss function in machine learning. At this time, learning progresses so that the reconstruction error and the KL divergence are balanced. This balancing of the reconstruction error and the KL divergence corresponds to the fusion of observed data and predicted data in data assimilation.

In addition, it has been newly discovered that maximizing the variational lower bound has the aspect of generalizing minimum variance estimation. In other words, by averaging the observed values o and the predicted data x with their respective errors as weights (performing a weighted average), it is possible to estimate high-resolution analysis data y. In other words, since observed values are generally more accurate than predicted data, the analysis data can be estimated by weighting the observed values greater than the weighting of the predicted data and averaging them.

10 is a diagram for explaining a method of learning the components according to the first embodiment using the variational Bayes method. When learning using the variational Bayes method, the learning processing unit 110 has a sampler 112 and an observation data generating unit 114. The sampler 112 and the observation data generating unit 114 can be regarded as a generation model that generates an observation value (pseudo observation data Da3). This generation model can calculate the reconstruction error E _q [p(o|x, y)]. Specifically, the observation data generating unit 114 calculates p(o|x, y). p(o|x, y) indicates the distribution of the pseudo observation data o when the prediction data x and the high-resolution analysis data y are determined.

On the other hand, as described above, the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 can be regarded as a recognition model. This recognition model can calculate the KL divergence. Specifically, the recognition model calculates q(y|x, o) in the KL divergence. q(y|x, o) indicates the distribution of the high-resolution analysis data y when the predicted data x and the observed data o are input. Note that p(y|x) is a prior distribution, and can be obtained by appropriate assumptions or by prior learning.

The sampler 112 samples the probability distribution corresponding to the high-resolution analysis data generated by learning. As a result, the sampler 112 replaces the neural network with a probability distribution (probability model). That is, since a neural network usually outputs deterministically, it is difficult to output a random value such as a probability distribution. Therefore, the sampler 112 combines a random number sampled from a Gaussian distribution with the output from the neural network (high-resolution analysis data) to pseudo-express the probability distribution (re-parametrization trick). This makes it possible to estimate the error of the high-resolution analysis data. Specifically, the probability distribution is pseudo-expressed as shown in the following formula (11). Note that μ and σ are deterministic variables given by the neural network, and ε is a random number sampled from a Gaussian distribution.

...(11)

Specifically, the sampler 112 receives the high-resolution analysis data Da1 (corresponding to y in formula (10)) from the high-resolution analysis data acquisition unit 160. The sampler 112 samples random numbers from a Gaussian distribution and adds noise to the high-resolution analysis data. This adds randomness to the high-resolution analysis data provided by the neural network, making it possible to acquire data that can be considered as values sampled from a probability distribution. This makes it possible to express the high-resolution analysis data as a probability distribution. In other words, the sampler 112 acquires sampling data when the high-resolution analysis data is expressed as a probability distribution. The sampler 112 then outputs the sampling data of the high-resolution analysis data to the observation data generation unit 114. Note that when expressing a more complex probability distribution, that is, when noise is generated from a distribution more complex than a Gaussian distribution, a mixture distribution or normalizing flows may be used. In this case, a complex distribution can be obtained by overlapping nonlinear transformations on random variables that follow a simple probability distribution such as a Gaussian distribution.

The observation data generation unit 114 generates pseudo observation data Da3 using sampling data of the high-resolution analysis data generated by the sampler 112. In other words, the observation data generation unit 114 converts the sampling data of the high-resolution analysis data into pseudo observation data Da3. The observation data generation unit 114 can generate pseudo observation data Da3 that is free of loss in the time-space direction. The observation data generation unit 114 may be realized by a neural network that has been trained in advance by machine learning. The observation data generation unit 114 may be realized by a neural network that reflects physical symmetry. By generating the pseudo observation data Da3, unsupervised learning can be realized.

Specifically, the observation data generating unit 114 may generate the pseudo observation data Da3 by performing the reverse process of the process performed by the above-mentioned structurizer (structure conversion unit 130). In other words, the observation data generating unit 114 may generate the pseudo observation data Da3 by a technique substantially similar to that of the structurizer. More specifically, the observation data generating unit 114 picks up data of a lattice point at an arbitrary time and position from the sampling data of the high-resolution analysis data, which is lattice data, and repeats linear transformation and nonlinear transformation on the data. In this way, the observation data generating unit 114 acquires pseudo observation data Da3 in a format substantially similar to the format of the observation data o acquired by the observation data acquiring unit 122. Therefore, the pseudo observation data Da3 may be non-lattice data. Furthermore, the pseudo observation data Da3 may indicate the numerical value of a non-physical quantity.

As described above, the learning processing unit 110 to which the variational Bayes method is applied generates pseudo observation data Da3 from the high-resolution analysis data Da1. Here, in the variational lower bound expressed by the fourth equation of equation (10), the inferred high-resolution analysis data y is a hidden state. It should be noted that this hidden state is not the final output within the framework of the variational Bayes method. In the variational Bayes method, the pseudo observation data Da3 is the final output in the learning stage. And since the high-resolution analysis data y is in a hidden state, there is no need to prepare ground truth data corresponding to the high-resolution analysis data y in the learning stage. Therefore, it is no longer necessary to prepare highly accurate and high-resolution weather data, which is necessary for supervised learning.

(Hardware configuration example)
An example of the configuration of hardware resources for implementing the devices and systems according to the above-mentioned embodiments using one calculation processing device (information processing device, computer) will be described. However, the device (information processing device) according to each embodiment may be realized physically or functionally using at least two calculation processing devices. Furthermore, the device according to each embodiment may be realized as a dedicated device or a general-purpose information processing device.

FIG. 11 is a block diagram showing an example of the hardware configuration of a computing device capable of realizing the device and system according to each embodiment. The computing device 1000 has a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF 1007 (IF: Interface). Therefore, it can be said that the device according to each embodiment has a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF 1007. The computing device 1000 may be connectable to an input device 1005 and an output device 1006. The computing device 1000 may include an input device 1005 and an output device 1006. The computing device 1000 can also transmit and receive information to and from other computing devices and communication devices via the communication IF 1007.

The non-volatile recording medium 1004 is a computer-readable medium, such as a compact disc or a digital versatile disc. The non-volatile recording medium 1004 may also be a universal serial bus (USB) memory, a solid state drive, or the like. The non-volatile recording medium 1004 holds the relevant program without the need for a power supply, making it possible to carry it around. The non-volatile recording medium 1004 is not limited to the above-mentioned media. The relevant program may also be supplied via the communication IF 1007 and a communication network, instead of the non-volatile recording medium 1004.

The volatile memory device 1002 is computer-readable and can temporarily store data. The volatile memory device 1002 is a memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM).

In other words, when the CPU 1001 executes a software program (computer program: hereinafter simply referred to as a "program") stored on the disk 1003, it copies the program to the volatile storage device 1002 and executes the arithmetic processing. The CPU 1001 reads data required for executing the program from the volatile storage device 1002. When display is required, the CPU 1001 displays the output result on the output device 1006. When a program is input from the outside, the CPU 1001 obtains the program from the input device 1005. The CPU 1001 interprets and executes the program corresponding to the function (processing) of each component shown in the above-mentioned Figures 4, 6, and 10. The CPU 1001 executes the processing described in each of the above-mentioned embodiments. In other words, the functions of each component shown in the above-mentioned Figures 4, 6, and 10 can be realized by the CPU 1001 executing the program stored on the disk 1003 or the volatile storage device 1002.

In other words, each of the above-mentioned embodiments can be realized by the above-mentioned programs. Furthermore, each of the above-mentioned embodiments can be realized by a computer-readable non-volatile recording medium on which the above-mentioned programs are recorded.

(Modification)
The present invention is not limited to the above embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, in the above-mentioned flowchart, the order of each process (step) can be changed as appropriate. Also, one or more of the multiple processes (steps) may be omitted. For example, in the flowchart of FIG. 5, the process of S22 may be executed before the process of S20. Also, the process of S24 may be executed before the process of S22. Also, the process of S70 may be omitted.

In the above embodiment, a case where weather forecasting is performed has been described, but this embodiment is not limited to the case where weather forecasting is performed. This embodiment can be applied to any predictive simulation that uses grid data. For example, this embodiment can also be applied to ocean forecasting. This embodiment can also be applied to space physics simulations.

Furthermore, in this embodiment, the dimensions of "space-time" are not limited to four dimensions consisting of three-dimensional space and one-dimensional time. The dimensions of "space-time" may be three dimensions consisting of two-dimensional space and one-dimensional time. Alternatively, the dimensions of "space-time" may be a dimension greater than four, such as ten dimensions.

In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium may include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium may include electrical, optical, acoustic, or other forms of propagated signals.

This application claims priority based on Japanese Patent Application No. 2022-155991, filed on September 29, 2022, the entire disclosure of which is incorporated herein by reference.

10 Information processing device 20 Simulation unit 22 Observation data acquisition unit 24 Prediction data acquisition unit 30 Structure conversion unit 40 Latent space-time mapping unit 50 Nonlinear conversion unit 60 High-resolution analysis data acquisition unit 70 Low-resolution analysis data calculation unit 100 Information processing device 110 Learning processing unit 112 Sampler 114 Observation data generation unit 120 Simulation unit 122 Observation data acquisition unit 124 Prediction data acquisition unit 130 Structure conversion unit 140 Latent space-time mapping unit 150 Nonlinear conversion unit 160 High-resolution analysis data acquisition unit 170 Low-resolution analysis data calculation unit

Claims

a structure conversion unit that converts a structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values defined on lattice points arranged at predetermined intervals in space-time;
a latent space-time mapping unit that maps the observation data converted into the lattice data and prediction data, which is lattice data in a space-time obtained by a simulation and includes at least a time of the observation data and a time before the time, from a first real space-time to a latent space-time having a smaller number of elements than the first real space-time;
a nonlinear transformation unit that performs a nonlinear transformation on the mapped observation data and the predicted data in the latent space-time;
a high-resolution analysis data acquisition unit that acquires high-resolution analysis data, which is lattice data in space-time and has a higher resolution in space-time than the prediction data, by mapping the observation data and the prediction data that have been subjected to the nonlinear transformation from the latent space-time to a second real space-time that has a larger number of elements than the latent space-time and has a higher resolution than the first real space-time;
having
The latent space-time mapping unit and the nonlinear transformation unit perform data assimilation between the observation data and the prediction data.
Information processing device.
a low-resolution analysis data calculation unit that calculates low-resolution analysis data having a lower resolution in time and space than the high-resolution analysis data using the high-resolution analysis data;
The information processing apparatus according to claim 1 , further comprising:
The nonlinear transformation unit performs super-resolution in the time direction by transforming a data array on the data mapped to the latent space-time.
The information processing device according to claim 1 .
The high-resolution analysis data acquisition unit acquires the high-resolution analysis data by performing super-resolution in a spatial direction independently for each time in the time direction on the data that has been super-resolved in the time direction in the latent space-time.
The information processing device according to claim 3 .
the predicted data is predicted data for a time of the observed data and a time including a time in the past and a time in the future relative to the observed data,
The high-resolution analysis data acquisition unit acquires high-resolution analysis data at a time including the past and future of the observation data.
The information processing device according to claim 1 .
The structural transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model trained by a machine learning algorithm.
The information processing device according to claim 1 .
The structure transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model trained by supervised learning using data having a higher spatiotemporal resolution than the prediction data as training data.
The information processing device according to claim 6.
The structural transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model trained by unsupervised learning so as to reduce a loss function.
The information processing device according to claim 6.
Converting a structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values defined on lattice points arranged at predetermined intervals in space-time;
mapping the observation data converted into the lattice data and prediction data obtained by simulation, the prediction data being lattice data in a space-time and including at least a time of the observation data and a time before the time of the observation data, from a first real time-space to a latent time-space having a smaller number of elements than the first real time-space;
In the latent space-time, a nonlinear transformation is performed on the mapped observation data and the predicted data;
The observation data and the prediction data that have been subjected to the nonlinear transformation are mapped from the latent space-time to a second real space-time that has a larger number of elements than the latent space-time and a higher resolution than the first real space-time, thereby obtaining high-resolution analysis data that is lattice data in space-time and has a higher resolution in space-time than the prediction data;
performing data assimilation between the observation data and the prediction data by performing a mapping from the first real time space to the latent time space and performing a nonlinear transformation on the observation data and the prediction data in the latent time space;
Information processing methods.
A process of converting a structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values defined on lattice points arranged at predetermined intervals in space-time;
a process of mapping, from a first real time-space to a latent time-space having a smaller number of elements than the first real time-space, the observation data converted into the lattice data and prediction data, which is lattice data in a space-time obtained by a simulation and includes at least a time of the observation data and a time before the time of the observation data;
A process of performing a nonlinear transformation on the mapped observation data and the predicted data in the latent space-time;
a process of mapping the observation data and the prediction data that have been subjected to the nonlinear transformation from the latent space-time to a second real space-time that has a larger number of elements than the latent space-time and a higher resolution than the first real space-time, thereby acquiring high-resolution analysis data that is lattice data in space-time and has a higher resolution in space-time than the prediction data;
Run the following on your computer:
A process of mapping from the first real time space to the latent time space and a process of performing a nonlinear transformation on the observation data and the prediction data in the latent time space, thereby performing data assimilation between the observation data and the prediction data.
program.