WO2023198282A1

WO2023198282A1 - Generative emulator for controllable physical systems

Info

Publication number: WO2023198282A1
Application number: PCT/EP2022/059794
Authority: WO
Inventors: Bin Xiao; Toni MASTELIC; Darko Huljenic; Peter VON WRYCZA
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2022-04-12
Filing date: 2022-04-12
Publication date: 2023-10-19

Abstract

According to an aspect, there is provided a computer-implemented method of emulating data. In an embodiment the method comprises accumulating received time-series data (Xt-2, t-1, 5 Xt, Xt+1) into sliding windows (203) corresponding to one or more Actual DataFrames (206) of time-series data and reflecting a condition or a state or an action of a controllable physical system (105); and using a generative model (212, 512) to process the time-series data in each sliding window to output corresponding generated data emulating a variation in the condition or the state or the action of the controllable physical system.

Description

Generative emulator for controllable physical systems

Technical Field

This disclosure relates to the provision of emulated variations of conditions, states and actions in relation to controllable physical systems such as communications networks and factories.

Background

The Internet of Things (loT) is widely regarded as an enabler for the digital transformation of consumers and industries. Monitoring facility status (e.g. various equipment, manufacture, logistics, autonomous vehicle, etc.,), environments, and industrial processes are an important components of digital transformation. Continuously monitoring is achieved by employing a large variety of sensors in the environment or controllable physical system such as a radio access network (RAN), automated factory, or building management system. loT approaches can be used to gather and transmit the monitoring data.

The collected data can be used to create digital models for simulating equipment or even the surroundings of a controllable physical system. Such digital twin models can show punctual and individual problems, as well as utilize those data to precisely predict the probability of any issues that could happen in the near future. During this process, the models need to be generated automatically and take into consideration many possibilities in different conditions and for different purposes. This digital twin approach can be used to emulate variations in the surroundings and/or the system itself for verification of the physical assets and check for anomalies before any deployment changes affect the existing physical assets of the controllable physical system.

Digital Twin (DT) indicates a process of mapping physical entities to a digital representation using loT to enable continuous communication between physical and digital twins. The communicated data provides the ability to describe the physical entity as realistically as required by the use case using the DT model built as its digital representation. DT can use an loT platform to manage the operations by sharing and reusing resources (data, devices, models, etc..) across different use cases.

US20210141870A1 proposes the automatic creation of 3D digital twin models of some mechanical parts that can be imported into a simulation platform and be simulated. However, this is limited to a 3D model that can be used in a simulator. US20210138651 A1 proposes a direct connection between a robot program and a simulation platform so that the simulation can be performed on a 3D model with the real robot logics. US20190266295A1 proposes a DT based solution for preventive maintenance of vehicles by simulating future trips and predicting car component failures during those trips, as well as suggesting car mechanic stops along the way. The proposed solution utilizes historical data for simulation and uses expert models for simulating component failures.

Summary

In one aspect there is provided a computer-implemented method of emulating data. The method comprises accumulating received time-series data into sliding windows corresponding to one or more Actual DataFrames of time-series data and reflecting a condition or a state or an action of a controllable physical system; and using a generative model to process the time-series data in each sliding window to output corresponding generated data emulating a variation in the condition or the state or the action of the controllable physical system.

By using generative emulation, variations in conditions (e.g. robot arm temperature from an loT sensor), states (e.g. “robot arm is overheating”) and actions (e.g. “switch off robot arm”) can be considered when predicting possible system error or failure modes or other anomalies, as well as validating system components over a wider range of conditions than can be covered by simulations of the system. Simulations are limited to exactly modeling system behavior and therefore cannot consider variations that may occur in the future and which may result in failure modes. By contrast generative emulation enables the exploration of different but similar situations and possibilities that remain within the bounds of likely system behavior. The variations are diverse enough so that the majority of potential conditions and environment assumptions that have not happened before can still be evaluated and adapted for. The use of emulation also reduces the need for expert domain knowledge as the variations are based on actual controllable physical system data. This also means the approach can be readily adapted to changes in the controllable physical system.

Current digital twin (DT) implementations have a number of shortcomings. Different digital twin modeling is needed for different use cases and relies upon different domain knowledge. This is typically proprietary for specific use cases, which are created based on the proprietary knowledge previously gain from the specific scenario and use cases. The knowledge used to model the digital solutions is usually pre-existing and relies on the knowledge of domain experts. These limiting factors result in a number of problems with current DT implementations including: the created models are highly domain-specific and so are only application to one solution or use case making them hard to reuse; the models need manual interferences to be adapted using domain knowledge; and are applicable to limited conditions and situations.

By employing the emulation method in a digital twin, verification and anomaly checking can be conducted online without physical impact on the existing physical assets. This also enables system component verification and anomaly checking before the physical assets are hard settled. This is advantageous in many domains such as automotive, manufacturing, and communication networks such radio access network (RAN) because once the changes on hardware deployment are settled it can be highly costly to adjust the existing deployment. For example, frequently turning the system offline for error correction brings negative effects on customers’ experiences.

In another aspect there is provided a computer-implemented emulator for emulating data. The emulator comprises a processor and memory and is configured to accumulate received timeseries data into sliding windows corresponding to one or more Actual DataFrames of time-series data and reflecting a condition or a state or an action of a controllable physical system, and to use a generative model to process the time-series data in each sliding window to output corresponding generated data emulating a variation in the condition or the state or the action of the controllable physical system.

According to certain embodiments described herein there is provided a computer program comprising instructions which, when executed on a processor, causes the processor to carry out the methods described herein. The computer program may be stored on a non-transitory computer readable media.

Those skilled in the art will be aware of other benefits and advantages of the techniques described herein.

Brief Description of the Drawings

Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings, in which:

Fig. 1 is a schematic of a controllable physical system and a digital twin according to an embodiment;

Fig. 2 is a schematic of an emulator according to an embodiment;

Fig. 3 illustrates a method of emulating data for a controllable physical system according to an embodiment;

Fig. 4 is a schematic of an emulator according to another embodiment;

Fig. 5 is a schematic of an emulator according to another embodiment; and

Fig. 6 illustrates deployment of an emulator according to an embodiment in different use cases.

Detailed Description

Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used. All references to a/an/the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

The following sets forth specific details, such as particular embodiments or examples for purposes of explanation and not limitation. It will be appreciated by one skilled in the art that other examples may be employed apart from these specific details. In some instances, detailed descriptions of well-known methods, nodes, interfaces, circuits, and devices are omitted so as not obscure the description with unnecessary detail. Those skilled in the art will appreciate that the functions described may be implemented in one or more nodes using hardware circuitry (e.g., analog and/or discrete logic gates interconnected to perform a specialized function, ASICs, PLAs, etc.) and/or using software programs and data in conjunction with one or more digital microprocessors or general purpose computers. Nodes that communicate using the air interface also have suitable radio communications circuitry. Moreover, where appropriate the technology can additionally be considered to be embodied entirely within any form of computer-readable memory, such as solid-state memory, magnetic disk, or optical disk containing an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein.

Hardware implementation may include or encompass, without limitation, digital signal processor (DSP) hardware, a reduced instruction set processor, hardware (e.g., digital or analogue) circuitry including but not limited to application specific integrated circuit(s) (ASIC) and/or field programmable gate array(s) (FPGA(s)), and (where appropriate) state machines capable of performing such functions. Memory may be employed to storing temporary variables, holding and transfer of data between processes, non-volatile configuration settings, standard messaging formats and the like. Any suitable form of volatile memory and non-volatile storage may be employed including Random Access Memory (RAM) implemented as Metal Oxide Semiconductors (MOS) or Integrated Circuits (IC), and storage implemented as hard disk drives and flash memory.

Embodiments described herein relate to methods and apparatus for emulating data associated with a controllable physical system. This data may relate to conditions (e.g. robot arm temperature from an loT sensor), states (e.g. “robot arm is overheating”) and actions (e.g. “switch off robot arm”). By using generative emulation, variations in these parameters can be considered when predicting possible system error or failure modes or other anomalies, as well as validating system components over a wider range of conditions than can be covered by simulations of the system. Generative emulation enables the exploration of different but similar situations and possibilities that remain within the bounds of likely system behavior. However, the variations are diverse enough so that the majority of potential conditions and environment assumptions that have not happened before can still be evaluated and adapted for. In some embodiments the emulator can be used in a digital twin (DT) of the controllable physical system. The controllable physical system itself may vary across different domains, such as a communications system, a factory, a building or city, a transportation system such as a car or ship.

Fig. 1 illustrates a system 100 including a controllable physical system 105, an loT platform 110, a digital twin platform with emulators 150 and an evaluation and/or updating function 180.

In this example, the controllable physical system 105 is a factory, however other types of controllable physical system may be substituted, for example a radio access network (RAN), an apartment building, an autonomous truck. The factory 105 is associated with a number of sensors 115 which may be positionally distributed and may be used for measuring physical properties at different locations, for example: temperature; light; humidity; sound; pressure; number of physical parts; movement; vibration; component location. The controllable physical system 105 is also associated with a number of actuators 125 which may control or adjust various aspects of the factory. Examples include: switching on/off or speed control of cooling fans; switching on/off components such as robot arms or autonomous vehicles; switching on/off of power control of lights.

The sensors 115 may be Internet of Things (loT) devices that communicate with the rest of the system 100 using Third Generation Project Partnership (3GPP) Fourth Generation (4G) or Fifth Generation (5G) cellular, WiFi™ or other communications technology. This enables real-time data to be received from the sensors. Similarly, the actuators 125 may be loT devices that can be controlled in real-time by the rest of the system.

The loT platform 110 converts raw output from the sensors 115 into structured data that is delivered to the digital twin (DT) platform 150. The loT platform 110 comprises a data model 130 and a semantic model 135 for interpreting output from the sensors 115 and controlling input to the actuators 125. The data model 130 takes a sensor input such as a voltage and converts this into structured data (e.g. 20 units) that can be understood by other parts of the system. This may be implemented by one or more computing units in the loT platform which may be located on premises or on a network edge to reduce latency. An example of the structured data output by the data model is JSON (JavaScript Object Notation). The data model may communicate with the sensors and actuators using local communications devices such as Zigbee, Z-Wave and WiFi modules.

The semantic model 135 puts the sensor data in context such as adding that the reading is from a temperature sensor (e.g. 20C) and what and where this is located within the controllable physical system 105 (e.g. robot arm 1 enclosure). The semantic model generates knowledge representations of the controllable physical system 105 representing logic and knowledge for a certain use case such as a factory. In other words the semantic model 135 receives uniformly formatted data from the data model 130 and provides context for this using schemes such as RDF (resource description framework), OWL (web ontology language), NGSI-LD (next generation service interface for linked data).

The DT platform 150 comprises a behavior model 155, a decision model 160, together with a first emulator 165 associated with the behavior model and a second emulator 170 associated with the decision model. The behavior model 165 receives time series data from the semantic model 135 corresponding to sensor measurements from the controllable physical system 105. The received data corresponds to conditions within the physical system 105, such as “robot arm 1 temperature". The behavior model 155 models one or more states of the physical system 105, such as “the robot arm is overheating". In other words, the behavior model 155 generates an understanding of the current states of the controllable physical system 105. These state outputs may be generated using rules, logic and algorithms based on expert knowledge and/or may be employ a machine learning model trained on sensor based data corresponding and known states.

The decision model 160 receives the state outputs from the behavior model 155 and generates actions which can then be used to activate changes in operating parameters of the controllable physical system, such as “switch off the robot arm". The actions are passed through the semantic model 135 and data model 130 to control an actuator to make the desired system change. As with the behavior model, the action outputs may be generated using rules, logic and algorithms based on expert knowledge and/or may employ a machine learning model trained on states and known responsive actions.

In this way the controllable physical system 105, the sensors 115, the data model 130, the semantic model 135, the behavior model 155, the decision model 160 and the actuators 125 form a control loop to control the controllable physical system so that it operates as intended, whilst avoiding failure modes, faults and/or component failures. For the example above, the controllable physical system operates using a robot arm whilst it remains below a threshold temperate, and switches this off when the threshold temperature is exceeded.

The behavior model 155 and decision model 160 may be located at a network edge to reduce latency. The data model 130 and/or the semantic model 135 may alternatively be located within the DT platform 150. Any suitable loT platform may be employed, for example using computing processors and one or more communications networks.

The emulators 165, 170 are described in more detail below, but these function to emulate variations in the inputs to the behavior model 155 and the decision model 160 so that these variations can be evaluated to consider as yet unencountered conditions and states. This is useful because in complex controllable systems 105 it is not possible to anticipate all possible conditions or states and so the behavior and decision models will necessarily be limited compared with the wide variety of possible conditions and states in their physical twin 105. It is not useful to consider extreme conditions or states that would never happen in practice because these may skew the models 155, 160 too much for the normal range of operations. Therefore, emulators of an embodiment use a generative approach to generate different but similar conditions and states to allow for evaluation and upgrading of behavior model 155 and the decision model 160 to improve operation of the controllable physical system 105.

The emulators 165, 170 may be located in the Cloud as latency is less important as the evaluation can be done offline without interrupting operation of the controllable physical system. In different embodiments, different numbers of emulators can be employed, for example only one of the behavior model emulator 165 or the decision model emulator 160 may be used. In another alternative, another emulator may be used to emulate variations in actions output from the decision model 160. This may be useful when commissioning the controllable physical system 105 to test a wider range of possible actuator inputs and their effects on the system 105. Thus unexpected faults may be detected before the system goes live and therefore avoid taking the system offline to repair the fault when it does occur.

The first emulator 165 takes as its input the time-series data received by the behavior model 155 and corresponding to conditions in the controllable physical system 105. The first emulator 165 generates variations in the time-series data (corresponding to conditions) that are similar to the actual time-series data received by the behavior model 155. The second emulator 170 takes as its input the outputs received by the decision model 160 from the behavior model 155 and corresponding to states in the controllable physical system 105. The second emulator 170 generates variations in the states that are similar to the states actually received by the behavior model 155. Were a third emulator used for actions, this would receive the state outputs from the decision model 160 and generate variations of these which are still similar.

The emulated data (conditions, states, actions) is then stored and/or communicated to an evaluate and update function 180. This function may operate offline compared with the behavior model and decision model 160 which remain online to help operate the controllable physical system 105. Emulated time-series data from the first emulator 165 may be feed through an offline clone of the behavior model to generate states which can then be evaluated for out-of-range or other negative situations to be avoided. This is turn may be used to modify or update the behavior model to better represent states of the controllable physical system based on a wider range of likely conditions. Emulated states from the second emulator 170 may be feed through an offline clone of the decision model to generate actions which can then be evaluated for out-of-range or other negative situations to be avoided. This is turn may be used to modify or update the decision model to offer better actions for operating the controllable physical system based on a wider range of likely conditions.

Once these models are updated, they may replace the existing behavior model 155 and decision model 160. Variations in actions generated by an emulator may be stored for later use and then run on the controllable physical system 105 during commissioning, to test a wider range of actions than would be generated by the decision model 160.

As the emulation is variational based on actual data used by the DT, it is more diverse such that the majority of potential conditions and environment assumptions that have not yet happened can be included in the models. In embodiments the emulators use a generative model so whilst their outputs vary they are still similar to the actual models they are emulating. For example the emulated data obeys the same distribution patterns are it’s input samples.

Fig. 2 illustrates an emulator according to an embodiment. This emulator can be used in the first or second emulator 165, 170 described above although alternative emulator arrangements can also be used. The emulator 200 comprises a generative model 212, a timeseries model 222, and a discriminator network 252. The emulator receives and accumulates timeseries data X_t-2, XM , X_t, X_t+i into sliding windows 203, each sliding window in a time-frame corresponding to Actual DataFrames 206.

The generative model 212 of this embodiment is a variational autoencoder (VAE) so the output datasets have the same probabilistic features as the datasets to be emulated. In alternative embodiments, different generative models may be used, for example a Contractive Auto-Encoder (CAE) . The time-series model 222 is based on gated recurrent units (GRU) to handle continuously generated data from a controllable physical system. This ensures that timely relations between different batches of data is reflected during training. In alternative embodiments, different time-series models may be used, for example a long short term memory (LSTM). The discriminator network 252 is the discriminator of a generative adversarial network (GAN) which strongly forces the data generated by the VAE under each time unit to closely follow the original data. This ensures that the emulated data is similar to the original data, following similar probabilistic features. Other methods of unsupervised learning may alternatively be used.

The illustrated emulator architecture is known here as a VGHT or Variational Generator for High-dimensional Time Series model architecture. The VGHT has the following features: generative learning, supervised learning, and online learning. These features enable it to continuously generate emulating data sets from the data generated by physical twins which takes multivariable time series data as input.

The emulator can receive both online and historical datasets of high-dimensional time series data which have detected or no relations between each dimension between the datasets. The datasets can be generated both by physical (e.g. 105) and digital (e.g. 155) entities in a digital twin framework. The datasets output by the emulator are infinitely approximating to the input datasets by infinitely approximating to the probabilistic distribution; whilst each portion of generated datasets are not the same as the portion of actual datasets.

A specific size of the time window is defined with which to accumulate the received data - for example the data from the semantic model corresponding to conditions in the physical system. This groups the data into mini-batches and is configurable dependent on the use case. Each sliding window outputs an Actual DataFrame in temporal order with a defined interval.

The data from different sensors 115 is integrated into multiple dimensions for example by the semantic model 135 which supplies formatted data to the behavior model 155 (and the behavior model emulator 165). In a very simple example, a multidimensional data item at time t (e.g. X_t) may comprise {1 ; 12; 67; 21} corresponding to: {robot arm power status; robot arm location 1 temperature; robot arm location 2 temperature; pressure}. The dimension of the data frames can be considered as a set of parameters, where all those parameters together describe how the physical system works. Therefore, there is no specific requirements on what each parameter should be. As explained previously, these dimensions may be collected from heterogenous sensors with different communications protocols and data formats. These are universalized by the data model and semantic model.

Once enough data has been accumulated, the emulator can be trained, for example in the previously described digital twin framework, until it is ready for deployment. Once deployed, the emulator can continue to be trained in situ. The emulated output of the (continuously) trained emulator minimizes Kullback-Leibler (KL) divergence with the input sample. The VGHT approach enables emulated data obeying the same probabilistic patterns as the input data with no or only limited domain expertise and no human input. The generative models is trained in an unsupervised manner whilst the discriminator network is trained in a supervised manner.

Using a generative model in the emulator ensures that the emulated (condition, state or action) data is based on actual (condition, state or action) data so that it will be similar to the actual data and thereby representing likely scenarios. The emulated data has the similar probabilistic features as the actual data. This contrasts with emulated data that may be generated in other ways and may therefore represent very unlikely situations which is less useful for fine tuning digital twin models 155, 160.

In the embodiment, processing of the input data is performed in two dimensions, namely the feature dimensions (using generative model 212) and the time dimension (using time-series model 222). In the generative model 212, data are processed in the feature dimension. The generative model 212 comprises a number of stacked variational autoencoder (VAE) elements, corresponding to the number of data items in the sliding window (in this example four data items corresponding to X_t.2, XM, X_t, X_t+i). These include VAE encoders 214, VAE latent spaces 216 and VAE decoders 218. The latent spaces 216 are connected in time sequence as shown however the VAE-DE 218 are not connected in sequence unlike in the situation where the VAE are simply chained in time sequence.

To approximate the real probabilistic distributions of the inputted data, variations from the generative model 212 are trained to re-construct the input during each training epic so that after each training epic the output of the generative model 212 is forced to move close to the samples in the targeted distributions. After training the generative model the time-series model 222 will be used to generate emulated data samples from the output of the generative model 212. Each data item X represents a data item whose width is the same as the number of variables and the number of data items and the length of the dataframe 206 is the same as a predetermined time period (for example, a data frame with multivariables collected within an hour). Each accumulated dataframe 206 (from received multivariables at respective time periods t-2, t-1 , t, t+1 etc) corresponds to the length of the sliding window 203 which may be defined by a time period or a number of samples.

The stacked VAE units 214, 216, 218 for each data item concentrates information from the high-dimensional data items and then reconstructs it, so that the loss is more meaningful than accuracy The use of the generative model results in lots of variations whilst ensuring the generated output follows the same distributions as the input does, for example to minimize KL divergence). This is because the training labels and input are the same datasets, and each VAE stack tries to minimize the distances between the original data and the reconstructed data. The computation of accuracy/loss may be conducted using K-folder cross-validation, where the K number has defaulted as 5 unless further configuration is provided.

In the emulator 200, data are processed in the time dimension using the time-series model 222. In this embodiment, the time-series model 222 comprises one or more stacked gated recurrent units (GRU) 224, 226, each connected to the output of a stacked VAE of the generative model 212. Each GRU is connected in sequence and an additional GRU stack for time t+2 may be included as input for another sliding window so that all sliding windows are chained. In an example, the deployed sensor transmits data in every 10 milliseconds and the sliding window length is 10 seconds, therefore each sliding window will accumulate 1000 data items. Such time series units support the neural network to conduct training on the input data with continuations on different time slots.

The output of each GRU stack 224, 226 is an emulated data item X’ corresponding to a data item X in the sliding window 203. The data items in a sliding window correspond to an Actual DataFrame 206 { X_t.2, XM , X_t, X_t+i} and the data items output by the time-series model 222 in a corresponding sliding window 233 represent a Generated DataFrame 246. The Generated DataFrames 246 are output from the emulator 200 as emulated data.

The discriminator network 252 is used to train the generative model 212 and time-series model 222 to force the generated time series data { X’_t.2, X’_t.i, X’_t, X’_t+i} to be highly alike the received time series data { X_t.2, XM, X_t, X_t+i}, in other words so that the Generated DataFrames and the Actual DataFrames exhibit similar probability features. In the embodiment, the discriminator network 252 is from a generative adversarial network (GAN) which the generative model 212 and the time-series model 222 form the generative part of the GAN. The GAN training makes an impact on each sliding window and between different sliding windows. This supplements the maximum-likelihood in each VAE training, as the reinforced likelihood of each VAE only makes impacts on the respective data items within a sliding window. Any GAN training function for the discriminator network may be employed, which takes the output of the time-series model and the original dataset to ensure the output is very similar.:

Training of the generative model and the time-series model uses a number K of sliding windows or the length of the time series and M datasets of generated datasets for emulation. As M and K increase, the datasets are approaching a normal distribution.

Fig. 3 illustrates a method of operating a digital twin of a controllable physical system as well as an emulator according to an embodiment. The method 300 is described with respect to system 100 however may be implemented in any other suitable system.

At 305, the method collects sensor measurements or readings of a controllable physical system 105. This may be implemented using a loT platform having corresponding loT devices. At 310, the sensor outputs are transformed into structured time series data such as a multivariable data item X for each time period t. The sensor measurements and structured data correspond to conditions in the physical system 105, such as a temperature in a robot arm enclosure or a pressure at a press plate.

At 315, the method processes the structured time series data using a behavior model 155 which outputs data corresponding to states of the controllable physical system 105, based on the input data corresponding to conditions in the controllable physical system. An example state may be that the robot arm is overheating.

At 320, the method determines an action response to the state using a decision model 160. The action may be to switch the robot arm off in response to the state that it is overheating. The action may also be dependent on one or more other states such as a press plate is over-pressure.

Both the behavior and decision models correspond to a digital twin (DT) of the controllable physical system 105. Both models may be trained to control the controllable physical system 105 to operate within a normal range of operational parameters and to shut down some or all parts of the system in response to unexpected or failure conditions. However, the models cannot be trained for all possible situations that the system may encounter.

At 325, the method implements the action, for example by signaling an actuator 125 to adjust an operational parameter of the controllable physical system 105, such as switching of a robot arm.

At 330, the method accumulates the received structured time series data into sliding windows 203 of a given length, by time or number of data items for example. At 335, the sliding windows are separately processes through a generative model 212 which outputs generated data. At 340, the generated data is processed through a time series model 222 to generate emulated data 233, 246. Each sliding window 203 corresponds to an Actual DataFrame 206 and the emulated data 233 output for each sliding window 203 corresponds to a Generated DataFrame 246.

At 345, the Generated DataFrames 246 and corresponding Actual DataFrames are processed to reinforce similarity by further training of the generative model 212 and the timeseries model 222. This is achieved using a GAN where the generative and time series models 212, 222 are configured as the generative part of the GAN and the discriminator part of the GAN is used to update these models 212, 222.

At 350, the Generated DataFrames 246 are output as emulated data which is similar to the input data corresponding to the Actual DataFrames 206. The emulated time-series data comprises variations on the actual data used by the behavior model and may be used to detect anomalies or to verify system components across a wider range of conditions.

In this way, emulated data is output in parallel with the online operation of the DT platform, and may be used offline to evaluate the behavior and decision models, for example using clone models. If appropriate, these models may then be updated to accommodate the variations created by the emulator. Whilst emulating variations in condition related time series input data has been described, an emulator may also be used to generate variations in state related data or even action related data as previously described.

Fig. 4 illustrates an implementation of an emulator according to an embodiment. The emulator 400 comprises one or more processors 432, one or more memory elements 442 and one or more communications interfaces 434. The memory 442 contains processor readable instructions 448 and one or more machine learning models 446.

The machine learning models 446 may include a generative model 212 comprising VAE units and a time series model 222 comprising GRU configured as a GAN using a discriminator network.

The processor readable instructions 444 may include instructions 452 to accumulate received time series data into sliding windows. The accumulated time series data may correspond to data received by a behavior model of a DT (system conditions), data received by a decision model of the DT (system states), or data output by the decision model of the DT (system actions).

Instruction 454 causes the processor to process the accumulated data using the generative model 212. Instruction 456 causes the processor 432 to process the generated data output from the generative model using the time-series model 222 in order to output emulated data.

Instruction 458 causes the processor to use a discriminator network 252 to update the generative and time series models.

Fig. 5 illustrates operation of an emulator according to an embodiment. In manufacturing, it is useful to emulate the production line and automatically figure out any impact of possible changes. However, a domain expert may not be available to understanding the data, especially when the production line is newly introduced or assembled. The categories of deployed equipment and/or sensor devices can be very complex (high-dimensional data); the speed of generating data can be very quick (data streams come in high-density); the equipment/sensor devices may be serving different production units in different geographical locations (heterogeneity of devices and distributed topology); the production can be scale up and down for serving different environments; and the data should be ingested quickly in order to provide online results quickly. To address such a situation, a DT based emulator machine may be employed.

The illustrated embodiment can provide online emulation of a physical deployment with online, where each function of the emulator 500 is implemented with a separate computing unit as shown. However, these functions may alternatively be implemented using a single (or another number of) computing unit. The data receiver 504 receives all data from physical entities such as sensors in a controllable physical system. The data parser 507 transforms the raw data (such as sensor voltage outputs or data in proprietary formats) into a structured dataframes each comprising multivariable data items.

The generator units 512 may be stacked VAE units for each data item in a dataframe to output generated data for each dataframe which comprises a number of data items corresponding to the length of a sliding window used to accumulate the data items into dataframes. The timeseries units 522 may be stacked GRUs for each data item in the dataframe to output emulated data for each dataframe which comprises a number of emulated data items corresponding to the length of a sliding window used to accumulate the data items into dataframes. Each of the timeseries units handles generated data from the physical entities over a certain time period. The VAE 512 units interact with time-series units 522 for back-propagation during training and inferencing.

The GAN units only work for GAN computations which take input form the actual dataframes and the generated dataframes from the output of the GRU layer and trains the VAE and GRU to force the actual and generated dataframes to be highly alike.

The exposer 556 handles buffering and sending of emulated datasets.

Examples follow of use cases where emulators according to embodiments are employed in digital twins of controllable physical systems. The DT emulator takes device data from any loT environment as an input (i.e., various time-series sensor data) and provides diverse and variational data as an output (i.e., data that looks like the input sensor data). The DT emulator generates output by first training a VGHT model to learn the pattern of input data and then generating data from the trained model where variations are added. Device data flows into the emulator as a live stream, while the DT emulator uses sliding windows to process the incoming data. The content of each sliding window is fed into the VGHT and the emulator runs continuously and takes input from the moving sliding windows continuously.

Online Device verification: In many use-cases, complex hardware is deployed and needs to be upgraded/changed based on different requirements. To avoid the situation that the running hardware systems have to be stopped fortesting (which may cost quite some time to test all the possibilities), the Emulator enables engineers to do online device verifications. That is to use the emulator to learn a pattern from all the collected hardware data, and then generate data emulating the hardware systems (with possible environment conditions) with variations. The emulation data sets enable tests on the hardware systems without having to shut these down. Once data is emulated it can be used for verifying physical device state, not only by observing its current state but by observing all possible states that might occur.

Sustainable urban environment:

A urban environment may be an apartment building, an office building, a university campus with different deployed sensors (e.g. temperature sensors) and actuators (e.g. heater with several power levels). A DT emulator learns a pattern that when the heater is turned on the temperature of houses in the urban area can logarithmically go up depending on the power level. Consequently, DT emulator can emulate different conditions to verify states. If the desired temperature is set to 35 degrees and the DT emulator did not find any set of conditions that would achieve that temperature, it can alarm the user that the desired temperature is unreachable. Moreover, when any changes towards the heating system are needed, the data output by the emulator can be used to test the change proposal before making any physical impact on the existing physical entities.

Manufacturing:

A factory environment may include a robotic arm and a conveyor belt on a factory floor, as well as a camera detecting items on a conveyor belt being placed by the robotic arm. A DT emulator can take camera outputs, robotic arm axes, and the conveyor belt speed and learn their correlation and continuously update them as new data arrives. By creating variations of those conditions, the DT emulator can check if any device can reach an undesirable state during its operation. For instance, it can be used to predict that the minimum speed of the conveyor belt would cause the robotic arm to place items one on top of the other if the robotic arm itself were to be set to its maximum speed. In case any changes are needed for the running physical setup, the data generated by the emulator can be used to verify the proposed changes before any impacts are made on the physical system.

Mobile Network:

In an example there are 200 LTE base stations covering an area, all being digitally twinned. Input data includes base station load and a number of mobile devices connected to it. DT emulator begins to emulate mobile device positions and connections and finds a condition where one base station fails due to overload. Mobile devices that were connected to that base station switch to other base stations, which also gets overloaded as they now have to handle new connections along with the existing ones, making them fail as well. This can create a chain reaction that knocks off all 200 base stations. In this way, the DT emulator is capable of verifying the setup/configuration of the entire LTE network in the area. Such verification is particularly helpful when any changes need to be done on the hardware setup because the running hardware system does not need to be stopped to test potential consequences of the planned changes.

Anomaly detection:

The emulator generates data emulating the hardware system, which can be also used to detect anomalies in the existing hardware system because the DT emulator is also aware of various environment's acceptable states because of the variations in emulated data.

Sustainable urban environment:

If the temperature starts going up while a heater is turned off, this may correspond to a fire in a room for instance. This data can be used as an input to the discriminator of the DT emulator along with other emulated data representing various acceptable situations. That being done, the DT emulator would be able to identify the fire situation as an anomaly amongst other conditions. Moreover, using the emulating data with variations, it is possible to test whether the current hardware setup can work as the expected way under different (with variations) circumstances/assumptions (can the system respond to anomalies correctly under different circumstances/assumptions).

Manufacturing:

For the factory above, consider the conveyor belt stops operating due to malfunction, while the robotic arm continues its operation. The DT emulator would be able to detect this as an anomaly as these conditions would not resemble any other previously emulated working conditions. Moreover, using the emulating data with variations, it is possible to test whether the current hardware setup can work as the expected way under different (with variations) circumstances/assumptions (can the system respond to anomalies correctly under different circumstances/assumptions).

Mobile Network:

For the example above, one base station stops sending sensor data due to an internal error, while continuing to operate, i.e., while mobile devices are still connected to it. The DT would be able to emulate these conditions and this data can be used to realize that if the base station has actually failed some of the neighboring base stations would receive new connections, while the failed base station doesn’t. Hence this situation can be detected as an anomaly. Moreover, using the emulating data with variations, it is possible to test whether the current hardware setup can work as the expected way under different (with variations) circumstances/assumptions (can the system respond to anomalies correctly under different circumstances/assumptions).

Fig. 6 illustrates a Cloud implementation. Some of the units in the emulator can be implemented in a different location in a distributed way. Instead of relating to cloud implementation only, the whole loT landscape may be employed consisting of devices, edge gateways, base stations, network infrastructure, fog nodes, and/or cloud. The Generator Units, Time-series Units, and GAN units could be implemented in the cloud which can benefit by the computation power of the cloud. Other units, such as data receiver, exposer, and data parser can be located in the edge, depending on the needs of the scenario.

The cloud based implementation 600 may include a controllable physical system 605 such as a manufacturing, automotive, agricultural, urban or transportation environment. These are connected to communications gateways 610 such as 3GPP base stations, WiFi, BT or other indoor/outdoor small cells, or a local loT gateway. These in turn are connected to network infrastructure 615 such as radio network and optical nodes which in turn is connected to a cloud environment 620 such as a datacenter. In order to reduce latency whilst enabling significant computing resources for model processing, a data receiver 504 may be located at the gateways 610, a data parser 507 may be located in the network infrastructure 615 and the VAE, GRU and GAN models 512, 522, 552 may be located in the cloud 620.

Embodiments provides an emulator using a novel machine learning model (i.e., VGHT ) to generate high-dimensional time-series data sets which are reinforced to approach the targeted original data sets in probabilistic space without dependency on domain expertise, which is hence highly replicable and reusable for various industrial applications. The emulator inherits and integrating the advantages and features of VAE, GRU, and GAN providing synergy effects. This may be employed in DT platforms and learns and generates data in a variational way. After training, each data generation is a process of sampling data from the learned probabilistic distributions. So that each generated set is different but obeys (approaching to) the probabilistic distribution learnt from input data. Therefore, it is diverse such that the majority of potential conditions and environment assumptions that have not happened before can be included in the model. Use in a DT platform makes it easy to expand and add other new functionalities by interacting with other components in the platform, such as logic-reasoning to answer “what-if’ type questions of the operation of the twins controllable physical system. Some embodiments apply online batch-based machine learning: continuously applying batch-based machine learning algorithms on time series to generate data sets for simulating the data collected from the physical entities, which can concentrate information in both KPI dimension and time dimension using VAE and Sliding Windows. Embodiments do not require domain expertise for providing data labels for either training the model or creating the model; the solution is automated without human intervention. Embodiments are also highly reusable and replicable: since the solution is semisupervised reinforcement learning without requirement on either training labels or domain expertise, thus it is highly replicable and reusable in different industrial applications.

Embodiments may provide a number of advantages. For example, the emulation is variational, therefore it is diverse such that the majority of potential conditions and environment assumptions that have not happened before can be included in the model. The VGHT emulator is a generative learning model which inherits some core functionality of VAE, GRU, and GAN with a clear functional synergy that a single parent model could not provide. Creating emulations from the data flow in an online manner using batch-based algorithms, where the batch-based algorithms are usually conducted in an offline manner. The online manner provides faster results than the offline manner, and the batch-based algorithms enable to handle advanced analysis. The emulator emulated data sets from high-dimensional time-series data flows and does not require pre-existing domain expertise to create the models used. Instead performance evaluation and training the models are done autonomously from the data flow. The solution is highly reusable and replicable for different scenarios due to being flexible, scalable, and does not require preexisting domain expertise.

Whilst the embodiments have described with respect to using emulators in a DT platform, the emulator could be employed in other settings.

Some or all of the described apparatus or functionality may be instantiated in cloud environments such as Docker, Kubernetes or Spark. This cloud functionality may be instantiated in the network edge, apparatus edge, or on a remote server coupled via a network such as 4G or 5G. The network edge in this sense may be computing resources located adjacent the network end points such in a Radio Access Network (RAN) to enable low delay and low latency when determining actions for controlling the controllable physical system. This improves real-time monitoring and control, however other functions may be performed in more remote Cloud locations, such as using the emulations to test and adapt model behavior. In a similar way, some functions such as the data model or semantic model may be performed at the apparatus edge which is adjacent the end or connection points of sensors and actuators located within the controllable physical system. Alternatively, this functionality may be implemented in dedicated hardware.

Modifications and other variants of the described embodiment(s) will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is understood that the embodiment(s) is/are not limited to the specific examples disclosed and that modifications and other variants are intended to be included within the scope of this disclosure. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1 . A computer-implemented method of emulating data, the method comprising: accumulating received time-series data (Xt-2, t-1 , Xt, Xt+1) into sliding windows (203) corresponding to one or more Actual DataFrames (206) of time-series data and reflecting a condition or a state or an action of a controllable physical system (105); using a generative model (212, 512) to process the time-series data in each sliding window to output corresponding generated data emulating a variation in the condition or the state or the action of the controllable physical system.

2. The method of claim 1 , comprising using a time-series model (222, 522) to process the generated data for each sliding window to output a Generated DataFrame (246) of emulated time-series data (X’t-2, X’t-1 , X’t, X’t+1).

3. The method of claim 1 or 2, comprising using a discriminator network (252, 552) to update the generative model (212, 512) using the Actual DataFrames (206) and the Generated DataFrames (246) or the generated data.

4. The method of claim 3, wherein: the generative model comprises a Variational Autoencoder, VAE, for each data in the sliding window: the timer-series model comprises one or more Gated Recurrent Units, GRU, corresponding to each data in the sliding window; the VAE, the GRU and the discriminator network are configured as a Generative Adversarial Network, GAN.

5. The method of any one preceding claim, comprising using the emulated variations in the condition or the state or the action in the controllable physical system to predict an anomalous state or action or condition in the controllable physical model.

6. The method of claim 5, comprising representing and/or controlling the controllable physical system using a digital twin model of the controllable physical system, and using the digital twin model and the emulated time-series data to predict the anomalous state or action or condition.

7. A computer-implemented method of emulating data associated with a controllable physical system, the method comprising: receiving time series data corresponding to a condition in the controllable physical system (305, 310); using a behavior model to determine a state of the controllable physical system using the time series data (315); using a decision model to determine an action dependent on the state of the controllable physical system, the action corresponding to a change in a condition of the controllable physical system (320); emulating time-series data corresponding to variations in the condition and/or state of the controllable physical system , the emulating according to the method of any one of claims 1 to 5 (330, 335, 340, 345).

8. The method of claim 7, comprising using the behavior model and the emulated variations in the condition to predict an anomalous state and using the decision model and the emulated variations in the state to predict an anomalous action.

9. The method of claim 8, updating the behavior model or the decision model in response to predicting an anomalous state or action.

10. The method of any one of claims 7 to 9, comprising emulating time-series data corresponding to variations in actions to be applied to the controllable physical system, the emulating according to the method of any one of claims 1 to 5 (330, 335, 340, 345).

11 . The method of claim 10, comprising monitoring the controllable physical system to determine an anomalous condition in response to variations in actions.

12. The method of any one of claims 7 to 11 , wherein the behavior model and the decision model are implemented in a network edge and the emulating is implemented on a network cloud.

13. The method of any one preceding claim, wherein the received time-series data depends on sensor measurements.

14. The method of any one preceding claim wherein the controllable physical system comprises sensors, actuators and physical components and is associated with one of the following: a communications network; manufacturing site; a controllable urban environment.

15. A computer-implemented emulator for emulating data and comprising a processor (432) and memory (442) configured to: accumulate received time-series data (Xt-2, t-1 , Xt, Xt+1) into sliding windows (203) corresponding to one or more Actual DataFrames (206) of time-series data and reflecting a condition or a state or an action of a controllable physical system (105); use a generative model (212, 512) to process the time-series data in each sliding window to output corresponding generated data emulating a variation in the condition or the state or the action of the controllable physical system.

16. The emulator of claim 15, configured to use a time-series model (222, 522) to process the generated data for each sliding window to output an Generated DataFrame (246) of emulated time-series data (X’t-2, X’t-1 , X’t, X’t+1).

17. The emulator of claim 15 or 16, configured to use a discriminator network (252, 552) to update the generative model (212, 512) using the Actual DataFrames (206) and the Generated DataFrames (246) or the generated data.

18. The emulator of claim 17, wherein: the generative model comprises a Variational Autoencoder, VAE, for each data in the sliding window: the timer-series model comprises one or more Gated Recurrent Units, GRU, corresponding to each data in the sliding window; the VAE, the GRU and the discriminator network are configured as a Generative Adversarial Network, GAN.

19. The emulator of any one of claims 15 to 18, configured to use the emulated variations in the condition or the state or the action in the controllable physical system to predict an anomalous state or action or condition in the controllable physical model.

20. The emulator of claim 19, configured to represent and/or control the controllable physical system using a digital twin model of the controllable physical system, and using the digital twin model and the emulated time-series data to predict the anomalous state or action or condition.

21 . A computer-implemented digital twin system (DT) associated with a controllable physical system, the DT comprising a processor (432) and memory (442) configured to: receive time series data corresponding to a condition in the controllable physical system (305, 310); use a behavior model to determine a state of the controllable physical system using the time series data (315); use a decision model to determine an action dependent on the state of the controllable physical system, the action corresponding to a change in a condition of the controllable physical system (320); use an emulator according to any one of claims 15 to 19 to emulate time-series data corresponding to variations in the condition and/or the state of the controllable physical system .

22. The DT of claim 21 , configured to use the behavior model and the emulated variations in the condition to predict an anomalous state and to use the decision model and the emulated variations in the state to predict an anomalous action.

23. The DT of claim 22, configured to update the behavior model or the decision model in response to predicting an anomalous state or action.

24. The DT of any one of claims 21 to 23, configured to use the emulator of any one of claims 15 to 19 to emulate time-series data corresponding to variations in actions to be applied to the controllable physical system.

25. The DT of claim 24, configured to monitor the controllable physical system to determine an anomalous condition in response to variations in actions.

26. The DT of any one of claims 21 to 25, wherein the behavior model and the decision model are implemented in a network edge and the emulating is implemented on a network cloud.

27. The DT of any one of claims 21 to 26, wherein the received time-series data depends on sensor measurements.

28. The DT of any one of claims 21 to 27 wherein the controllable physical system comprises sensors, actuators and physical components and is associated with one of the following: a communications network; a manufacturing site; a controllable urban environment.

29. A computer program comprising instructions which, when executed on a processor, cause the processor to carry out the method of any one of claims 1 to 14.

30. A computer program product comprising non-transitory computer readable media having stored thereon a computer program according to claim 29.