Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/44—Arrangements for executing specific programs
  • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/90—Details of database functions independent of the retrieved data types
  • G06F16/903—Querying
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F30/00—Computer-aided design [CAD]
  • G06F30/20—Design optimisation, verification or simulation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2111/00—Details relating to CAD techniques
  • G06F2111/02—CAD in a network environment, e.g. collaborative CAD or distributed simulation

Definitions

• the present invention relates to the field of large-scale information processing and, in particular, to a system and a method for optimizing a simulation tool.
• Elasticsearch MC is a distributed, free and open search and analysis engine for all types of data, including textual, numerical, geospatial, structured and unstructured.
• Elasticsearch MC is based on Apache Lucene MC and was first released in 2010 by Elasticsearch NV (now known as Elastic).
• Elasticsearch MC is the central component of Elastic Stack MC , a set of free and open tools for ingestion, data enrichment, storage, analysis and visualization.
• Elastic Stack TM includes a collection of lightweight shipping agents known as Beats TM for sending data to Elasticsearch TM .
• Kibana TM is a free and open “front-end” application that sits on top of Elastic Stack TM , providing search and data visualization capabilities for data indexed in Elasticsearch TM .
• Kibana TM also serves as the user interface for monitoring, managing, and securing an Elastic Stack TM cluster, as well as the centralized hub for integrated solutions developed on the Elastic Stack TM .
• Kibana MC allows searching, displaying and visualizing indexed data in Elasticsearch TM and data analysis through the creation of bar charts, pie charts, tables, histograms and maps.
• a dashboard view combines these visual elements to then be shared through a browser to provide real-time analytical views into large volumes of data in support of use cases such as enterprise risk and decision analysis.
• R MC is a free software environment used for statistical calculation and graphics. It compiles and runs on a wide variety of UNIX TM , Windows TM and MacOS TM platforms. R MC provides a wide variety of statistical (linear and nonlinear modeling, classical statistical tests, time series analysis, classification, clustering, ...) and graphical techniques, and is highly extensible.
• Js MC is an open source, cross-platform, "back-end" JavaScript runtime environment that runs on an engine and executes JavaScript code outside of a web browser.
• MongoDB MC is a document-oriented cross-platform database program available in source. Classified as a NoSQL database program, MongoDB MC uses JSON MC type documents with optional schemas.
• Kubernetes MC Pods are the smallest and most basic deployable objects of Kubernetes MC . A Pod represents a single instance of a process running in a cluster.
• Patent US10803401 B2 (HAMMOND) is known in the field, as are patent applications US20210064346A1 (EXERTIER), US20180357047A1 (BROWN) and W02020136680A1 (SABHARWAL).
• the aim is to integrate a data simulation application interface into a dashboard delivered by a Kibana MC plugin.
• This additional capacity added to the ELK MC suite requires additional components: The Python MC simulation application interface (API) and the task queue system (Python MC , NodeJs MC , MongoDB MC ) and the integration a large amount of data for each simulation. For example, we can use more than 8 million data points to simulate a large company for 1 1 years each month.
• Kibana Server MC Node.Js MC and R-Engine MC or Python MC are in the same Kubernetes MC Pod. Due to the specificities of the R-Engine MC [1], we face complexity in communication (by dynamic command line) and in error returns between the Kibana MC API and R-Engine MC . Additionally for certain target environments, when R-Engine and Kibana-Server are in the same Kubernetes Pod, this introduces a vulnerability.
• a simulation method comprising steps of: receiving a first simulation request from an agent via an application interface (API) and creating a simulation task; verification by a first service if the simulation request is a main task, if yes, the first service creates sub-tasks by entity to carry out the simulation simultaneously and a calculation of the simulation; otherwise, if the task is a subtask, we calculate a simulation for a corresponding entity; indexing of the simulation in a second service; update of the simulation status by the first service; compiling a natural language generation result into the first service; polling the result of the agent simulation by the application interface (API) at regular intervals until the simulation is completed; request of the simulation result to the second service; and transmitting the simulation result to the application interface.
• API application interface
• the first service and second service are installed in a Kubernetes MC cluster.
• the application interface can be a Kibana MC plugin which is installed on a Pod of a Kubernetes MC cluster.
• the first service can include Python MC and the MongoDB MC client library in a scalable Pod on a Kubernetes MC cluster.
• the second service (28) may include Elasticsearch TM installed on a virtual machine cluster or a Kubernetes TM cluster.
• a simulation system comprising: an application interface (API) configured to receive a first simulation request from a user via an agent; a work queue installed in a Pod of a Kubernetes MC cluster to receive the first simulation request from the user to distribute a workload across different instances of a first service; and an analytical engine installed in the Kubernetes MC cluster Pod to receive simulation results and transmit them to the application interface (API).
• API application interface
• Figure 1 is a schematic diagram of a system representing a first prototype, according to a preferred embodiment of the present invention.
• Figure 2 is a schematic diagram showing an operating method of the first prototype, according to a preferred embodiment of the present invention.
• Figure 3 is a schematic diagram of a system representing a second prototype, according to a preferred embodiment of the present invention
• Figure 4 is a schematic diagram showing a system, according to a preferred embodiment of the present invention.
• Figure 5 is a schematic diagram showing an operating method, according to a preferred embodiment of the present invention.
• Figure 6 is a schematic diagram showing a system, according to a preferred embodiment of the present invention.
• aspects described herein may be implemented in the form of a device, a method or a computer program product (e.g. (e.g., non-transitory computer-readable media or memory having computer-executable instructions that, when executed, carry out the operations or steps).
• a computer program product e.g. (e.g., non-transitory computer-readable media or memory having computer-executable instructions that, when executed, carry out the operations or steps).
• these aspects can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.
• Such aspects may take the form of a computer program product stored by one or more storage media computer-readable devices having computer-readable program code, or instructions, embodied in or on the storage media.
• Any suitable computer-readable storage media may be used, including hard disks, CD-ROMs, optical storage devices, magnetic storage devices, and/or any combination thereof.
• Kibana Server MC Node.Js MC and R-Engine MC or Python MC and all other analytical engines are in the same Kubernetes MC Pod. Due to the specificities of the R-Engine MC [1], the inventors faced complexity in communication (by dynamic command line) and in error returns between the Kibana MC API and R-Engine MC . In addition, for certain target environments, when R-Engine MC and Kibana-Server MC are in the same Kubernetes MC Pod, this introduces a vulnerability.
• R-Engine MC is made to reside separately and the latter is invoked by REST API.
• the “Backend” infrastructure includes: Kibana MC for visualization; ElasticSearch MC for persistence; and a mechanism or means (PLUMBER MC ) to transform R functions into REST API and build the http server.
• the PLUMBER MC identifies and routes http requests to invoke the R-Engine MC services.
• a service invokes the REST API and with the return, updates ElasticSearch and constructs the response returned by the "Kibana MC server" API to the client to refresh its dashboard with the requested scenario.
• the clients of a dashboard do not only view it, they influence its visualization via scenario simulation.
• R-ENGINE MC will not give control back to the competing processes until the scenario has been constructed. This is particularly harmful for very short scenarios, which remain unnecessarily waiting for longer scenarios.
• PLUMBER MC is interleaved in the R MC scripts means that the two technological limits "single http" of PLUMBER MC and "single threaded" of R-ENGINE MC cannot be addressed separately.
• R-ENGINE MC scripts are made to collaborate with an ElasticSearch MC indexing engine. This allows a user to simulate and integrate scenarios to perform an analysis of results over several years.
• a preferred embodiment of the present invention uses several processes processing in parallel.
• the structure is transformed into a FIFO (First In First Out) list, each simulation in the pool which ends will fetch an http request from the FIFO list.
• the consumption of simulations is done by recursive call.
• Analysis of the logs for a test bench of several batches of 500 http requests, of random duration, with 2,8,16 and 32 instances of R-ENGINE MC showed that at all times the number of asynchronous tasks in course was equal to the number of R-ENGINE MC instances.
• This advancement is based on the notion of a promise pool, a notion that does not exist in Node.Js MC that we had to develop.
• This pool takes the form of an array of promises which recursively, by exploiting the promise chaining capacity, will search in a FIFO queue for the next request to execute.
• Kibana MC , Node.js MC and R-Engine MC are brought together on the same Kubernetes MC cluster in the architecture.
• Command line communication turned out to be non-generic, with not very robust error handling between Kibana and R-Engine. It is therefore preferable to reside the R-ENGINE MC in a different Kubernetes MC Pod and invoke its services by http REST call.
• PLUMBER MC builds the server that listens to REST calls and invokes functions from a script, via decorations in the R MC script.
• the CPU intensive nature of the scenarios is incompatible with the single threaded aspect of R-Engine MC .
• a first prototype involves several processes: Kibana MC , ElasticSearch MC , PLUMBER MC to build the server backed by the R-Engine MC .
• Kibana MC i.e. an API simulation 16
• a second http call i.e. a POST simulation 18
• This last server 20 invokes, via a simulation calculation request 22, the functions 24 of the R-Engine MC to carry out the simulation and returns the simulation data, i.e. the indexing of the simulation 26, which will be used for updating ElasticSearch MC in Elastic MC and the Kibana MC API to relay them to the user 30, 32.
• a second prototype involves several processes: Kibana MC , ElasticSearch MC , MongoDB MC to build the Kubernetes MC 11 Pod backed by the R-Engine MC with Python MC
• Kibana MC ElasticSearch MC
• MongoDB MC MongoDB MC to build the Kubernetes MC 11 Pod backed by the R-Engine MC with Python MC
• the diagram represents the interactions between the different actors (customers, services or data persistence) as well as the chronological sequence of the different events.
• the client or user 10 via the agent 14 starts a new simulation 12, a new task 34 is created by the simulation API 16.
• the Python MC service 38 will take the next unprocessed task 35 and execute it .
• the Python MC service checks by making a request 40 to the MongoDB MC service 36 if there is a new task. This check is part of a process that is repeated in block 37.
• the Python service 38 checks in block 39 the type of the task 40: if it is a main task type 42, the Python service 38 distributes the processing of the task by creating a subtask 44 per entity (an entity is a set of data on which a simulation can be applied consistently), otherwise the service performs the calculation of the simulation 46 for the entity.
• a simulation can be run in parallel depending on the number of Python MC service instances and the number of entities.
• the result of the simulation is indexed 48 in Elastic MC 28.
• the status of the main task is updated 50 and the result of natural language generation or “Natural Laguage Generation: NLG” 52 of each entity is compiled in order to produce an aggregated result on all the entities processed.
• the agent 14 requests the result of the simulation 54 from the simulation API 16 at regular intervals until the simulation is completed.
• the simulation API 16 requests the result of the simulation 56 from the second service 28, i.e. Elastic MC .
• the second service 28 then transmits the result of the simulation 58 to the simulation API 16.
• Kibana MC i.e. an API simulation 16 which accommodates a second http call, i.e. a new simulation task 34 , to server 36 generated by MongoDB MC .
• the second prototype is aimed at large companies, the use of the instances must be close to 100%. Not only to be able to scale or extend to more than two instances without loss of utilization rate, but also to avoid timeout problems linked to the use of listening on http ports, which is naturally limited in time.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Software Systems (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Databases & Information Systems (AREA)
• Evolutionary Computation (AREA)
• Geometry (AREA)
• Computational Linguistics (AREA)
• Computer Hardware Design (AREA)
• Data Mining & Analysis (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=86184327

Family Applications (1)

Country Status (3)

Families Citing this family (1)

Family Cites Families (1)

• 2022
  • 2022-03-14 WO PCT/CA2022/050377 patent/WO2023173193A1/fr not_active Ceased
  • 2022-03-14 CA CA3188740A patent/CA3188740C/fr active Active
  • 2022-03-14 EP EP22931271.5A patent/EP4494004A1/de active Pending

Also Published As

Similar Documents

Legal Events