RU2683690C1 - Method and system for automatic generation of a program code for an enterprise data warehouse - Google Patents

Abstract

FIELD: information technology.SUBSTANCE: invention relates to a method and system for automatic generating of a program code for an enterprise data warehouse. In the method, metadata are obtained that describe the configuration of data transformation mechanisms for loading them to the level of a detailed data storage layer and calculating data mart storefronts, data update template of the detailed data storage layer and data marts of the data storage are obtained, a program code for loading data into the detailed data storage layer is generated and for calculation the data marts based on the received metadata and the generated data update template, program code generated in the previous step is installed on the data storage medium to perform the downloads, metadata are reused to update the detailed data storage layer and data marts by making delta changes to the mentioned data update template in case of making changes to the problem statement or by switching data update templates in case of changing the relational database type to use the received data update template during the subsequent generation of the program code.EFFECT: technical result is to manage the update of a data warehouse.10 cl, 5 dwg, 1 tbl

Description

FIELD OF TECHNOLOGY

[1] This technical solution, in General, relates to the field of computer technology, and in particular to systems and methods for automatically generating program code for a corporate data warehouse.

BACKGROUND

[2] Analytical data warehouses are now becoming a necessary attribute for corporations working with large volumes of data and are used to identify and create new financial instruments, maintain existing ones, control and identify accounting problems, create new and correct working financial strategies.

[3] The data source for such systems is almost always heterogeneous systems serving both front-office systems that represent the corporation's face to customers and back-office applications that carry out financial accounting. In addition, very often storages also store middle office data to analyze their work. Thus, the data source for corporate storage is hundreds of systems that are constantly changing in accordance with the requirements of business development.

[4] Under such conditions, the creation of a clear uniform system for collecting data in a single model is a necessary requirement for obtaining relevant and reliable data coming from the source systems.

[5] Patent No. 6604110 B1, “Automated software code generation from a metadata-based repository”, patent holder: International Business Machines Corp, publication date: 10/31/2000 is known from the prior art. This technical solution discloses a method for providing executable program code for use in an enterprise data model (EDM) management application that transfers data from one or more data sources to an EDM application database for use in response to user commands.

[6] This invention only works interactively in response to user commands, which requires a certain amount of labor and affects the speed of work. Moreover, this invention provides only executable code, which significantly reduces the possibilities of a technical solution and does not provide scripts or metadata, based on which in execution mode these metadata will determine the execution script.

SUMMARY OF THE INVENTION

[7] This technical solution is aimed at eliminating the disadvantages inherent in existing solutions known from the prior art.

[8] The technical problem (or technical problem) in this technical solution is the implementation of automatic generation of program code for a corporate data warehouse.

[9] The technical result manifested in solving the above technical problem is to increase the stability of the algorithms of the detailed layer and data marts, as well as to reduce the number of incidents in the data warehouse.

[10] An additional technical result is to increase the efficiency of the data warehouse and reduce the time it takes to perform standard operations for analysts through the use of metadata and data update templates.

[11] Additionally, the speed of data migration to another platform is increased due to the reuse of metadata and data templates.

[12] The specified technical result is achieved through the implementation of a method for automatically generating program code for a corporate data warehouse, in which metadata is obtained that describes the configuration of data transformation mechanisms for loading them to the level of the detailed layer and calculation of data warehouse windows; forming at least one data update pattern for the detail layer and data store data storefronts; generating code for loading data into a detailed storage layer and calculating data marts based on the obtained metadata and the generated data update template; install the code generated in the previous step on the data warehouse environment to perform downloads; reuse metadata updates of the detailed layer and data marts.

[13] In some embodiments, metadata is stored in tables in a relational database.

[14] In some embodiments, metadata is a description of table structures, relationships between them, partition rules, description of data marts.

[15] In some embodiments, the metadata is constant and calculated.

[16] In some embodiments, the metadata is placed in the form of a string-like structure of objects and their primary keys in a database.

[17] In some embodiments, a data update pattern is described in an S2T file.

[18] In some embodiments, additionally, when generating program code, a work log is received that includes all macro input variables as well as a trace file.

[19] In some embodiments, reuse of the update metadata of the detail layer and data marts is performed by making delta changes to the S2T file.

[20] In some embodiments, reuse of the update metadata of the detail layer and data marts is accomplished by switching basic update patterns.

BRIEF DESCRIPTION OF THE DRAWINGS

[21] The features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

[22] In FIG. 1 shows an example implementation of a method for automatically generating program code for an enterprise data warehouse;

[23] In FIG. 2 shows the architecture of a corporate storage database storage area;

[24] In FIG. Figure 3 schematically shows the logical structure and ETL processes of constructing a single semantic layer;

[25] In FIG. 4 shows a circuit for calculating surrogate keys;

[26] In FIG. Figure 5 shows the full path for automatic code generation.

DETAILED DESCRIPTION OF THE INVENTION

[27] This technical solution can be implemented on a computer, in the form of an automated system (AS) or a machine-readable medium containing instructions for performing the above method.

[28] The technical solution can be implemented as a distributed computer system.

[29] In this solution, a system means a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems, and any other devices that can perform a given, well-defined sequence of computing operations (actions, instructions).

[30] A command processing device is understood to mean an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs).

[31] An instruction processing device reads and executes machine instructions (programs) from one or more data storage devices. Data storage devices may include, but are not limited to, hard disks (HDDs), flash memory, ROM (read only memory), solid state drives (SSDs), and optical drives.

[32] A program is a sequence of instructions for execution by a computer control device or an instruction processing device.

[33] The terms and concepts necessary for the implementation of this technical solution will be described below.

[34] ETL (from Eng. Extract, Transform, Load - “extraction, transformation, loading”) is one of the main processes in managing data warehouses, which includes: extracting data from external sources; transforming and refining them to fit the needs of the business model; and loading them into the data warehouse.

[35] S2T is an Excel file containing information about all data transformations of the subject area, which is subject to ETL development.

[36] Surrogate keys - a set of metadata that defines the rules for creating primary keys of a data warehouse, as well as the rules for converting from natural keys of source data to primary keys of a warehouse.

[37] A natural key is a set of attributes of an entity described by a record that uniquely identifies it (for example, a passport number for a person).

[38] A detailed layer is a set of tables of a specific physical area of a data warehouse, which is the final layer for ETL processes and which contain all the business data of the data warehouse.

[39] Metadata - service information of the solution that describes the settings of the mechanisms and the results of the operation of the data integration subsystems, data warehouse and analytical application (for example, information about downloaded files).

[40] The ETL framework is server-based software that manages and downloads data from external systems to a data warehouse.

[41] The semantic layer is a set of physical and virtual data marts that provide FSLDM model data in a transformed form in accordance with the purpose of a particular storefront.

[42] Data showcase (Data Mart; other translation options: specialized data warehouse, data kiosk, data market) - a data warehouse slice, which is an array of thematic, narrowly focused information, oriented, for example, to users of one working group or department.

[43] OLAP is a data processing technology consisting in the preparation of summary (aggregated) information based on large data arrays structured according to a multidimensional principle. Implementations of OLAP technology are components of business intelligence class software solutions.

/ - заплатка) - информация, предназначенная для автоматизированного внесения определенных изменений в компьютерные файлы и базы данных.[44] Patch

/ - patch) - information intended for the automated introduction of certain changes to computer files and databases.

[45] The method for automatically generating program code for an enterprise data warehouse, shown in FIG. 1 is carried out as follows.

[46] Step 101: obtain metadata describing the setup of data transformation mechanisms for loading them onto the level of the detailed layer and calculation of data store windows;

[47] In the corporate data warehouse 200 shown in FIG. 2, a clear system of stages of loading and converting data from external automated systems is fixed, accompanied by a unified logging of all stages, dynamic and static parallelization, and the possibility of recovery in the event of a failure from any download location.

[48] Metadata is an important part of the data warehouse architecture 200. Metadata is data describing the rules by which the warehouse operates. For example, from the point of view of the warehouse database, metadata is a description of table structures, relationships between them, partition rules, description of data marts, etc. From the point of view of ETL, metadata is a description of the rules for extracting and transforming data, the frequency of execution of ETL processes, etc.

[49] In some embodiments, corporate storage 200 metadata includes:

- information about the data in the storage, their business description and storage structure;

- a description of the structures of data sources, both external and internal, their availability;

- information on the structure of ETL processes, the frequency of their implementation, the applicable rules for cleaning and transforming data;

- a description of the data presentation that helps the user to work with the B1 application;

- information about security settings, authentication rules and assigned access rights;

- statistics on the utilization of resources, data access, etc., which helps the administrator optimize the operation of the warehouse database.

[50] In some embodiments, metadata management is performed by separate tools for each of the storage components. For example, for an Oracle database whose metadata is stored in system tables and configuration files, this tool is Oracle Enterprise Manager.

[51] In some embodiments, metadata may be stored in special PostgreSQL tables.

[52] Attributes can be used as metadata, which are parameters that allow managing the ETL process. In some embodiments, metadata, and therefore attributes, can be constant and calculated. The values of constant attributes do not depend on the results of previous data loads into the storage. The calculated attributes are variable values that depend on the results of previous ETL downloads. Attribute values can be entered into ETL structures via the web interface. Such attributes can be the loading mode (for example, archived, incremental), the version of the loading specification, the serial number of the incremental loading, the lower unloading limit for each loaded entity, etc.

[53] Statistics can be used as metadata, which are values stored during the ETL process that can be used to calculate calculated attributes in subsequent processes. Statistics values can be stored in tables using the stored procedures of a relational database. As a relational DBMS can be used such as MySQL, PostgreSQL or Oracle, but not limited to.

[54] In some embodiments of the invention, the metadata is placed in the form of a string structure of objects and their primary keys in a database. This data can be presented either in the form of a tabular presentation and / or in the form of Excel spreadsheets, and / or in any relational database.

[55] Metadata is obtained from the primary external data systems 230 and loaded into the data temporary storage area 210 (FIG. 2).

[56] The temporary storage area 210 is an intermediate layer between external data sources and a permanent storage area in which the detailed layer and data marts are located. The data extracted from external data sources (DBMS, csv, dbf, xml files, web services, etc.) are stored in this area, they are cleaned, transformed, enriched, and prepared for loading into the area 220 for permanent storage. Often, the next cycle of processing and loading data into the storage cannot be started until all the necessary data has been extracted from various external data sources, and for a number of reasons (geographic distribution, different cycles of system functioning, etc.), the data in the sources can be available at different times. The temporary storage area 210 serves to collect all the necessary data before starting the transformation.

[57] Step 102: generating at least one data update template for the detail layer and data store data storefront;

[58] The detail layer is the primary corporate data warehouse. In this area, transformed and cleared detailed data obtained from external systems 230 and data sources, and basic classifiers are stored. This area contains the following types of entities: directories and classifiers; Entities containing actual values entities describing relationships.

[59] Directories and classifiers may define the following information:

- participants in the main business processes - customers, suppliers, branches, services, products, etc .;

- basic directories - date and time, currency, countries, etc .;

- other directories - reflecting the needs of the business in the necessary data analytics, determining in the context of which directories it is necessary to analyze the actual data.

[60] Entities containing actual values are transactional data from external systems 230 and data sources. For example, information about customer payments made, invoices, transactions, etc.

[61] Entities containing relationships define relationships between all other entities. For example, the connection may be as follows: client-service-bank.

[62] Data marts are objects of storage of analytical information aimed at supporting specific business functions, specific departments of the company. At the database level, storefronts are usually implemented according to the “star” or “snowflake” scheme and contain data from the area of the detailed layer. Also, data marts can be implemented as a multidimensional OLAP cube. Data marts are the foundation for multi-dimensional data analysis (OLAP). In the corporate data warehouse 200, a detailed layer and data marts are located in the area 220 for permanent data storage (Fig. 2).

[63] Implementation of metadata transformations and their framing into stored procedures is implemented using data update templates that work like java run-time substitutions, similar to the Apache Jakarta Project, where this technique was used as the basis for the Model-View Controller template for web applications. Conversion rules can be stored in a layout view. This data can be presented in the form of a table view and / or in the form of Excel spreadsheets, and / or in any relational database.

[64] In some embodiments, the data update pattern is an SQL procedure pattern.

[65] This technical solution uses the strategy of updating the data of the detailed layer and data marts, which is a way of writing the original data set in the format of the detailed layer table to the detailed layer table, as a result of which the problem of building the history is solved.

[66] In contrast to the methods of formatting the source data in the format of the detailed layer (layer B → layer N), the data update template (layer N → layer T) is completely standard, and can be described in the S2T file by specifying the name of the update strategy and its parameters in the form of a data update template.

[67] In this technical solution, for example, the following data update patterns “data snapshot - data snapshot”, “data snapshot - data history”, “data history - data history” can be used. These strategies are implemented depending on the type of source data and requirements for the final data.

[68] The data refresh strategy “data snapshot - data snapshot” works as follows. Non-historical data from a data source is transferred to non-historical tables in a detailed layer of the data warehouse. This data update template can be used for surrogate key containers (contracts, customers, etc.), as well as for transactional data (financial transactions, transactions, etc.). If the source fields used to calculate the primary key of the detailed layer can be overwritten on the source, then emulation of the replication mode is supported. Replication is a process that refers to copying data from one source to another (or to many others) and vice versa. Moreover, this data update pattern can work with data accumulation and without data accumulation.

[69] The data refresh strategy “data snapshot - data history” works as follows. Data for which the change history is not kept at the source is stored in the detailed layer of the permanent data warehouse with the creation of a history. In this case, history is deleted from the detailed layer for later periods than the one that came in the current service pack. If the source fields used to calculate the primary key of the detailed layer can be overwritten on the source, then emulation of the replication mode is supported.

[70] The data updating strategy “data history - data history” works as follows. A complete replication of the data history maintained on the source with reference to the date / time of changing the attribute value (or with reference to date periods - dates / times of the beginning and end of the action) is performed to the detailed data warehouse layer. At the same time, to eliminate the problem of updating the "date of relevance" field on the source, emulation of the replication mode is supported. When loading source datasets with history on a date, gaps in the history of the detail layer are not allowed. Intersections in the history of the detailed layer are not allowed; truncation is subject to an earlier period of two intersecting. A story for a certain period can also be loaded with a “snapshot - snapshot” template, if there is no need to eliminate the intersection of periods of history. To do this, it is necessary to emulate the replication mode, and the key should include both the fields on the basis of which the primary key is calculated in the detailed layer, and the fields containing the beginning and end of the relevance period.

[71] The generation of a surrogate key can be carried out by maintaining and mapping the original sets of natural keys into a sequence of integer values or, as shown in FIG. 4 can be determined based on the value of the unique key of the entity according to the following formula:

[72] For entities with a unique key consisting of a single attribute.

[73] For entities with a composite unique key:

[74] where

[75] NNN - 3-character code of the source system;

[76] SHA1 - function to obtain the hash value by the SHA1 algorithm;

- колонки, входящие в уникальный ключ таблицы из исходной системы, если уникальный ключ представлен более чем одним полем, то используется конкатенированное значение всех полей входящих в ключ с разделителем

;[77]

- columns included in the unique key of the table from the source system, if the unique key is represented by more than one field, then the concatenated value of all fields included in the key with a separator is used

;

- операция конкатенации.[78]

- concatenation operation.

[79] Surrogate keys obtained according to this scheme can be in the form of a 160 bit hexadecimal code. In the future, such keys are conveniently used to bind non-unified entities.

[80] In some embodiments, the implementation uses the Apache Hive representation to calculate surrogate key values. Hive is an engine that turns SQL queries into chains of map-reduce tasks. The engine includes components such as Parser (parses incoming SQL queries), Optimizer (optimizes the query for greater efficiency), Planner (schedules tasks for execution), Executor (runs tasks on the MapReduce framework.

[81] Hive also requires a metadata repository. SQL involves working with objects such as a database, table, columns, rows, cells, etc. Since the data itself that Hive uses is stored simply in the form of files on hdfs, it is necessary somewhere to store the correspondence between Hive objects and real files.

[82] Step 103: generating code for loading data into a detailed storage layer and calculating data marts based on the obtained metadata and the generated data update template;

[83] In FIG. 5 shows the process of automatically generating program code, which consists of the following steps.

[84] 1) Obtaining SQL code based on a data update template representing at least one stored procedure from an S2T file;

[85] The S2T file may look like this:

[86] 2) encapsulation of this program code in an executable module (procedure), with standard input and output parameters, a system for logging and error handling and ways to restore the boot process in the event of a technological failure;

[87] 3) inserting a call to this procedure into a metadata script that is read by the framework in run-time mode and sequentially and / or simultaneously executes instructions from the template.

[88] Inside the procedure templates (Fig. 5), various “case” or “if else” implementation branches are implemented, depending on the launch flags. In turn, the launch flags are set by the general project flags, conversion stage flags, specific conversion flags, etc.

[89] The result of processing each template is the directly generated program working code. Additionally, as a result of the work of the template, a work log is received, which includes all input macro variables, as well as a trace file (trace file), which can contain all the values of variables, data structures and arrays (collections) used.

[90] Each transformation may contain either a name or a symbol for the patterns used.

[91] The input data for the procedure template can be: scalar variables, vector elements (arrays, lists, hash tables), as well as multidimensional elements (vector vectors). Elements can be either simple elements or complex objects (objects based on developed Java classes).

[92] Step 104: automatically install the program code generated in the previous step on the data warehouse environment to perform downloads;

[93] At this step, a patch is created that contains the generated program code in the format and structure of the release-configuration management standards that is installed in run-time mode on the development environment. A patch installation script with instructions for the administrator console may also be included in the patch. The patch, which is ready to be installed on the development environment of the corporate data warehouse, is compatible with the system for automatically distributing patches to stands.

[94] The objects and databases included in the patch are installed in the same order on the development environment. If errors occur, they are present in the event log and are visible to the user. The set of event logs contains a detailed analysis of possible errors and warnings. Objects of the same type are grouped in a directory by data type and are located inside the suffixes of the database names and their subsequent placement.

[95] Table 1 below shows a typical patch structure. Particular attention should be paid to SetupManual_000000.txt, which automatically creates a sequence of steps for installing a patch (distribution) on all environments.

[96] In the "Compile" = "true" mode, all patch objects are created on the development environment. That is, in the code generation execution mode, all objects that will be included in the patch are created on the development environment.

[97] Step 105: reuse the update metadata of the detail layer and data marts.

[98] When making changes to the statement of the problem, reuse the metadata of updating the detailed layer and data marts by introducing a delta of changes into the above S2T form.

[99] When changing the type of relational database, the metadata of the update of the detailed layer and data marts are reused to form a content scenario by switching the basic update patterns.

[100] Then reuse the generated program code and metadata. After implementing the first version of a patch (distribution) for a task, changes to this task almost always follow, caused by both new business requirements and changes caused by corrections of the first version. The result of these changes can be the creation of new objects, the removal of old objects, changing procedures, changing scripts.

[101] Two approaches can be used to implement these changes.

[102] The first approach is to make changes to the existing S2T file (structure) and recreate the entire patch, that is, implement a new patch (cumulative patch). After that, changes can be highlighted by the version control system (git) used to maintain the code.

[103] The second option is to make automatic markings in S2T bookmarks of the modified transformations, and only then they will be included in the patch. In the interface, the patch must be marked as incremental.

[104] Changes occur due to the needs of the process. These changes are made to the available metadata described in S2T and during generation, reuse of previously entered metadata is performed.

[105] Aspects of the present invention can also be implemented using a data processing device that is a computer or system (or tools such as a central / graphics processor or microprocessor) that reads and executes a program recorded on a storage device to perform the functions of the above Embodiment (s) and the method shown in FIG. 1, the steps of which are performed by a computer or device by, for example, reading and executing a program recorded on a memory device to perform the functions of the above-described embodiment (s) of implementation. To this end, the program is recorded on a computer, for example, through a network or from a recording medium of various types, serving as a storage device (for example, a computer-readable medium).

[106] The data processing device may have additional features or functionality. For example, the data processing device may also include additional data storage devices (removable and non-removable), such as, for example, magnetic disks, optical disks or tape. Storage devices may include volatile and non-volatile, removable and non-removable media implemented in any way or using any technology for storing information, such as machine-readable instructions, data structures, program modules or other data. A storage device, removable storage, and non-removable storage are examples of computer storage media. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact ROM a disc (CD-ROM), universal digital disks (DVDs) or other optical storage devices, magnetic tapes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store the desired information and to which the data processing device can access. The data processing device may also include an input device (s), such as a keyboard, mouse, pen, voice input device, touch input device, and so on. An output device (a), such as a display, speakers, printer, and the like, may also be included in the system.

[107] The data processing device comprises communication connections that allow the device to communicate with other computing devices, for example over a network. Networks include local area networks and wide area networks along with other large, scalable networks, including, but not limited to, corporate networks and extranets. Communication connection is an example of a communication environment. Typically, a communication medium can be implemented using computer-readable instructions, data structures, program modules or other data in a modulated information signal, such as a carrier wave, or in another transport mechanism, and includes any information delivery medium. The term "modulated information signal" means a signal, one or more of its characteristics are changed or set in such a way as to encode information in this signal. By way of example, but without limitation, communication media include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, radio frequency, infrared, and other wireless media. The term “machine-readable medium”, as used herein, includes both storage media and communication media. The process sequences described in this document may be performed using hardware, software, or a combination thereof. When processes are performed using software, the program in which the sequence of processes is recorded can be installed and can be executed in the memory of a computer built into specialized hardware, or the program can be installed and can be run on a general-purpose computer that can perform various processes .

[108] For example, a program may be pre-recorded on a recording medium such as a hard disk or ROM (read only memory). Alternatively, the program can be temporarily or permanently saved (recorded) on a removable recording medium such as a floppy disk, CD-ROM (compact disc, designed for playback only), MO (magneto-optical) disc, DVD (digital universal disc) , magnetic disk or semiconductor memory. Removable recording media may be distributed as so-called software sold through a retail network.

[109] The program may be installed from the removable recording medium described above to a computer, or may be transmitted by cable from the download site to a computer, or may be transferred to a computer via network data channels such as a LAN (local area network) or The Internet. The computer can receive the program transmitted in this way and can install it on a recording medium such as an internal hard drive.

[110] The processes described herein may be performed sequentially in time, as described, or may be performed in parallel or separately, depending on the processing characteristics of the device performing the processes, or as necessary. The system described in this document is a logical set of multiple devices and is not limited to the structure in which these devices are installed in one enclosure.

Claims (23)

1. A method for automatically generating program code for a corporate data warehouse, implemented using a processor, including the following steps: - receive metadata that describes the configuration of data transformation mechanisms for loading them onto the level of the detailed layer of the data warehouse and calculation of data store windows; - form at least one data update pattern of a detailed data warehouse layer and data store data storefront; - carry out the generation of program code for loading data into a detailed layer of the data warehouse and calculating data marts based on the received metadata and the generated data update template; - install the program code generated in the previous step on the data warehouse environment to perform downloads; - reuse metadata to update the detailed layer of the data warehouse and data marts by making delta changes to the above data update template in case of changes in the statement of the problem or by switching data update templates in case of changing the type of relational database to use the received data update template for subsequent code generation. 2. The method according to claim 1, characterized in that the metadata is stored in the tables of the relational database. 3. The method according to p. 1, characterized in that the metadata is a description of the structures of the tables, the relationships between them, the rules of sectioning, the description of the data marts. 4. The method according to p. 1, characterized in that the metadata is constant and calculated. 5. The method according to p. 1, characterized in that the metadata is placed in the form of a structure of objects and their primary keys in the database. 6. The method according to claim 1, characterized in that the data update pattern is described in the S2T file. 7. The method according to claim 1, characterized in that, additionally, when generating the program code, a work log is received that includes all input macro variables, as well as a trace file. 8. The method according to claim 1, characterized in that the metadata of the update of the detailed layer and data marts is reused by making delta changes to the S2T file. 9. The method according to claim 1, characterized in that the reuse of the update metadata of the detailed layer and data marts is carried out by switching the basic update templates. 10. A system for automatically generating program code for a corporate data warehouse, comprising: - at least one data processing device; - at least one data storage device; at least one program, where one or more programs are stored on one or more data storage devices and executed on one or more data processing devices, and one or more programs include the following steps: i. receive metadata describing the configuration of data transformation mechanisms for loading them onto the level of the detailed data warehouse layer and calculation of data store windows; ii. forming at least one data update pattern of the detailed data warehouse layer and data store data storefronts; iii. generating code for loading data into a detailed layer of the data warehouse and calculating data marts based on the received metadata and the generated data update template; iv. install the code generated in the previous step on the data warehouse environment to perform downloads; v. reuse metadata to update the detailed layer of the data warehouse and data marts by making delta changes to the above data update template in case of changes in the statement of the problem or by switching data update templates in case of changing the type of relational database to use the received data update template for subsequent generation program code.

Images