CN114880303A

CN114880303A - Business data output method, device, equipment, medium and product

Info

Publication number: CN114880303A
Application number: CN202210503352.3A
Authority: CN
Inventors: 洪子迪; 罗兰宇; 刘强; 洪子灏; 童话; 杨鑫博
Original assignee: CCB Finetech Co Ltd
Current assignee: CCB Finetech Co Ltd
Priority date: 2022-05-09
Filing date: 2022-05-09
Publication date: 2022-08-09

Abstract

The invention discloses a method, a device, equipment, a medium and a product for outputting service data. The invention relates to the technical field of big data. The method comprises the following steps: acquiring data of a data pasting layer of a data architecture; the data architecture comprises a data pasting layer, a data common layer and a data application layer; integrating data of the data pasting source layer through a pre-designed integration model on the data common layer to obtain integrated output data; the integration model is obtained based on the design of a data logic model and a data physical model; and processing the integrated output data according to the service requirements acquired in advance at the data application layer to generate service output data. The scheme solves the problems of inconsistent data caliber processing, dispersed data redundancy and the like, can fundamentally ensure the consistency of data, eliminates the data dispersion and the data redundancy, is favorable for promoting the integration and the co-construction and sharing of a market database, comprehensively improves the data value, and provides support for service expansion, operation decision and customer service.

Description

Business data output method, device, equipment, medium and product

Technical Field

The present invention relates to the field of big data technologies, and in particular, to a method, an apparatus, a device, a medium, and a product for outputting service data.

Background

In the current big data era, the informatization degree of a bank supervision and delivery system is not high, and certain trouble is caused to practitioners in both business and technical aspects.

On the business level, the change of a supervision report is inevitable because new requirements are frequently proposed by a supervision department outside, so that the corresponding caliber is required to be modified according to business rules for reporting; the internal data source systems are different, so that the transverse comparison between the cross-system application data is not facilitated, the application calibers are different, the application calibers are processed independently, and the conditions of the same report field and different processing rules occur. On the technical level, the processing paths of a plurality of reports are long, and the report processing efficiency is low due to excessive external dependence after multiple times of unnecessary processing and checking.

Disclosure of Invention

The invention provides a business data output method, a device, equipment, a medium and a product, which are used for solving the problems of inconsistent data caliber processing, dispersed data redundancy and the like, can fundamentally ensure the consistency of data, eliminate the data dispersion and the data redundancy, are favorable for promoting the integration and co-construction sharing of a market database, comprehensively improve the data value and provide support for business expansion, business decision and customer service.

According to an aspect of the present invention, there is provided a service data output method, including:

acquiring data of a data pasting layer of a data architecture; the data architecture comprises a data pasting layer, a data common layer and a data application layer;

integrating the data of the data pasting source layer through a pre-designed integration model on the data common layer to obtain integrated output data; the integration model is obtained based on the design of a data logic model and a data physical model;

and processing the integrated output data according to the service requirements acquired in advance at the data application layer to generate service output data.

According to another aspect of the present invention, there is provided a service data output apparatus, including:

the data pasting layer data acquisition module is used for acquiring data pasting layer data of a data framework; the data architecture comprises a data pasting layer, a data common layer and a data application layer;

the integrated output data generation module is used for integrating the data of the data pasting source layer through a pre-designed integration model on the data common layer to obtain integrated output data; the integration model is obtained based on the design of a data logic model and a data physical model;

and the service output data generation module is used for processing the integrated output data according to the service requirements acquired in advance at the data application layer to generate service output data.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the service data output method according to any of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the service data output method according to any one of the embodiments of the present invention when the computer instructions are executed.

According to another aspect of the present invention, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a business data output method as in any one of the embodiments of the present invention.

According to the technical scheme of the embodiment of the invention, the acquired data of the data pasting source layer are integrated through the integration model, and the integrated output data are processed according to the service requirement, so that the service output data are obtained. The technical scheme can solve the problems of inconsistent data caliber processing, dispersed data redundancy and the like, can fundamentally ensure the consistency of data, eliminate the data dispersion and the data redundancy, is favorable for promoting the integration and the co-construction sharing of a market database, comprehensively improves the data value, and provides support for business expansion, business decision and customer service.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1A is a flowchart of a service data output method according to an embodiment of the present invention;

FIG. 1B is a block diagram of a data architecture provided in accordance with an embodiment of the present invention;

fig. 2A is a schematic diagram illustrating division of a theme domain according to a second embodiment of the present invention;

FIG. 2B is a schematic diagram of a single granularity partition according to a second embodiment of the present invention;

FIG. 2C is a schematic diagram of the division of the combined granularity according to the second embodiment of the present invention;

fig. 3 is a schematic structural diagram of a service data output device according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of an electronic device implementing the service data output method according to the embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. According to the technical scheme, the data acquisition, storage, use, processing and the like meet relevant regulations of national laws and regulations.

Example one

Fig. 1A is a flowchart of a service data output method according to an embodiment of the present invention, where the embodiment is applicable to a service data output scenario, and the method may be executed by a service data output apparatus, where the apparatus may be implemented in a form of hardware and/or software, and the apparatus may be configured in an electronic device. As shown in fig. 1A, the method includes:

s110, acquiring data of a data pasting layer of a data architecture; the data architecture comprises a data pasting layer, a data common layer and a data application layer.

The scheme can be executed by a business data processing system, and the business data processing system can execute the processing task of the business data according to the data architecture of the data warehouse. The data architecture may include a data pasting layer, a data common layer, and a data application layer. Fig. 1B is a schematic structural diagram of a Data architecture according to an embodiment of the present invention, and as shown in fig. 1B, Data in a Data source layer (ODS) is consistent with a service system, and no Data processing is performed. The business data processing system can acquire pasting source data, real-time database data and the like from the data pasting layer.

S120, integrating the data of the data pasting source layer through a pre-designed integration model on the data common layer to obtain integrated output data; the integration model is obtained based on the design of a data logic model and a data physical model.

As shown in fig. 1B, the Common Data Model (CDM) layer may also be referred to as a Common Data Model layer, and may process Data of the Data paste layer. The data common layer can be further divided into sub-layers such as a common dimension layer, a detail granularity fact layer, a theme wide surface layer and a common summary granularity fact layer. The common dimension layer can establish enterprise consistency dimensions based on a dimension modeling idea. The risk that the data calculation aperture and the algorithm are not unified is reduced. The tables of the common dimension layer are also commonly referred to as logical dimension tables, and the dimension and dimension logical tables are generally in one-to-one correspondence. The fine granularity fact layer can be used for carrying out normalized code conversion, cleaning, uniform format, desensitization and the like on data without transverse integration. The theme wide surface layer can integrate various information in the detail granularity fact layer and output the theme wide surface. The output main body wide table can be oriented to business processes, information of different business processes is not redundantly constructed, and a foreign key form is adopted. The common summary granularity fact layer can use an analyzed subject object as a modeling drive, construct a summary index fact table of the common granularity based on the application of the upper layer and the index requirements of the product, and physically model by a wide-tabulation means. And constructing statistical indexes with standard naming and consistent calibers, providing public indexes for the upper layer, and establishing an aggregated broad table and a detailed fact table. Tables in the common summary granular fact layer may be generally referred to as summary logic tables for storing derived index data.

The business data processing system can unify data modeling modes in the data common layer, and the modeling modes can be based on a data logic model and a data physical model integration mode, so that a data integration model is generated. The consistency of data and caliber processing in the data public layer can be ensured by the integration model, and data dispersion and data redundancy are easily eliminated. The integration model can be formalized processes and products expressing data requirements, can be understood as abstract representation of services, and can also be understood as a data-defined process. The integration model may enable an organization to understand, manage, and use all of its data assets, such as the core concept customers, products, and employees of a business, etc., through the description of entities, attributes, relationships, etc. The data of the data pasting source layer is subjected to data processing of the integration model in the data common layer, and integrated output data can be obtained.

S130, processing the integrated output data according to the service requirements acquired in advance at the data application layer to generate service output data.

As shown in fig. 1B, the Data Application layer (ADS) may be customized and developed for business requirements, and is used to store personalized statistical index Data of Data products. And according to the acquired service requirement, the service data processing system can output service output data through the corresponding application interface. The business output data can be used for various data application scenarios, such as report presentation, data analysis, data mining, data query and the like.

In this embodiment, optionally, the design process of the integrated model includes:

acquiring demand data and change data of data levels in a market database; wherein the data hierarchy comprises a basic data layer, a detail derivative data layer and a summary data layer;

determining entities and/or attributes of a data logic model according to the demand data and the change data;

determining whether the entity is a newly added entity and/or determining whether the attribute is a newly added attribute;

if the entity is a new entity and/or the attribute is a new attribute, integrating the entity and/or the attribute into the data logic model according to a preset data logic model design mode;

generating a data physical model according to the data logical model and a preset mapping rule;

and determining an integration model of the mart database according to the data physical model of each data hierarchy.

It will be appreciated that the mart database in the data warehouse may include three levels of a base data layer, a detail derivative data layer, and a summary data layer. As shown in fig. 1B, the basic data layer may be a preparation area of a data warehouse, and provides basic model data for the detail derivative data layer, so as to reduce the influence on the business system. The detail derived data layer can process basic model data to generate detail derived data, provide source detail data for a summarized data layer, provide long-term sediment of service system detail data, and provide historical data support for expansion of future analysis type requirements. The summarized data layer can summarize detailed data and provide fine-grained data for the data application layer.

The business data processing system can acquire the demand data and the change data of each data layer, and determine the entity and/or the attribute of the data logic model according to the demand data and the change data. The entity can be an abstract or concrete thing managed and maintained by a service requirement, and the entity is derived from a service information requirement and is a service basic data concept for storing information. Specifically, the entities may include a basic entity, an attribute entity, a relationship entity, and the like. The attribute may be a specific description of an entity, the entity being described by a plurality of attributes, all attributes belonging to the entity being dependent on the entity. Such as preference, sex, etc. of the participating persons.

The business data processing system may determine whether the entities and/or attributes determined based on the demand data and the change data are new based on the entities and attributes in the current data logic model. If new entities and/or new attributes exist, the entities and/or the attributes can be integrated into the data logic model according to the design mode of the data logic model so as to ensure that the new entities and/or the attributes are consistent with the modeling mode of the existing entities and/or attributes.

According to the integrated data logic model, the business data processing system can generate a data physical model according to a certain mapping rule. Specifically, the mapping rule includes a storage policy; wherein the storage strategy comprises at least one of a zipper table format, a full-volume slice table format, and an incremental slice table format. The business data processing system can obtain an integrated model of the whole market database according to the data physical model of the data hierarchy.

The scheme can keep the modeling mode of the newly added entity and/or attribute consistent with the modeling mode of the original data logic model, so that the data logic model of the whole market database has a uniform modeling mode, the data link can be shortened, and the timeliness of service processing can be improved.

In a possible implementation, optionally, after determining whether the entity is a newly added entity and/or determining whether the attribute is a newly added attribute, the method further includes:

and if the entity is not the new entity and/or the attribute is not the new attribute, updating the change data into the data logic model.

The scheme can update the data logic model in time when no new entity and/or new attribute exists so as to ensure the accuracy and high efficiency of service data output.

On the basis of the above scheme, optionally, the data logic model design method includes:

determining a data analysis range according to the use data, the calculation data and the acquisition data in the data hierarchy;

determining data granularity and entities according to the data analysis range;

according to the data items in the entity, taking the preset attribute hooking principle as the entity hooking attribute;

and defining the attribute according to a preset attribute definition principle.

The data logic model mainly comprises the design of entities and attributes, wherein the design of the entities and the attributes can be realized through data granularity. The data hierarchy includes data such as usage data, calculation data, and acquisition data. The usage data may be input data of the data hierarchy, and the data hierarchy needs to complete a related processing task by using the input data. It is to be understood that the calculation data may be data participating in calculation, and the collected data may be collected full-volume data, real-time data, or the like. The business data processing system can determine the data analysis range according to the data.

Specifically, the determining a data analysis range according to the usage data, the calculation data, and the collected data in the data hierarchy includes:

according to the use data, the calculation data and the acquisition data in the data hierarchy, performing data division according to a pre-divided subject domain to obtain a data division result;

and determining the data analysis range of each topic domain according to the data division result.

The subject domain may include products, contracts, organizations, channels, events, locations, business directions, marketing, users, employees, and resource items, among others. The business data processing system can form clear knowledge on the whole data by combing and analyzing the business system and the data flow direction, then generalize and classify the whole business, arrange the whole business into various theme domains, define the range of the data theme domain, list the content which each theme domain should contain, define and describe each theme domain in relative detail, and finally determine the divided theme domains.

Based on the above scheme, optionally, the determining the data granularity and the entity according to the data analysis range includes:

and determining the data granularity according to the data analysis range of each topic domain and a preset data granularity definition principle, and determining an entity according to the data granularity.

Specifically, the data granularity definition principle may include analyzing data requirements, defining source data, defining granularity definition and classification, and the like. The data granularity classification can be classified according to a single granularity and a combined granularity, the single granularity can be determined by taking the data of a single subject field as an analysis range, and the combined granularity can be determined by taking the data of a plurality of subject fields as an analysis range. After the granularity is defined, the entity is actually determined, and the entity and the granularity are in one-to-one relation at the level of the logic model. The naming of the data entities is named according to an entity naming specification, and the definition of the entities needs to describe business meanings contained by the entities.

After the data granularity and the entity are determined, the granularity and all data items contained in the entity need to be analyzed in sequence in the next step, and the main work of the analysis comprises the following steps: whether attribute naming is standard or not, what is the source field of the attribute, what is the processing rule of the attribute, what is the attribute definition, and the like. The hooking attribute is to hook the attributes with the same granularity to the same granularity, and perform processing such as deduplication, merging, splitting and the like on the attributes.

After the attributes with the same granularity are attached to the same granularity and the same entity, the attributes in the entity need to be defined in detail one by one, and detailed information contained in the attributes is explicitly described. Defining attribute details may include specifying attribute names, determining value range ranges, specifying data sources, describing business rules, and the like.

The scheme can completely define the data logic model, and is beneficial to constructing the high-quality data logic model and further constructing a good integration model.

According to the technical scheme, the acquired data of the data pasting source layer are integrated through the integration model, and the integrated output data are processed according to business requirements, so that the business output data are obtained. The technical scheme can solve the problems of inconsistent data caliber processing, dispersed data redundancy and the like, can fundamentally ensure the consistency of data, eliminate the data dispersion and the data redundancy, is favorable for promoting the integration and the co-construction sharing of a market database, comprehensively improves the data value, and provides support for business expansion, business decision and customer service.

Example two

The embodiment is based on the above embodiment, and is directed to a specific example of a banking business data output scenario. In this embodiment, the following scheme may be adopted in the design process of the integrated model.

1. Design of data logic model

The data logic model in the integration model is an overview of data capability and business capability exposed and provided for a user, and is a data asset list which can directly support the independent number checking and using of related personnel. The design of the data logic model is carried out in a layer-by-layer refinement mode, and the main parts comprise: and (4) a subject domain, granularity division, attribute determination and output of a related data logic model. And then, carrying out hierarchical splitting according to the attributes, and dividing the hierarchical splitting into a basic data layer, a detail derivative data layer and a summarized data layer.

(1) The design idea is as follows:

<1> topic Domain

The design of the data theme domain model is that a business system and a data flow direction are sorted and analyzed, the overall data theme domain is clearly recognized, the overall business is summarized and classified, the data theme domain is sorted into the data theme domains, the range of the data theme domain is defined, the content contained in each data theme domain is listed, each data theme domain is defined and described in relative detail, and finally the divided data theme domains are determined. The design of the data theme domain model mainly carries out the design of a primary data theme domain framework model. Fig. 2A is a schematic diagram of partitioning the theme domain according to the second embodiment of the present invention.

As shown in FIG. 2A, the primary data topic Domain: and 11 primary data theme fields comprising products, contracts, organizations, channels, events, positions, business directions, marketing, users, staff and resource items.

Second and third level data topic fields: the secondary data topic domain is further refined based on the primary data topic domain, the tertiary data topic domain is further refined based on the secondary data topic domain, and meanwhile, important entities under the data topic domain are designed.

As shown in fig. 2A, assuming that a product is selected as a primary data subject field, the secondary data subject field may be composed of deposit, loan, agency service, etc.; the deposit is further selected as a secondary data subject field, and the tertiary data subject field can be composed of a public deposit, a private deposit and the like; finally, the public deposit is selected as a third-level data subject field, and specific entities such as a public current deposit account, a public periodic deposit account, a public debit card contract, a public goods deposit account statistical transaction and the like can be detailed.

<2> granularity of data

The design data granularity is the core work of the design of a data center platform model, and the main working steps comprise:

analyzing data requirements: analyzing the data requirement, and determining the data range contained in the data requirement;

secondly, source data are determined: analyzing the source data, and determining the data range contained in the source data;

defining granularity definition and classification: the data granularity is named and defined in a standard mode in sequence, and the granularity is classified according to the refinement degree of the data granularity and by combining a data source and an application scene, wherein the granularity classification comprises the following steps: single particle size, combined particle size. Fig. 2B is a schematic diagram of a single-granularity partition provided according to the second embodiment of the present invention, and fig. 2C is a schematic diagram of a combined-granularity partition provided according to the second embodiment of the present invention.

Taking the client as an example, performing single granularity division with the application scenario and the collected data as the analysis range may obtain the data granularity division result shown in fig. 2B. Taking the organization + the client as an example, the data granularity division result shown in fig. 2C can be obtained by performing the combined granularity division with the application scenario as the analysis range.

After the data granularity is determined, the granularity needs to be attached to the data domain; the data granularity can be named and defined by referring to the business meaning expressed by the business main key, but the data granularity naming is not the simple superposition of the main key, but the business meaning is expressed in a refining way.

Defining an entity: after the granularity is defined, the entity is actually determined, and the entity and the granularity are in one-to-one relation at the level of the logic model. Naming the data entities according to an entity naming specification, wherein the definition of the entities needs to describe business meanings contained by the entities;

clear data range: when designing an entity of a data model, it is necessary to explicitly describe the data ranges that the entity contains. The main work of the step is to determine the granularity and design the entity, and define and describe the entity in detail, and the information items to be designed, maintained and filled mainly comprise the following contents: primary data field, secondary data field, tertiary data field, data granularity, entity name, entity definition, data range, internal and external marks, registration information and the like.

<3> Property

Hanging connection attribute

After determining the granularity and the entity of the data, the next step needs to sequentially analyze all data items contained in the granularity and the entity, and the main work of the analysis includes: whether attribute naming is standard or not, what the source field of the attribute is, what the processing rule of the attribute is, what the attribute definition is and the like. The hooking attribute is to hook the attributes with the same granularity to the same granularity, and perform processing such as deduplication, merging, splitting and the like on the attributes.

In the scheme, the detailed steps of the hooking attributes can be as follows:

selecting the data granularity: selecting a certain data granularity determined in the model design process;

analyzing the data items: and analyzing all data items at the same granularity in sequence, wherein the analysis content comprises the following steps: attribute naming, attribute definition and attribute service caliber, and distinguishing whether the data items are synonymy with the same name, different synonymy names and different synonymies with the same name according to the attribute name, the attribute definition and the attribute processing caliber;

data item deduplication: carrying out duplicate removal processing on data items with the same name and the same meaning, and ensuring the uniqueness of the data;

merging the data items: merging the data items with different synonyms to ensure the consistency of the data;

splitting a data item: and splitting the data items with the same name and different synonyms to ensure the accuracy of the data.

② defining attributes

After the attributes with the same granularity are attached to the same granularity and the same entity, the attributes in the entity need to be defined in detail one by one, and detailed information contained in the attributes is explicitly described.

Defining attributes includes the following steps:

specification attribute naming: naming the attributes contained in the entity according to the attribute naming specification;

detailed description of the attributes: detailed description and definition are carried out on attributes, including business meanings and the like;

determining a value range: determining a value range of the attribute value;

clear data sources: defining source components, a source table and source field information of the attributes;

describing the business rules: clearly defining the information of the business rule, the business caliber and the like of the attribute.

The step is a process of defining the attributes of the model entity in detail, and the information items to be described comprise: whether primary key, whether foreign key, reference entity, internal and external flag, derived flag, attribution data area, attribute definition, source component, component source table Chinese name, component source table English name, component source field Chinese name, component source field English name, caliber description, statistical frequency, registration information, etc.

(2) Model hierarchy:

the logic model of the basic data layer covers various data of the financial industry. The model follows an enterprise-level data standard and comprises a source business table and a pre-connection table, wherein the pre-connection table is mainly used for organizing data with the same business and granularity in a wide table mode from the consideration of convenience of data processing and access, and is properly pre-connected with some key element information, so that repeated correlation calculation of subsequent use number is reduced.

The logical model architecture of the detail derivative layer remains substantially consistent with the underlying data layer, again following enterprise-level data-level standards, but may include intra-enterprise standards and regulatory agency data standards.

The logical model of the summary layer is to perform summary calculation with higher granularity on the basic data layer and the detail derivative layer, and the summary granularity should be consistent with the granularity of the application interface. It contains the primary dimensions of the primary data domain, such as organization, customer, and contract, etc.

(3) Design criteria

The model is complete: the model covers the application range of data, including various types of native and derivative data;

the definition is unique: the data granularity and the data item service definition, the service caliber and the data caliber in the model are clear and accurate and are unambiguous, and the condition that the same name is different from synonymy and different from synonymy needs to be avoided;

the particle size is finest: in the logical model, all the traffic data of the finest granularity needs to be contained. For example, investment financing services include funds, treasures, securities, etc., where each service should be modeled separately at the finest granularity. Under the condition that all fine-grained service data are combined with the input model, a summary granularity model after fine-grained combination can be considered to be established by combining with an actual data using scene;

the attribution is unique: each entity in the model needs to belong to a unique tertiary data domain and cannot belong to two different tertiary data domains;

model layout: in the data item ordering in the entity, the main service key and the external granularity key related to other granularity model tables should be placed on the data item at the front of the model entity; data items commonly used by the service are placed in front, so that the inquiry by a plurality of parties is facilitated; data items with similar business meanings should be stored in a connected mode; data items from the same data source should be uniformly put together, so that data processing, use and query are facilitated.

2. Design of data physical model

The data logical model is applicable to multiple database implementations, meaning that one data logical model corresponds to multiple data physical models. The main purpose of the data physical model design is to convert the data logical model into a concrete implementable data table and relation, optimize application design, optimize storage and improve data access efficiency. The method can be divided into the following table forms according to different storage strategies: a linked list, a full slice table, and an incremental slice table.

(1) Design idea

<1> zipper watch

A technical mode of data storage and processing can record historical information of data, and record all information of the data from the beginning to the time when all changes exist.

<2> full slice table

The full amount of data within the update period is recorded, and recording is required regardless of whether the data changes.

<3> incremental slice Table

And recording newly-added data in the updating period, and newly-adding new data generated in the period on the basis of the data in the original table.

(2) Model hierarchy

The market database can be divided into three layers, wherein the first layer is a basic data layer, the second layer is a detail derivative data layer, and the third layer is a summary data layer.

The basic data layer covers detailed data generated by each business component of an enterprise level, and data storage is mainly performed through algorithms such as a historical zipper, incremental slices and full slices. And key algorithmic processing such as customer merge, organization withdrawal, customer account translation, and transcoding is performed. The data storage period is at least 13 months of data according to the application use number.

The detail derived data layer covers some data items required by the application, the data granularity is consistent with the basic data, but the detail derived data layer cannot be obtained through direct mapping or simple code conversion of the data items of the basic data layer. Mainly comprises a historical zipper, an incremental slice, a full-scale slice table and the like. The data storage period is at least 13 months of data according to the application use number.

The summary data layer contains all summary data required by the application, the summary calculation with higher granularity is carried out on the basic data layer and the detail derivative data layer, the summary granularity is consistent with the granularity of the application interface, the detail is collected to the middle summary granularity between the application interfaces, and the common summary result is only fallen to the ground to avoid repeated calculation. Typically in full slice mode. The data storage period is at least 13 months of data according to the application use number.

The data application layer contains all interface data required by the application. The layer is obtained by simply splicing basic data, detail derivation and summarized data without complex data processing. Typically in full slice mode. The data storage period is required by the application usage.

(3) Design criteria

Naming specification of basic data layer: t66_ BXX _ table service meaning _ storage policy. XX is the primary topic domain number.

Specification of detail derived data layer naming: t66_ DXX _ table service meaning _ storage policy. XX is the primary topic domain number, which is consistent with the underlying data layer.

Summary data layer naming convention: t66_ S _ data master dimension _ table service name _ storage policy. The data primary dimension is the primary dimension of the smallest granularity in the summary dimensions of the table.

Application interface layer naming specification: t66_ AXX _ interface traffic meaning _ storage policy. XX is the application number, numbered starting with the number 01.

In the whole design process, the service is ensured to acquire required data in time; after the access request is submitted, the related query result can be quickly provided; considering a proper data organization form facing an application scene; data privacy protection is enhanced, and data use authorized according to user authority is supported.

3. Design of integrated model

(1) Identifying entities and attributes according to data requirements;

(2) identifying whether the existing asset or the newly added asset is needed;

(3) analyzing data influence or designing entities and attributes;

(4) the logic model is reintegrated, and a mapping rule is designed;

(5) and designing an integration model.

The integration model is important content in data architecture design, and good integration model design and definition can intuitively reflect the essence of business activity patterns. The integration model can also ensure that the data architecture provides comprehensive and consistent high-quality data for service requirements, and can provide analysis basis for dividing application system boundaries, defining reference relations between data applications, defining integration interfaces between application systems.

EXAMPLE III

Fig. 3 is a schematic structural diagram of a service data output device according to a third embodiment of the present invention. As shown in fig. 3, the apparatus includes:

a data pasting layer data obtaining module 310, configured to obtain data pasting layer data of the data architecture; the data architecture comprises a data pasting layer, a data common layer and a data application layer;

an integrated output data generating module 320, configured to integrate, at the data common layer, the data of the data source layer through a pre-designed integration model to obtain integrated output data; the integration model is obtained based on the design of a data logic model and a data physical model;

and a service output data generating module 330, configured to process, at the data application layer, the integrated output data according to a service requirement acquired in advance, so as to generate service output data.

In this embodiment, optionally, the apparatus further includes: an integrated model design module, the integrated model design module comprising:

the data acquisition unit is used for acquiring the demand data and the change data of the data hierarchy in the mart database; wherein the data hierarchy comprises a basic data layer, a detail derivative data layer and a summary data layer;

the entity and/or attribute determining unit is used for determining the entity and/or attribute of the data logic model according to the demand data and the change data;

a newly added determining unit, configured to determine whether the entity is a newly added entity, and/or determine whether the attribute is a newly added attribute;

the integration unit is used for integrating the entity and/or the attribute into the data logic model according to a preset data logic model design mode if the entity is a new entity and/or the attribute is a new attribute;

the data physical model generating unit is used for generating a data physical model according to the data logical model and a preset mapping rule;

and the integration model generation unit is used for determining an integration model of the mart database according to the data physical model of each data hierarchy.

In a possible implementation, optionally, the newly-increased determining unit is further configured to:

On the basis of the above scheme, optionally, the apparatus further includes a data logic model design module, where the data logic model design module includes:

the data analysis range determining unit is used for determining a data analysis range according to the use data, the calculation data and the acquisition data in the data hierarchy;

the data granularity and entity determining unit is used for determining the data granularity and the entity according to the data analysis range;

the attribute hooking unit is used for hooking the attributes of the entity according to the data items in the entity and a preset attribute hooking principle;

and the attribute definition unit is used for defining the attributes according to a preset attribute definition principle.

On the basis of the above scheme, optionally, the data analysis range determining unit is specifically configured to:

In this scheme, optionally, the data granularity and entity determining unit is specifically configured to:

In a preferred embodiment, optionally, the mapping rule includes a storage policy; wherein the storage strategy comprises at least one of a zipper table format, a full-volume slice table format, and an incremental slice table format.

The service data output device provided by the embodiment of the invention can execute the service data output method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example four

FIG. 4 shows a schematic block diagram of an electronic device 410 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 4, the electronic device 410 includes at least one processor 411, and a memory communicatively connected to the at least one processor 411, such as a Read Only Memory (ROM)412, a Random Access Memory (RAM)413, and the like, wherein the memory stores computer programs executable by the at least one processor, and the processor 411 may perform various appropriate actions and processes according to the computer programs stored in the Read Only Memory (ROM)412 or the computer programs loaded from the storage unit 418 into the Random Access Memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, ROM 412, and RAM 413 are connected to each other by a bus 414. An input/output (I/O) interface 415 is also connected to bus 414.

Various components in the electronic device 410 are connected to the I/O interface 415, including: an input unit 416 such as a keyboard, a mouse, or the like; an output unit 417 such as various types of displays, speakers, and the like; a storage unit 418, such as a magnetic disk, optical disk, or the like; and a communication unit 419 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 419 allows the electronic device 410 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

Processor 411 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 411 performs various methods and processes described above, such as a service data output method.

In some embodiments, the business data output method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and/or installed onto electronic device 410 via ROM 412 and/or communications unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the service data output method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured to perform the traffic data output method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

EXAMPLE five

An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for outputting service data provided in any embodiment of the present application is implemented.

Computer program product in implementing the computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for outputting service data, the method comprising:

and processing the integrated output data according to the service requirement acquired in advance at the data application layer to generate service output data.

2. The method of claim 1, wherein the design process of the integrated model comprises:

3. The method of claim 2, wherein after determining whether the entity is a newly added entity and/or determining whether the attribute is a newly added attribute, the method further comprises:

4. The method of claim 2, wherein the data logic model design comprises:

determining data granularity and entities according to the data analysis range;

5. The method of claim 4, wherein determining a data analysis scope from the usage data, the calculation data, and the collection data in the data hierarchy comprises:

6. The method of claim 5, wherein the determining data granularity and entities from the data analysis scope comprises:

7. The method of claim 2, wherein the mapping rule comprises a storage policy; wherein the storage strategy comprises at least one of a zipper table format, a full-volume slice table format, and an incremental slice table format.

8. A service data output apparatus, characterized in that the apparatus comprises:

9. An electronic device, characterized in that the electronic device comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the traffic data output method of any of claims 1-7.

10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions for causing a processor to implement the service data output method according to any one of claims 1 to 7 when executed.

11. A computer program product, characterized in that the computer program product comprises a computer program which, when being executed by a processor, implements a service data output method according to any one of claims 1-7.