CN110795509B - Method and device for constructing index blood-margin relation graph of data warehouse and electronic equipment - Google Patents
Method and device for constructing index blood-margin relation graph of data warehouse and electronic equipment Download PDFInfo
- Publication number
- CN110795509B CN110795509B CN201910930860.8A CN201910930860A CN110795509B CN 110795509 B CN110795509 B CN 110795509B CN 201910930860 A CN201910930860 A CN 201910930860A CN 110795509 B CN110795509 B CN 110795509B
- Authority
- CN
- China
- Prior art keywords
- data
- business
- merging
- index
- tables
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 239000008280 blood Substances 0.000 claims abstract description 30
- 210000004369 blood Anatomy 0.000 claims abstract description 30
- 238000010586 diagram Methods 0.000 claims abstract description 25
- 238000012545 processing Methods 0.000 claims description 36
- 238000003860 storage Methods 0.000 claims description 27
- 230000000007 visual effect Effects 0.000 claims description 19
- 238000010276 construction Methods 0.000 claims description 17
- 230000003993 interaction Effects 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 11
- 230000000694 effects Effects 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 230000006399 behavior Effects 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims 2
- 230000009466 transformation Effects 0.000 claims 2
- 238000013523 data management Methods 0.000 abstract description 5
- 238000004590 computer program Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 4
- 238000012954 risk control Methods 0.000 description 4
- 230000006872 improvement Effects 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 239000002360 explosive Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000013058 risk prediction model Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/288—Entity relationship models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Pathology (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a device for constructing an index blood relationship diagram of a data warehouse and electronic equipment. The method comprises the following steps: acquiring a plurality of source data tables from a database server or a database server cluster, wherein the database server or the database server cluster provides data services for a plurality of financial business applications; merging the plurality of source data tables according to the business topics related to the plurality of financial business applications to generate a plurality of merged data tables distinguished by the business topics; providing indexes for each merging data table, and recording the data corresponding relation of index data from the source data table to the merging data table to form a data warehouse; and constructing an index blood relationship graph applied to the financial business based on the data corresponding relationship. The index blood relationship graph can provide reports quickly and accurately, or can access specific information in the database table quickly, so that large-scale tables and indexes can be supported more easily, and meanwhile, the data management and query performance is improved.
Description
Technical Field
The invention relates to the field of computer information processing, in particular to a method and a device for constructing an index blood-margin relation graph of a data warehouse and electronic equipment.
Background
With the development of electronic commerce, for example, a large number of users, commodities and production related data generated and accumulated in daily operations by internet companies show explosive growth, the data structure starts to be diversified, more information is contained in the data, and the company is paying more attention to the data operation. The data warehouse is used for carrying out sub-processing work on data and plays a great role. However, the data warehouse coming down in the big data age is slowly turned into a distributed architecture to meet the explosive growth of computing and storage requirements.
A Data repository (DW or DWH) is a strategic collection that provides all types of Data support for all levels of decision-making processes of an enterprise. It is a single data store created for analytical reporting and decision support purposes. To the business that needs business intelligence, provide and guide business process improvement, monitoring time, cost, quality and control. The basic architecture of the data warehouse mainly comprises the process of data inflow and outflow, and can be divided into source data, data warehouse and data application. Specifically, a variety of business data are stored in the data warehouse, with different business data being stored in different business tables.
In the related art, an ER diagram is usually made according to a service system, and when the existing BI (business intelligence) analyst performs service analysis, an ER table is required to be used, which is inconvenient. In addition, the BI analysts directly use data sources (a large number of data tables) to support business applications such as data mining or data analysis, and the data granularity is coarse, and because the source data is not integrated and classified, each BI analyst needs to understand the whole system to understand the relationship between different businesses, and the business data volume is large, so that the workload of data warehouse manager is large. Therefore, the learning cost is high. Furthermore, calculating a large amount of data is wasteful.
In summary, there is still much room for improvement in terms of improving data management and query performance.
Disclosure of Invention
In order to solve the problems, the invention provides a construction method of an index blood relationship graph of a data warehouse, which comprises the following steps: acquiring a plurality of source data tables from a database server or a database server cluster, wherein the database server or the database server cluster provides data services for a plurality of business applications; merging the plurality of source data tables according to the business topics related to the plurality of business applications to generate a plurality of merged data tables distinguished by the business topics; providing an index for each merging data table, and recording the data corresponding relation of index data from the source data table to the merging data table to form a data warehouse; and constructing an index blood relationship graph applied to the service based on the data corresponding relationship.
Preferably, the construction method further comprises: graphically displaying the index blood-margin relation graph on a visual interaction interface; and receiving user operation on the visual interaction interface, and inquiring data corresponding to the target inquiry information in the source data table or the merging data table according to the index blood-edge relation diagram.
Preferably, graphically displaying the index blood-edge relationship graph on the visual interaction interface includes: and displaying the index blood relationship graph through a graph with a hierarchical structure.
Preferably, the business topic includes events, wind control, transactions, product operations and recommendations, and the hierarchy includes a plurality of nodes corresponding to the business topic.
Preferably, the merging includes forming a plurality of merging data tables corresponding to the business topic by associating corresponding data with a user click ID, event ID, transaction ID, or activity ID as an index.
Preferably, the plurality of merging data tables are merging data tables of different dimensions.
Preferably, the plurality of merging data tables are mutually converted by a conversion rule.
In addition, the invention also provides a device for constructing the index blood relationship graph, which comprises the following steps: the system comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module acquires a plurality of source data tables from a database server or a database server cluster, and the database server or the database server cluster provides data services for a plurality of business applications; the first data processing module is used for merging the plurality of source data tables according to the business topics related to the plurality of financial business applications to generate a plurality of merged data tables distinguished by the business topics; the second data processing module is used for providing indexes for the merging data tables and recording the data corresponding relation of index data from the source data table to the merging data table to form a data warehouse; and the data construction module is used for constructing an index blood relationship graph applied to the service based on the data corresponding relationship.
Preferably, the building device further comprises: the data query module graphically displays the index blood-margin relation graph on a visual interaction interface; and receiving user operation on the visual interaction interface, and inquiring data corresponding to the target inquiry information in the source data table or the merging data table according to the index blood-edge relation diagram.
Preferably, graphically displaying the index blood-edge relationship graph on the visual interaction interface includes: and displaying the index blood relationship graph through a graph with a hierarchical structure.
Preferably, the business topic includes events, wind control, transactions, product operations and recommendations, and the hierarchy includes a plurality of nodes corresponding to the business topic.
Preferably, the merging includes forming a plurality of merging data tables corresponding to the business topic by associating corresponding data with a user click ID, event ID, transaction ID, or activity ID as an index.
Preferably, the plurality of merging data tables are merging data tables of different dimensions.
Preferably, the plurality of merging data tables are mutually converted by a conversion rule
In addition, the invention also provides electronic equipment, wherein the electronic equipment comprises: a processor; and a memory storing computer executable instructions that, when executed, cause the processor to perform the method of constructing an index blood relationship graph for a business according to the present invention.
In addition, the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores one or more programs, and the one or more programs, when executed by a processor, implement the method for constructing the index blood-edge relationship graph applied to the service.
Advantageous effects
Compared with the prior art, the data warehouse effectively integrates and classifies the existing data tables in enterprises according to the business subjects related to a plurality of financial business applications, and constructs the index data blood-edge relation graph with the data corresponding relation, so that a report can be provided rapidly and accurately, or specific information in the database table can be accessed rapidly; the index blood-edge relation graph makes the support of large-scale tables and indexes easier, and improves the data management and query performance; the merging data table corresponding to the business theme is used, an ER table is not needed, the workload of an analyst is reduced, and the learning cost is reduced.
Drawings
In order to make the technical problems solved by the present invention, the technical means adopted and the technical effects achieved more clear, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted, however, that the drawings described below are merely illustrative of exemplary embodiments of the present invention and that other embodiments of the present invention may be derived from these drawings by those skilled in the art without undue effort.
FIG. 1 is a flow chart of an example of a method of constructing an index blood relationship map of a data warehouse of the present invention.
FIG. 2 is a schematic block diagram of an example of a data warehouse of the present invention.
Fig. 3 is a schematic structural block diagram of another example of the data warehouse of embodiment 1 of the present invention.
Fig. 4 is a flowchart of another example of the construction method of the index blood-relationship diagram of the data warehouse of embodiment 1 of the present invention.
Fig. 5 is a schematic diagram of a query display interface of the index blood relationship diagram of embodiment 1 of the present invention.
FIG. 6 is a diagram of a query display interface of the index blood relationship graph of embodiment 1 of the present invention.
Fig. 7 is a schematic diagram of an example of a construction apparatus of an index blood relationship map of a data warehouse according to embodiment 2 of the present invention.
Fig. 8 is a schematic diagram of another example of the construction apparatus of the index blood relationship map of the data warehouse of embodiment 2 of the present invention.
Fig. 9 is a block diagram of an exemplary embodiment of an electronic device according to the present invention.
Fig. 10 is a block diagram of an exemplary embodiment of a computer readable medium according to the present invention.
Detailed Description
Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. The same reference numerals in the drawings denote the same or similar elements, components or portions, and thus a repetitive description thereof will be omitted.
The features, structures, characteristics or other details described in a particular embodiment do not exclude that may be combined in one or more other embodiments in a suitable manner, without departing from the technical idea of the invention.
In the description of specific embodiments, features, structures, characteristics, or other details described in the present invention are provided to enable one skilled in the art to fully understand the embodiments. However, it is not excluded that one skilled in the art may practice the present invention without one or more of the specific features, structures, characteristics, or other details.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various devices, elements, components or portions, this should not be limited by these terms. These words are used to distinguish one from the other. For example, a first device may also be referred to as a second device without departing from the spirit of the invention.
The term "and/or" and/or "includes all combinations of any of the associated listed items and one or more.
Example 1
Next, the method and apparatus for constructing an index blood-margin relationship map of a data warehouse of the present invention will be described with reference to fig. 1 to 4.
FIG. 1 is a schematic diagram of the flow of the method for constructing an index blood relationship diagram of a data warehouse according to the present invention. As shown in fig. 1, the data warehouse construction method of the present invention includes:
step S101, a plurality of data tables from a database server or a database server cluster are obtained, wherein the database server or the database server cluster provides data services for a plurality of financial business applications;
step S102, merging the source data tables according to the business topics related to the financial business applications to generate a plurality of merged data tables distinguished by the business topics;
step S103, providing indexes for the merging data tables, and recording the data correspondence relationship between index data from the source data table to the merging data table to form a data warehouse.
Step S104, based on the data corresponding relation, constructing an index blood relationship diagram applied to the financial business.
Next, a method of constructing a data warehouse of the present invention will be described taking a loan-type business system as an example.
In step S101, a plurality of data tables from a database server or a database server cluster is acquired, the database server or the database server cluster providing data services for a plurality of financial business applications. As shown in fig. 1, a business product, such as a loan product, a loan navigation, an insurance product, or a financial product, generates a large number of data tables in a database server or a database server cluster, in this embodiment, approximately 1000 data tables, and the 1000 data tables provide data support for financial business applications such as data analysis and data mining.
In the present invention, the 1000 data tables are used as a source data table, and the source data table includes raw data which is not processed by data and/or data which is processed by data. In addition, these data tables are stored in one or more databases.
Specifically, the data contained in the source data table includes business data and user behavior data.
The service data is data related to transactions, state circulation, users and the like generated in the service flow, is usually stored in a DB, is related to the service, and is designed by a service side, the data is not required to be excessively concerned and is collected according to actual needs.
In addition, the user behavior data is data generated in the process of interacting with the product, such as page browsing, clicking, staying, etc., by the user during the use of the product.
In this embodiment, the source data table is uniformly processed, so that data in the source data table is standardized, and a plurality of data tables for classification merging are formed. Preferably, the data in the source data table is normalized by data processing such as batch processing or stream processing.
Next, step S102 will be described. In step S102, a plurality of source data tables are merged according to the business topics related to the plurality of financial business applications, and a plurality of merged data tables distinguished by the business topics are generated. Therefore, the merging data table corresponding to the business theme is used, an ER table is not needed, the workload of an analyst is reduced, and the learning cost is reduced.
Preferably, each business product is divided into a plurality of business topics based on business process or user lifecycle. Specifically, in this embodiment, the business topics include events, wind controls, transactions, product operations, and recommendations, see in particular fig. 2 and 3. For example, events include line iterations, page clicks, etc.; wind control includes risk control of funds, etc.; the transaction comprises repayment, movable support and the like; the recommendation includes a short message sales, a telephone sales, etc.
However, it should be noted that the present invention is not limited to a specific division manner of the service theme. In different embodiments, the invention can divide specific business topics according to the division mode of enterprise business and the relevance of the usage data in the business. The aim of the division is that the business personnel can intuitively access the data table from understanding the business itself as an access point without the interference of excessive non-business factors.
Next, step S103 will be described. In step S103, an index is provided for each merging data table, and a data correspondence relationship between index data from the source data table to the merging data table is recorded to form a data warehouse.
As shown in fig. 2, in this embodiment, the merging is performed by associating corresponding data (i.e., data corresponding to different business topics) with a user click ID, event ID, transaction ID, or activity ID as an index to form a plurality of merging data tables. In other embodiments of the present invention, the present invention is not limited to the selection of specific index data, as long as the data of the same business topic can be concatenated as an index.
In this embodiment, the merging data table is 30 sheets, and the merging data table is generally a wide table. It should be noted that, in the relational database, the index is a separate, physical storage structure for sorting values of one or more columns in the database table, and the function of the index is equivalent to the directory of the book, and specific information in the database table can be accessed quickly by using the index.
The merging data table will be described below taking a transaction in a business topic as an example.
In this embodiment, the plurality of merging data tables are merging data tables of different dimensions. More specifically, the merged data table of different dimensions is obtained, for example, by performing an up-dimension process or a down-dimension process, or not performing a process, on the data in the source data table.
In addition, data in a plurality of merging data tables related to each business topic can be mutually converted by conversion rules. The data conversion is performed by, for example, a dimension reduction process, a dimension increase process, or the like.
In this embodiment, the categories of the plurality of merging data tables include at least one data table of a first data table, a second data table for dimension modeling, a third data table related to a business report, and a fourth data table applied to a data index of a financial business.
Specifically, for example, the first data table (Dwb) is a data table first data table including a large amount of index data. The second data table (Dw) is a data table including dimension values (such as age, sex, etc.) and attribute values (such as number of transactions, transaction amount, etc.). The third data table (Ads) is a data table in a database divided by a service main line, for example, a data table including a loan balance and the like. The fourth data table (St) is a data table including a white list, a VIP list, and the like.
Still further, the merge data table includes index data corresponding to the business topic, the index data being associated with the target query information. A plurality of merged data tables are stored in a database corresponding to the business topic.
In order to construct the index blood relationship graph applied to the financial service, the construction method of the invention further comprises recording processing of index data, specifically, recording the data corresponding relationship of the index data from the source data table to the merging data table (see fig. 5 in particular), so as to facilitate quick data query, improve the data management performance and provide the speed of accessing the database.
Next, step S104 will be described, and in step S104, an index blood-edge relationship map applied to the financial business is constructed based on the above-described data correspondence.
In this embodiment, as shown in fig. 6, the construction method of the present invention further includes graphically displaying the index blood-edge relationship graph on a visual interactive interface. And receiving user operation on the visual interaction interface, and inquiring data corresponding to target inquiry information in the source data table or the merging data table according to the index blood-edge relation diagram.
As can be seen from fig. 6, the index blood-edge relationship graph is graphically displayed on the visual interaction interface, preferably by a graph having a hierarchical structure, wherein the hierarchical structure includes a plurality of nodes corresponding to the business subjects.
In the present embodiment, the plurality of nodes represent nodes of each traffic topic partition, which are connection points between two adjacent data layers (a previous data layer and a next data layer), and there are two layers of nodes from the source data table to the target query information, but not limited thereto, and may include three or more layers. Further, each layer includes a plurality of nodes.
For example, the index data a corresponds to a in the source data table, a in the first data table of the merging data table, and a in the second data table 1 The corresponding data in the third data table is B A The corresponding data in the fourth data table is C A The target inquiry information corresponding to the index data a is the number of registered persons. When the user inputs the target registered person number, the user can inquire corresponding data in the first data table, the second data table and the third data table in the merging data table according to the index blood relationship diagram (A, A 1 、B A 、C A ) It is also possible to query the source data table for the corresponding data a, in other words, the data a, a 1 、B A 、C A Has a data correspondence relationship with data A, A 1 、B A 、C A Can be converted with each other. Thus, the index data blood-margin relation graph of the invention is constructed based on the data correspondence.
As can be seen from fig. 4, in other embodiments, the above construction method further includes step S105, and in step S105, the user inputs the target query information, and queries the data corresponding to the target query information in the source data table or the merging data table.
It should be noted that the procedure of the above construction method is only for explaining the present invention, and the order of steps is not particularly limited.
Compared with the prior art, the data warehouse effectively integrates and classifies the existing data tables in enterprises according to the business subjects related to a plurality of financial business applications, and constructs the index data blood-edge relation graph with the data corresponding relation, so that a report can be provided rapidly and accurately, or specific information in the database table can be accessed rapidly; the index blood-edge relation graph makes the support of large-scale tables and indexes easier, and improves the data management and query performance; the merging data table corresponding to the business theme is used, an ER table is not needed, the workload of an analyst is reduced, and the learning cost is reduced.
The foregoing is merely a preferred embodiment, and is not to be construed as limiting the invention. In other embodiments, the user risk control model may also be used to calculate the user's risk level, or as a prediction module in other risk prediction models, etc.
Those skilled in the art will appreciate that all or part of the steps implementing the above-described embodiments are implemented as a program (computer program) executed by a computer data processing apparatus. The above-described method provided by the present invention can be implemented when the computer program is executed. Moreover, the computer program may be stored in a computer readable storage medium, which may be a readable storage medium such as a magnetic disk, an optical disk, a ROM, a RAM, or a storage array composed of a plurality of storage media, for example, a magnetic disk or a tape storage array. The storage medium is not limited to a centralized storage, but may be a distributed storage, such as cloud storage based on cloud computing.
Embodiments of the data warehouse construction apparatus of the present invention are described below, which may be used to perform method embodiments of the present invention. Details described in the embodiments of the device according to the invention should be regarded as additions to the embodiments of the method described above; for details not disclosed in the embodiments of the device according to the invention, reference may be made to the above-described method embodiments.
Example 2
Referring to fig. 7 and 8, the present invention further provides a construction apparatus 500 for an index blood relationship map, the construction apparatus 500 comprising: a data acquisition module 501, configured to acquire a plurality of source data tables from a database server or a database server cluster, where the database server or the database server cluster provides data services for a plurality of financial service applications; the data processing module comprises a first data processing module 502 and a second data processing module 503, wherein the first data processing module 502 merges the plurality of source data tables according to the business topics related to the plurality of financial business applications to generate a plurality of merged data tables distinguished by the business topics; a second data processing module 503, configured to provide an index for each merging data table, record a data correspondence between index data from the source data table to the merging data table, and form a data warehouse; the data construction module 504 constructs an index blood relationship graph applied to the financial business based on the data correspondence.
Preferably, the index blood-margin relation graph is graphically displayed on a visual interaction interface.
As shown in fig. 8, the construction apparatus further includes a data query module 505, where the data query module 505 specifically receives a user operation through the visual interaction interface according to the index blood-edge relationship graph, in other words, inputs target query information on the visual interaction interface, and queries, according to the index blood-edge relationship graph, data corresponding to the target query information in the source data table or the merged data table.
Preferably, graphically displaying the index blood-edge relationship graph on the visual interaction interface includes: the index blood-edge relationship graph is shown through a graph with a hierarchical structure, and particularly, reference is made to fig. 5 and 6.
Preferably, the index blood relationship graph is displayed through a graph with a hierarchical structure.
Preferably, the business topic includes events, wind control, transactions, product operations and recommendations, and the hierarchy includes a plurality of nodes corresponding to the business topic.
Preferably, the merging includes forming a plurality of merging data tables corresponding to the business topic by associating corresponding data with a user click ID, event ID, transaction ID, or activity ID as an index.
Preferably, the plurality of merging data tables are merging data tables of different dimensions.
Preferably, the plurality of merging data tables are mutually converted by a conversion rule.
In example 2, the same parts as those in example 1 are omitted.
It will be appreciated by those skilled in the art that the modules in the embodiments of the apparatus described above may be distributed in an apparatus as described, or may be distributed in one or more apparatuses different from the embodiments described above with corresponding changes. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.
Example 3
The following describes an embodiment of an electronic device according to the present invention, which may be regarded as a specific physical implementation of the above-described embodiment of the method and apparatus according to the present invention. Details described in relation to the embodiments of the electronic device of the present invention should be considered as additions to the embodiments of the method or apparatus described above; for details not disclosed in the embodiments of the electronic device of the present invention, reference may be made to the above-described method or apparatus embodiments.
Fig. 9 is a block diagram of an exemplary embodiment of an electronic device according to the present invention. An electronic apparatus 200 according to the embodiment of the present invention is described below with reference to fig. 9. The electronic device 200 shown in fig. 9 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 9, the electronic device 200 is in the form of a general purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one memory unit 220, a bus 230 connecting the different system components (including the memory unit 220 and the processing unit 210), a display unit 240, and the like.
Wherein the storage unit stores program code that is executable by the processing unit 210 such that the processing unit 210 performs the steps according to various exemplary embodiments of the present invention described in the electronic prescription stream processing method section above in this specification. For example, the processing unit 210 may perform the steps shown in fig. 1.
The memory unit 220 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 2201 and/or cache memory 2202, and may further include Read Only Memory (ROM) 2203.
The storage unit 220 may also include a program/utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 230 may be a bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 200 may also communicate with one or more external devices 300 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 200, and/or any device (e.g., router, modem, etc.) that enables the electronic device 200 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 250. Also, the electronic device 200 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through a network adapter 260. Network adapter 260 may communicate with other modules of electronic device 200 via bus 230. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 200, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the exemplary embodiments described herein may be implemented in software, or may be implemented in software in combination with necessary hardware. Thus, the technical solution according to the embodiments of the present invention may be embodied in the form of a software product, which may be stored in a computer readable storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, or a network device, etc.) to perform the above-mentioned method according to the present invention. The computer program, when executed by a data processing device, enables the computer readable medium to carry out the above-described method of the present invention, namely: and training the created user risk control model by using APP downloading sequence vector data and overdue information of the historical user as training data, and calculating a financial risk prediction value of the target user by using the created user risk control model.
As shown in fig. 10, the computer program may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
In summary, the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functionality of some or all of the components in accordance with embodiments of the present invention may be implemented in practice using a general purpose data processing device such as a microprocessor or Digital Signal Processor (DSP). The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
The above-described specific embodiments further describe the objects, technical solutions and advantageous effects of the present invention in detail, and it should be understood that the present invention is not inherently related to any particular computer, virtual device or electronic apparatus, and various general-purpose devices may also implement the present invention. The foregoing description of the embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (14)
1. The method for constructing the index blood relationship diagram of the data warehouse is characterized by comprising the following steps of:
acquiring a plurality of source data tables from a database server or a database server cluster, wherein the database server or the database server cluster provides data services for a plurality of business applications, and the source data tables comprise business data and user behavior data;
dividing each business product into a plurality of business topics based on business processes related to business applications or user life cycles, or generating a plurality of business topics according to division modes of businesses and relevance division of use data in the businesses, and generating a plurality of merging data tables with different dimensionalities distinguished by the business topics by carrying out data dimension increasing processing, dimension decreasing processing or unprocessed merging on the plurality of source data tables according to the business topics related to the business applications;
the merging data table is a wide table, data in a plurality of merging data tables related to each service theme are mutually converted through conversion rules, the merging data table comprises index data corresponding to the service theme and is associated with target query information, and the merging data tables are stored in databases of the corresponding service themes;
the category of the merging data tables comprises at least one of a first data table, a second data table for dimension modeling, a third data table related to a business report and a fourth data table referencing data indexes of business;
providing indexes for a plurality of merging data tables by connecting data of the same service subject in series so as to quickly access specific information in the merging data tables of the database, and recording data corresponding relations of index data from the source data tables to the merging data tables to form a data warehouse;
and constructing an index blood relationship graph applied to the service based on the data corresponding relationship.
2. The method as recited in claim 1, further comprising:
graphically displaying the index blood-margin relation graph on a visual interaction interface;
and receiving user operation on the visual interaction interface, and inquiring data corresponding to the target inquiry information in the source data table or the merging data table according to the index blood-edge relation diagram.
3. The method of claim 2, wherein graphically displaying the index blood relationship graph on a visual interactive interface comprises: and displaying the index blood relationship graph through a graph with a hierarchical structure.
4. The method of claim 3, wherein the business topic comprises events, wind control, transactions, product operations, and recommendations, and the hierarchy comprises a plurality of nodes corresponding to the business topic.
5. A method according to any one of claims 1-3, wherein indexing a plurality of said merged data tables by concatenating data of the same business topic comprises: the user click ID, event ID, transaction ID or activity ID is used as index to associate corresponding data.
6. A method according to any one of claims 1-3, wherein the inter-transforming data in a plurality of said merged data tables associated with respective business topics by means of transformation rules comprises: dimension reduction processing or dimension increase processing.
7. An apparatus for constructing an index blood relationship graph, comprising:
the system comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module acquires a plurality of source data tables from a database server or a database server cluster, the database server or the database server cluster provides data services for a plurality of business applications, and the source data tables comprise business data and user behavior data;
the first data processing module divides each business product into a plurality of business topics based on business processes related to business applications or user life cycles, or generates a plurality of business topics according to division modes of the business and relevance division of use data in the business, and generates a plurality of merging data tables with different dimensionalities distinguished by the business topics by carrying out data dimension increasing processing, dimension decreasing processing or unprocessed merging on the plurality of source data tables according to the business topics related to the business applications;
the second data processing module is characterized in that the merging data table is a wide table, data in a plurality of merging data tables related to each business theme are mutually converted through conversion rules, the merging data table comprises index data corresponding to the business theme and is associated with target query information, and the merging data tables are stored in databases of the corresponding business themes; the category of the merging data tables comprises at least one of a first data table, a second data table for dimension modeling, a third data table related to a business report and a fourth data table referencing data indexes of business; providing indexes for a plurality of merging data tables by connecting data of the same service subject in series so as to quickly access specific information in the merging data tables of the database, and recording data corresponding relations of index data from the source data tables to the merging data tables to form a data warehouse;
and the data construction module is used for constructing an index blood relationship graph applied to the service based on the data corresponding relationship.
8. The apparatus as recited in claim 7, further comprising:
the data query module graphically displays the index blood-margin relation graph on a visual interaction interface; and receiving user operation on the visual interaction interface, and inquiring data corresponding to the target inquiry information in the source data table or the merging data table according to the index blood-edge relation diagram.
9. The apparatus of claim 8, wherein graphically displaying the index blood relationship graph on a visual interactive interface comprises: and displaying the index blood relationship graph through a graph with a hierarchical structure.
10. The apparatus of claim 9, wherein the business topic comprises events, wind control, transactions, product operations, and recommendations, and wherein the hierarchy comprises a plurality of nodes corresponding to the business topic.
11. The apparatus according to any one of claims 7-9, wherein indexing the plurality of merged data tables by concatenating data of the same business topic comprises: the corresponding data is associated by indexing with the user click ID, event ID, transaction ID, or activity ID.
12. The apparatus according to any one of claims 7-9, wherein the data in the plurality of merged data tables associated with respective business topics are mutually transformed by a transformation rule comprising: dimension reduction processing or dimension increase processing.
13. An electronic device, wherein the electronic device comprises:
a processor; the method comprises the steps of,
a memory storing computer executable instructions that, when executed, cause the processor to perform the method of any of claims 1-6.
14. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910930860.8A CN110795509B (en) | 2019-09-29 | 2019-09-29 | Method and device for constructing index blood-margin relation graph of data warehouse and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910930860.8A CN110795509B (en) | 2019-09-29 | 2019-09-29 | Method and device for constructing index blood-margin relation graph of data warehouse and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110795509A CN110795509A (en) | 2020-02-14 |
| CN110795509B true CN110795509B (en) | 2024-02-09 |
Family
ID=69440001
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910930860.8A Active CN110795509B (en) | 2019-09-29 | 2019-09-29 | Method and device for constructing index blood-margin relation graph of data warehouse and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110795509B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111400305B (en) * | 2020-02-20 | 2022-03-08 | 深圳市魔数智擎人工智能有限公司 | Characteristic engineering blood relationship based backtracking and visualization method |
| CN111427902B (en) * | 2020-03-24 | 2024-05-07 | 苏州亿歌网络科技有限公司 | Metadata management method, device, equipment and medium based on lightweight database |
| CN111651475B (en) * | 2020-08-07 | 2020-12-01 | 北京每日优鲜电子商务有限公司 | Information generation method and device, electronic equipment and computer readable medium |
| CN111949662A (en) * | 2020-08-13 | 2020-11-17 | 北京字节跳动网络技术有限公司 | Data display method and device, storage medium and electronic equipment |
| CN112818048A (en) * | 2021-01-28 | 2021-05-18 | 北京软通智慧城市科技有限公司 | Hierarchical construction method and device of data warehouse, electronic equipment and storage medium |
| CN112989151B (en) * | 2021-03-11 | 2024-05-14 | 北京锐安科技有限公司 | Data blood relationship display method and device, electronic equipment and storage medium |
| CN113298354B (en) * | 2021-04-28 | 2023-08-01 | 上海淇玥信息技术有限公司 | Automatic generation method and device of service derivative index and electronic equipment |
| CN113722289A (en) * | 2021-08-09 | 2021-11-30 | 杭萧钢构股份有限公司 | Method, device, electronic equipment and medium for constructing data service |
| CN113760866A (en) * | 2021-08-30 | 2021-12-07 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | Modeling assistance device and method |
| CN113760849B (en) * | 2021-11-10 | 2022-04-08 | 深圳市明源云科技有限公司 | Log processing method, system, electronic device and computer readable storage medium |
| CN115203277B (en) * | 2022-09-19 | 2023-01-06 | 北京必盈特信息技术有限公司 | Data decision method and device |
| CN116484084B (en) * | 2023-06-21 | 2023-11-17 | 广州信安数据有限公司 | Metadata blood-margin analysis method, medium and system based on application information mining |
| CN116932831B (en) * | 2023-09-14 | 2023-12-26 | 北京滴普科技有限公司 | Method and device for constructing data blood-lineage diagram |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902653A (en) * | 2014-02-28 | 2014-07-02 | 珠海多玩信息技术有限公司 | Method and device for creating data warehouse table blood relationship graph |
| CN108628894A (en) * | 2017-03-21 | 2018-10-09 | 阿里巴巴集团控股有限公司 | Data target querying method in data warehouse and device |
| CN108694195A (en) * | 2017-04-10 | 2018-10-23 | 腾讯科技（深圳）有限公司 | A kind of management method and system of Distributed Data Warehouse |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003150594A (en) * | 2001-11-12 | 2003-05-23 | Hitachi Ltd | Data warehouse system |
-
2019
- 2019-09-29 CN CN201910930860.8A patent/CN110795509B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902653A (en) * | 2014-02-28 | 2014-07-02 | 珠海多玩信息技术有限公司 | Method and device for creating data warehouse table blood relationship graph |
| CN108628894A (en) * | 2017-03-21 | 2018-10-09 | 阿里巴巴集团控股有限公司 | Data target querying method in data warehouse and device |
| CN108694195A (en) * | 2017-04-10 | 2018-10-23 | 腾讯科技（深圳）有限公司 | A kind of management method and system of Distributed Data Warehouse |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110795509A (en) | 2020-02-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110795509B (en) | Method and device for constructing index blood-margin relation graph of data warehouse and electronic equipment | |
| CN107918600B (en) | Report development system and method, storage medium and electronic equipment | |
| US9037579B2 (en) | Generating dynamic hierarchical facets from business intelligence artifacts | |
| US8065315B2 (en) | Solution search for software support | |
| US5446885A (en) | Event driven management information system with rule-based applications structure stored in a relational database | |
| Bjeladinovic | A fresh approach for hybrid SQL/NoSQL database design based on data structuredness | |
| CN110795478A (en) | Data warehouse updating method and device applied to financial business and electronic equipment | |
| CN110807016A (en) | Data warehouse construction method and device applied to financial business and electronic equipment | |
| US20120095956A1 (en) | Process driven business intelligence | |
| CN110766289A (en) | Dynamic wind control rule adjusting method and device and electronic equipment | |
| US11294915B2 (en) | Focused probabilistic entity resolution from multiple data sources | |
| US10140319B2 (en) | System for identifying anomalies by automatically generating and analyzing a structure | |
| US20180246951A1 (en) | Database-management system comprising virtual dynamic representations of taxonomic groups | |
| CN108875044B (en) | Contact searching method and device, storage medium and electronic equipment | |
| CN111125266A (en) | Data processing method, device, equipment and storage medium | |
| CN112949269A (en) | Method, system, equipment and storage medium for generating visual data analysis report | |
| CN117454278A (en) | Method and system for realizing digital rule engine of standard enterprise | |
| CN112463916A (en) | Computerized competitive analysis | |
| CN105474208A (en) | Document-based search with facet information | |
| US20200167347A1 (en) | Enhanced search construction and deployment | |
| CN116467291A (en) | Knowledge graph storage and search method and system | |
| CN115422202A (en) | Service model generation method, service data query method, device and equipment | |
| US20210286853A1 (en) | Platform, method, and system for a search en-gine of time series data | |
| CN115221337A (en) | Data weaving processing method and device, electronic equipment and readable storage medium | |
| KR100792322B1 (en) | Framework for Quality Control of DataBase |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |