CN113626438A - Data table management method and device, computer equipment and storage medium - Google Patents
Data table management method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113626438A CN113626438A CN202110925863.XA CN202110925863A CN113626438A CN 113626438 A CN113626438 A CN 113626438A CN 202110925863 A CN202110925863 A CN 202110925863A CN 113626438 A CN113626438 A CN 113626438A
- Authority
- CN
- China
- Prior art keywords
- data table
- target data
- information
- degree
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 238000007726 management method Methods 0.000 title description 23
- 239000008280 blood Substances 0.000 claims abstract description 91
- 210000004369 blood Anatomy 0.000 claims abstract description 91
- 238000012545 processing Methods 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000011144 upstream manufacturing Methods 0.000 claims description 75
- 238000004364 calculation method Methods 0.000 claims description 30
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000006378 damage Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Human Resources & Organizations (AREA)
- Entrepreneurship & Innovation (AREA)
- Databases & Information Systems (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Animal Behavior & Ethology (AREA)
- Software Systems (AREA)
- Game Theory and Decision Science (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application belongs to a method, a device, computer equipment and a storage medium for data table management, and belongs to the technical field of big data. In addition, the present application also relates to a blockchain technique, and the target data table can be stored in the blockchain. According to the technical scheme, the data table type is determined by generating the blood relation characteristic map, so that manual intervention can be greatly reduced, the processing efficiency is improved, and meanwhile, the risk of data leakage is reduced.
Description
Technical Field
The application belongs to the technical field of big data, and particularly relates to a method and a device for managing a data table, computer equipment and a storage medium.
Background
Data lifecycle management includes a full lifecycle management process for the generation, use, migration, cleaning, and destruction of data. The data scale of the production system can be effectively controlled through data life cycle management, and the data access efficiency is improved, so that the overall efficiency of system operation is improved, and enterprises are helped to obtain the maximum value at the lowest cost in each stage of data life.
At present, after the data activity condition is known, it can be judged when a data lifecycle management process needs to be initiated by a manual judgment mode according to the data activity condition, that is, usually, a data developer sets a data table type before data is on-line according to self understanding of the data type, sets a registered data retention period, and then performs data archiving or destruction according to the period. However, the data table type is set manually, which has the defects of low processing efficiency and low accuracy, and the data leakage is easily caused by manually setting the data table type, which makes it difficult to ensure the data security.
Disclosure of Invention
The embodiment of the application aims to provide a method, a device, computer equipment and a storage medium for managing a data table, so as to solve the technical problems that the existing scheme for artificially judging the type of the data table is low in processing efficiency and poor in accuracy, easily causes data leakage and is difficult to ensure data safety.
In order to solve the foregoing technical problem, an embodiment of the present application provides a method for managing a data table, which adopts the following technical solutions:
a method of data table management, comprising:
receiving a data table type identification instruction, and determining a target data table corresponding to the data table type identification instruction;
acquiring blood relationship information and access record information of a target data table;
converting the blood relationship information of the target data table into a blood relationship characteristic map by using a preset map processing frame;
obtaining the in-out degree information and the upstream and downstream table structures of a target data table based on the blood relationship characteristic map;
the type of the target data table is determined based on the in-out degree information, the upstream and downstream table structure, and the access record information.
Further, the step of obtaining the blood relationship information and the access record information of the target data table specifically includes:
reading an HQL statement related to a target data table;
and analyzing the read HQL sentences to obtain the blood relationship information and the access record information of the target data table.
Further, the preset graph processing frame is Spark graph x, and the step of converting the blood relationship information of the target data table into a blood relationship characteristic map by using the preset graph processing frame specifically includes:
generating an initial relation map of the target data table by utilizing Spark GraphX;
importing the target data table into an initial node of the initial relation graph to obtain a target node;
and filling the initial relation map based on the blood relationship information of the target data table to generate a blood relationship characteristic relation map of the target data table.
Further, the step of obtaining the entrance and exit degree information of the target data table based on the blood margin characteristic relationship map specifically includes:
performing graph calculation on the blood-related feature relationship graph based on a preset graph calculation engine to obtain a triple associated with a target node;
acquiring attribute information of a target node in the triple;
and obtaining the entrance and exit degree information of the target data table based on the attribute information of the target node in the triple.
Further, the triple includes a start node, an edge, and an end node, and the step of obtaining the entrance and exit degree information of the target data table based on the attribute information of the target node in the triple specifically includes:
counting the number of triples of the target node as an initial node to obtain the out-degree number of the target data table;
and counting the number of the triples of the target node as the termination node to obtain the input degree of the target data table.
Further, the step of obtaining the upstream and downstream table structures of the target data table based on the blood-related feature relationship map specifically includes:
determining an upstream table node and a downstream table node of the target data table based on the triple of the target node;
acquiring a triple of an upstream table node and a triple of a downstream table node;
an upstream table structure is determined based on the triplets of upstream table nodes, and a downstream table structure is determined based on the triplets of downstream table nodes.
Further, the entrance and exit degree information includes a homologous entrance and exit degree and a heterologous entrance and exit degree, the homologous entrance and exit degree includes a same-base entrance and exit degree and a heterologous-base entrance and exit degree, and the step of determining the type of the target data table based on the entrance and exit degree information, the upstream and downstream table structure and the access record information specifically includes:
when the heterogeneous access degree is greater than a first threshold value, the homogeneous access degree is equal to the first threshold value, and the table structure of the target data table is the same as that of the upstream table, the target data table is a direct source-attached table;
when the heterogeneous income number is a first threshold value, the same-base income number is equal to a second threshold value or the heterogeneous-base income number is equal to a second threshold value, and the table structure of the target data table is the same as the upstream table structure, the target data table is a secondary source-pasting table;
when the heterogeneous income degree is greater than a first threshold value, the sum of the same-base income degree and the heterogeneous-base income degree is greater than the first threshold value, the sum of the same-base export degree and the heterogeneous-base export degree is greater than the first threshold value, and the heterogeneous export degree is equal to the first threshold value, the target data table is a middle table;
when the sum of the same-warehouse entry degree and the different-warehouse entry degree is greater than a first threshold value, the same-warehouse exit degree is a first threshold value or the sum of the different-warehouse exit degree and the different-source exit degree is greater than a first threshold value, and access record information exists in a preset first time period, the target data table is a result table;
and when the different-base output degree is a first threshold value, the different-source output degree is a first threshold value, and no access record information exists in a preset second time period, the target data table is a temporary table.
In order to solve the foregoing technical problem, an embodiment of the present application further provides a device for managing a data table, where the following technical solutions are adopted:
an apparatus of data table management, comprising:
the instruction receiving module is used for receiving the data table type identification instruction and determining a target data table corresponding to the data table type identification instruction;
the information acquisition module is used for acquiring the blood relationship information and the access record information of the target data table;
the relationship conversion module is used for converting the blood relationship information of the target data table into a blood relationship characteristic map by using a preset map processing frame;
the atlas calculation module is used for acquiring the entrance and exit degree information and the upstream and downstream table structures of the target data table based on the blood relationship characteristic atlas;
and the type determining module is used for determining the type of the target data table based on the entrance and exit degree information, the upstream and downstream table structures and the access record information.
In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:
a computer device comprising a memory and a processor, the memory having stored therein computer readable instructions, the processor when executing the computer readable instructions implementing the steps of the method of data table management as described above.
In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:
a computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, implement the steps of the method of data table management as described above.
Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:
the application belongs to a data table management method, a data table management device, computer equipment and a storage medium, and belongs to the technical field of big data. According to the method and the device, the blood relationship characteristic relationship map of the target data table is used for obtaining the entrance and exit degree information and the upstream and downstream table structures of the target data table, and the type of the target data table is automatically identified by combining the access information of the target data table, so that the intelligence of the data life cycle strategy is determined, the manual intervention can be greatly reduced, the processing efficiency is improved, and meanwhile, the risk of data leakage is reduced.
Drawings
In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 illustrates an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 illustrates a flow diagram for one embodiment of a method of data table management according to the present application;
FIG. 3 illustrates a block diagram of one embodiment of an apparatus for data table management according to the present application;
FIG. 4 shows a schematic block diagram of one embodiment of a computer device according to the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that the method for managing the data table provided in the embodiment of the present application is generally executed by a server, and accordingly, the apparatus for managing the data table is generally disposed in the server.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The existing data sheet type confirmation is generally that a data developer sets the data sheet type before the data is on line according to self understanding of the data type, sets a registered data retention period, and then carries out data sheet filing or destruction according to the period. However, the data table type is set manually, which has the defects of low processing efficiency and low accuracy, and the data leakage is easily caused by manually setting the data table type, which makes it difficult to ensure the data security. The method and the device have the advantages that the blood relationship information of the target data table is converted into the blood relationship characteristic relationship map through the preset map processing frame Spark graph X, the entrance and exit degree information of the target data table and the upstream and downstream table structures of the target data table are obtained based on the blood relationship characteristic relationship map, and finally the entrance and exit degree information, the upstream and downstream table structures and the access information are combined, so that the server can automatically judge the type of the target data table.
With continued reference to FIG. 2, a flow diagram of one embodiment of a method of data table management is shown, in accordance with the present application. The data table management method comprises the following steps:
s201, receiving a data table type identification instruction, and determining a target data table corresponding to the data table type identification instruction.
In a specific embodiment of the present application, the target data table refers to a big data Hive table, and data processing of the Hive table is mainly implemented by an HQL (query language) statement.
Specifically, when the data table type identification requirement exists, the server receives a data table type identification instruction of the user, and determines a target data table corresponding to the data table type identification instruction based on the data table type identification instruction. The data table type identification instruction at least comprises cluster information of a target data table, database information and the name of the target data table, and the server searches the target data table according to the information.
In this embodiment, the electronic device (for example, the server/terminal device shown in fig. 1) on which the method for managing the data table operates may receive the data table type identification instruction through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a uwb (ultra wideband) connection, and other wireless connection means now known or developed in the future.
S202, obtaining the blood relationship information and the access record information of the target data table.
Each target data table has one or more associated data tables, and data in the associated data tables and data in the target data tables have a certain association relationship, for example, data in the target data tables are generated by data calculation of the associated data tables, and the target data tables and the associated data tables have a certain dependency relationship, that is, a blood relationship.
Specifically, the server obtains the blood relationship information and the access record information of the target data table by reading the HQL statement of the target data table. The blood relationship information and the access record information of the target data table can be obtained by reading and executing the HQL statement of the target data table.
S203, converting the blood relationship information of the target data table into a blood relationship characteristic map by using a preset map processing frame.
The preset graph processing framework is Spark graph X, the Spark graph X is a distributed graph processing framework based on Spark, and a simple, easy-to-use and rich interface for graph calculation and graph mining can be provided based on a Spark platform, so that the requirement for processing the distributed graph is greatly facilitated.
Specifically, the server converts the blood relationship information of the target data table into a blood relationship characteristic map for Spark graph x through a pre-arranged graph processing framework. The blood relationship information of the target data table is converted into the blood relationship characteristic relationship map through the preset map processing frame Spark graph X, and the entrance and exit degree information of the target data table and the upstream and downstream table structures of the target data table can be obtained based on the blood relationship characteristic relationship map.
And S204, acquiring the entrance and exit degree information and the upstream and downstream table structures of the target data table based on the blood-related characteristic relationship map.
Specifically, the server imports the blood relationship characteristic map into a map calculation engine for map calculation to generate a triple of the target data table, and obtains the in-out degree information and the upstream and downstream table structures of the target data table through the triple of the target data table. The triple is a combination of three elements as the name implies, the triple comprises an initial node, an edge and a termination node, each triple is an assembly in a blood-related feature relationship map, one blood-related feature relationship map is formed by a plurality of triples, and in the triple, the information transmission direction is transmitted from the initial node to the termination node through the edge. The graph calculation engine may use a Pregel engine to perform graph calculation on the blood-related feature relationship graph by using Pregel, so as to obtain a triple associated with a node where the target data table is located.
S205, the type of the target data table is determined based on the entrance and exit degree information, the upstream and downstream table structure and the access record information.
Specifically, after the server obtains the access degree information, the upstream and downstream table structures and the access record information, the server calls a target data table type judgment rule to judge the type of the target data table based on the access degree information, the upstream and downstream table structures and the access record information. The types of the data tables comprise a direct source pasting table, a secondary source pasting table, an intermediate table, a result table and a temporary table, wherein the temporary table is a data table which does not need to be used for a long time, the direct source pasting table is a data table which is synchronized from an online layer source database (such as pgsql) to hive, the secondary source pasting table is a data table which is restored by the direct source pasting table, the intermediate table is a data table which is specially used for storing intermediate calculation results in the database, and the result table is a data table used for supporting service application. The target data table type judgment rule is stored in a memory in advance by a developer according to business requirements, and the target data table type judgment rule is used for judging whether the number of the incoming and outgoing degrees of the target data table accords with a preset threshold value, whether the table structure of the target data table accords with the structures of an upstream table and a downstream table, whether an access record exists in a preset time period and the like.
In a specific embodiment of the present application, a specific target data table type determination rule is shown in table 1:
TABLE 1
Further, the entrance and exit degree information includes a homologous entrance and exit degree and a heterologous entrance and exit degree, the homologous entrance and exit degree includes a same-base entrance and exit degree and a heterologous-base entrance and exit degree, and the step of determining the type of the target data table based on the entrance and exit degree information, the upstream and downstream table structure and the access record information specifically includes:
when the heterogeneous access degree is greater than a first threshold value, the homogeneous access degree is equal to the first threshold value, and the table structure of the target data table is the same as that of the upstream table, the target data table is a direct source-attached table;
when the heterogeneous income number is a first threshold value, the same-base income number is equal to a second threshold value or the heterogeneous-base income number is equal to a second threshold value, and the table structure of the target data table is the same as the upstream table structure, the target data table is a secondary source-pasting table;
when the heterogeneous income degree is greater than a first threshold value, the sum of the same-base income degree and the heterogeneous-base income degree is greater than the first threshold value, the sum of the same-base export degree and the heterogeneous-base export degree is greater than the first threshold value, and the heterogeneous export degree is equal to the first threshold value, the target data table is a middle table;
when the sum of the same-warehouse entry degree and the different-warehouse entry degree is greater than a first threshold value, the same-warehouse exit degree is a first threshold value or the sum of the different-warehouse exit degree and the different-source exit degree is greater than a first threshold value, and access record information exists in a preset first time period, the target data table is a result table;
and when the different-base output degree is a first threshold value, the different-source output degree is a first threshold value, and no access record information exists in a preset second time period, the target data table is a temporary table.
In a triple, if the upstream table or the downstream table of the target data table and the target data table are not in the same data cluster, the access degree between them is called other data access degree, if the upstream table or the downstream table of the target data table and the target data table are in the same database from the same data cluster, the access degree between them is called same-base access degree, and if the upstream table or the downstream table of the target data table and the target data table are in different databases from the same data source, the access degree between them is called different-base access degree.
In the above-described embodiment, the first threshold is "0", the second threshold is "1", and "0" and "1" herein indicate the number of entries and exits, and for example, if the entry and exit information of one of the target data tables and the upstream and downstream table structures are used to obtain the entry of another data of the target data table as 2, the entry and exit numbers are both 0, and the table structure of the target data table matches the upstream table structure, the server determines that the target data table is the direct posting source table. The first time period is 90 days and the second time period is 180 days. If the access degree information, the upstream and downstream table structures and the access record information of a certain target data table do not meet any judgment rule, the prompt information of the error table type identification is output to prompt the user. In the above embodiment, the server can quickly determine the type of the target data table according to the access degree information, the upstream and downstream table structures, and the access record information of the target data table obtained in step S205 and the preset table type determination rule.
The application belongs to a data table management method, a data table management device, computer equipment and a storage medium, and belongs to the technical field of big data. According to the method and the device, the blood relationship characteristic relationship map of the target data table is used for obtaining the entrance and exit degree information and the upstream and downstream table structures of the target data table, and then the type of the target data table is automatically identified by combining the access information of the target data table, so that the life cycle strategy of the data table is determined by the type of the data table, the manual intervention can be greatly reduced, the processing efficiency is improved, and meanwhile, the risk of data leakage is reduced.
Further, the step of obtaining the blood relationship information and the access record information of the target data table specifically includes:
reading an HQL statement related to a target data table;
and analyzing the read HQL sentences to obtain the blood relationship information and the access record information of the target data table.
Specifically, data processing of the Hive table is mainly achieved through HQL sentences, the blood relationship information and the access record information of the target data table are stored through the HQL sentences, and the server can obtain the blood relationship information and the access record information of the target data table by reading and executing the HQL sentences of the target data table.
Further, the preset graph processing frame is Spark graph x, and the step of converting the blood relationship information of the target data table into a blood relationship characteristic map by using the preset graph processing frame specifically includes:
generating an initial relation map of the target data table by utilizing Spark GraphX;
importing the target data table into an initial node of the initial relation graph to obtain a target node;
and filling the initial relation map based on the blood relationship information of the target data table to generate a blood relationship characteristic relation map of the target data table.
The preset graph processing framework is Spark graph X, Spark is a quick, universal and extensible big data parallel computing framework, and is based on memory computing, so that the real-time performance of data processing in a big data environment is improved, high fault tolerance and high scalability are guaranteed, and a user is allowed to deploy Spark on a large amount of cheap hardware to form a cluster. Spark graph X is a Spark-based distributed graph processing framework, which can provide a simple, easy-to-use and rich interface for graph computation and graph mining based on Spark platform, thereby greatly facilitating the requirement for processing distributed graphs.
Specifically, the server generates an initial relationship graph of the target data table by using a graph processing frame Spark graph X, the initial relationship graph comprises a plurality of nodes, the target data table is imported into initial nodes of the initial relationship graph, the initial nodes are target nodes, other nodes in the initial relationship graph are respectively filled according to the blood relationship information, association nodes associated with the initial nodes are obtained, and the initial nodes and all the association nodes jointly generate the blood relationship characteristic graph of the target data table.
In the above embodiment, the blood relationship information is converted into the blood relationship characteristic relationship map through the map processing framework Spark graph x, which facilitates subsequent map calculation to obtain the triple of the target data table.
Further, the step of obtaining the entrance and exit degree information of the target data table based on the blood margin characteristic relationship map specifically includes:
performing graph calculation on the blood-related feature relationship graph based on a preset graph calculation engine to obtain a triple associated with a target node;
acquiring attribute information of a target node in the triple;
and obtaining the entrance and exit degree information of the target data table based on the attribute information of the target node in the triple.
Further, the triple includes a start node, an edge, and an end node, and the step of obtaining the entrance and exit degree information of the target data table based on the attribute information of the target node in the triple specifically includes:
counting the number of triples of the target node as an initial node to obtain the in-degree number of the target data table;
and counting the number of the triples of the target node as the termination node to obtain the output degree of the target data table.
The graph computation engine may use a Pregel engine, where Pregel is a graph computation engine, and may analyze the graph analysis process into several rounds of computation, where each vertex of each round independently executes a respective vertex program, and a state is synchronized between the vertices through message passing, where each round of iteration may also be referred to as a superstep (super step). The blood margin feature relation graph can be subjected to graph calculation by using the pregel, and the triples related to the nodes where the target data table is located are obtained.
Specifically, the server introduces the blood-related feature relationship graph into a preset graph calculation engine, performs graph calculation on the blood-related feature relationship graph through the graph calculation engine to obtain triples associated with target nodes, and then calculates the entry degree information by judging the target nodes in each triplet as starting nodes or terminating nodes, wherein the number of triples with the target nodes as the starting nodes is counted to obtain the number of the entries of the target data table, and the number of triples with the target nodes as the terminating nodes is counted to obtain the number of the entries of the target data table.
In the above embodiment, the graph calculation engine performs graph calculation on the blood-related feature relationship graph to obtain a triple of the target node, and then counts the degree of output of the target data table and the degree of input of the target data table based on the obtained triple to obtain the degree of output information of the target data table.
Further, the step of obtaining the upstream and downstream table structures of the target data table based on the blood-related feature relationship map specifically includes:
determining an upstream table node and a downstream table node of the target data table based on the triple of the target node;
acquiring a triple of an upstream table node and a triple of a downstream table node;
an upstream table structure is determined based on the triplets of upstream table nodes, and a downstream table structure is determined based on the triplets of downstream table nodes.
Specifically, in a triplet, when a target node is used as a start node of the triplet, a termination node of the triplet is a downstream table node of a target data table, when the target node is used as a termination node of the triplet, the start node of the triplet is an upstream table node of the target data table, an upstream table node and a downstream table node of the target data table are determined and determined by the above method, then a graph computation engine is used to obtain a triplet of each upstream table node and each downstream table node, an upstream table structure is determined based on the triplet of the upstream table node, and a downstream table structure is determined based on the triplet of the downstream table node.
In the above embodiment, the upstream table node and the downstream table node of the target data table are determined and determined according to the position of the target node in the triple, then the upstream table node and the downstream table node are respectively subjected to graph calculation to obtain the triple of the upstream table node and the downstream table node, the upstream table structure is determined based on the triple of the upstream table node, the downstream table structure is determined based on the triple of the downstream table node, and the target data table structure and the upstream table structure and the downstream table structure are generally compared in the subsequent process.
Further, after the step of determining the type of the target data table based on the entrance and exit degree information, the upstream and downstream table structures, and the access record information, the method further comprises the following steps:
acquiring a data table processing strategy matched with the type of a target data table from a preset processing strategy set;
and processing the target data table according to the matched data table processing strategy.
Specifically, a developer configures corresponding data table processing strategies for various types of data tables according to the processing requirements of each type table, the data table processing strategies are stored in a memory in advance, after the server judges the type of the target data table, the server calls the corresponding data table processing strategies in the memory according to the type of the target data table, and the target data table is processed according to the matched data table processing strategies.
In a specific embodiment of the present application, a specific data table processing strategy is shown in table 2:
TABLE 2
Wherein, the direct source pasting table and the result table are permanently reserved, the overdue automatic exit is realized, the secondary source pasting table and the intermediate table are reserved for 3 months by default, the overdue automatic deletion is realized, the temporary table is reserved for 1 month by default, and the overdue automatic deletion is realized. In the above-described embodiment, after determining the type of the target data table in step S205, the server automatically completes the subsequent processing of the target data table according to the preset data table processing policy.
It is emphasized that, to further ensure the privacy and security of the target data table, the target data table may also be stored in a node of a block chain.
The block chain referred by the application is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a computer readable storage medium, and when executed, can include processes of the embodiments of the methods described above. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of an apparatus for data table management, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 3, the apparatus for managing a data table according to this embodiment includes:
the instruction receiving module 301 is configured to receive a data table type identification instruction and determine a target data table corresponding to the data table type identification instruction;
an information obtaining module 302, configured to obtain information about a relationship between a blood vessel and an access record of a target data table;
the relationship conversion module 303 is configured to convert the blood relationship information of the target data table into a blood relationship characteristic map by using a preset map processing frame;
the map calculation module 304 is used for acquiring the in-out degree information and the upstream and downstream table structures of the target data table based on the blood-related characteristic relationship map;
a type determination module 305 for determining the type of the target data table based on the in-out information, the upstream and downstream table structure, and the access record information.
Further, the information obtaining module 302 specifically includes:
a sentence reading unit for reading an HQL sentence related to the target data table;
and the information acquisition unit is used for analyzing the read HQL sentences to obtain the blood relationship information and the access record information of the target data table.
Further, the preset graph processing framework is Spark graph x, and the relationship conversion module 303 specifically includes:
the map construction unit is used for generating an initial relation map of the target data table by utilizing Spark GraphX;
the node construction unit is used for importing the target data table into an initial node of the initial relation graph to obtain a target node;
and the relation filling unit is used for filling the initial relation map based on the blood relationship information of the target data table to generate a blood relationship characteristic relation map of the target data table.
Further, the atlas calculation module 304 specifically includes:
the graph calculation unit is used for carrying out graph calculation on the blood-related feature relation graph based on a preset graph calculation engine to obtain a triple associated with the target node;
the attribute information acquisition unit is used for acquiring the attribute information of the target node in the triple;
and the access degree information unit is used for obtaining access degree information of the target data table based on the attribute information of the target node in the triple.
Further, the triple includes a start node, an edge, and an end node, and the in-out degree information unit specifically includes:
the first counting subunit is used for counting the number of the triples of the target node as the starting node to obtain the out-degree number of the target data table;
and the second counting subunit is used for counting the number of the triples of the target node as the termination node to obtain the input number of the target data table.
Further, the atlas calculation module 304 also includes:
the map computing unit is used for determining an upstream table node and a downstream table node of the target data table based on the triple of the target node;
the triple acquiring unit is used for acquiring a triple of an upstream table node and a triple of a downstream table node;
and the table structure acquisition unit is used for determining the upstream table structure based on the triples of the upstream table nodes and determining the downstream table structure based on the triples of the downstream table nodes.
Further, the apparatus for data table management further comprises:
the strategy acquisition module is used for acquiring a data table processing strategy matched with the type of the target data table from a preset processing strategy set;
and the table processing module is used for processing the target data table according to the matched data table processing strategy.
The utility model belongs to a device of data table management, belong to big data technical field, this application converts the blood reason relation information of target data table into blood reason characteristic relation map through preset picture processing frame Spark graph X, and the upstream and downstream list structure of discrepancy degree information and target data table based on blood reason characteristic relation map obtains the target data table, combine discrepancy degree information, upstream and downstream list structure and access information at last, the type of automatic judgement target data table can be realized to the server. According to the method and the device, the blood relationship characteristic relationship map of the target data table is used for obtaining the entrance and exit degree information and the upstream and downstream table structures of the target data table, and the type of the target data table is automatically identified by combining the access information of the target data table, so that the intelligence of a data life cycle strategy is determined, the manual intervention is reduced, the processing efficiency is improved, and meanwhile, the risk of data leakage is reduced.
In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 4, fig. 4 is a block diagram of a basic structure of a computer device according to the present embodiment.
The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only computer device 4 having components 41-43 is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 41 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the computer device 4. Of course, the memory 41 may also include both internal and external storage devices of the computer device 4. In this embodiment, the memory 41 is generally used for storing an operating system installed in the computer device 4 and various types of application software, such as computer readable instructions of a method for managing a data table. Further, the memory 41 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 42 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions stored in the memory 41 or to process data, such as computer readable instructions for executing the method of data table management.
The network interface 43 may comprise a wireless network interface or a wired network interface, and the network interface 43 is generally used for establishing communication connection between the computer device 4 and other electronic devices.
The utility model belongs to a computer equipment, belong to big data technical field, this application converts the blood reason relation information of target data table into blood reason characteristic relation map through preset picture processing frame Spark graph X, and the trip list structure of discrepancy degree information and target data table based on blood reason characteristic relation map obtains target data table, combines discrepancy degree information, trip list structure and access information at last, and the type of automatic judgement target data table can be realized to the server. According to the method and the device, the blood relationship characteristic relationship map of the target data table is used for obtaining the entrance and exit degree information and the upstream and downstream table structures of the target data table, and the type of the target data table is automatically identified by combining the access information of the target data table, so that the intelligence of the data life cycle strategy is determined, the manual intervention can be greatly reduced, the processing efficiency is improved, and meanwhile, the risk of data leakage is reduced.
The present application provides yet another embodiment, which provides a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the method of data table management as described above.
The utility model belongs to a storage medium, belong to big data technical field, this application converts the blood reason relation information of target data table into blood reason characteristic relation map through preset picture processing frame Spark graph X to the discrepancy degree information and the upstream and downstream table structure of target data table are obtained based on blood reason characteristic relation map, combine discrepancy degree information, upstream and downstream table structure and access information at last, and the type of automatic judgement target data table can be realized to the server. According to the method and the device, the blood relationship characteristic relationship map of the target data table is used for obtaining the entrance and exit degree information and the upstream and downstream table structures of the target data table, and the type of the target data table is automatically identified by combining the access information of the target data table, so that the intelligence of the data life cycle strategy is determined, the manual intervention can be greatly reduced, the processing efficiency is improved, and meanwhile, the risk of data leakage is reduced.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.
Claims (10)
1. A method of data table management, comprising:
receiving a data table type identification instruction, and determining a target data table corresponding to the data table type identification instruction;
acquiring the blood relationship information and the access record information of the target data table;
converting the blood relationship information of the target data table into a blood relationship characteristic map by using a preset map processing frame;
acquiring the entrance and exit degree information and the upstream and downstream table structures of the target data table based on the blood relationship characteristic map;
determining a type of the target data table based on the in-out degree information, the upstream and downstream table structure, and the access record information.
2. The method of claim 1, wherein the step of obtaining the consanguinity information and the access record information of the target data table comprises:
reading an HQL statement related to the target data table;
and analyzing the read HQL statement to obtain the blood relationship information and the access record information of the target data table.
3. The method of data sheet management according to claim 1, wherein the preset graph processing framework is Spark graph x, and the step of converting the blood relationship information of the target data sheet into a blood relationship feature map by using the preset graph processing framework specifically includes:
generating an initial relation map of the target data table by using the Spark GraphX;
importing the target data table into an initial node of the initial relation graph to obtain a target node;
and filling the initial relation map based on the blood relationship information of the target data table to generate a blood relationship characteristic relation map of the target data table.
4. The method of claim 3, wherein the step of obtaining the entrance and exit information of the target data table based on the kindred feature relationship map comprises:
performing graph calculation on the blood-related feature relationship graph based on a preset graph calculation engine to obtain a triple associated with the target node;
acquiring attribute information of a target node in the triple;
and obtaining the entrance and exit degree information of the target data table based on the attribute information of the target node in the triple.
5. The method of claim 4, wherein the triplet includes a start node, an edge, and an end node, and the step of obtaining the in-out information of the target data table based on the attribute information of the target node in the triplet specifically includes:
counting the number of triples of the target node as an initial node to obtain the out-degree number of the target data table;
and counting the number of the triples of the target node as the termination node to obtain the in-degree number of the target data table.
6. The method for data sheet management as set forth in claim 4 wherein the step of obtaining upstream and downstream table structures of the target data sheet based on the kindred feature relationship map comprises:
determining an upstream table node and a downstream table node of the target data table based on the triples of the target node;
acquiring the triples of the upstream table nodes and the triples of the downstream table nodes;
an upstream table structure is determined based on the triplets of upstream table nodes, and a downstream table structure is determined based on the triplets of downstream table nodes.
7. The method of data table management according to any of claims 1 to 6, wherein the access information comprises a homologous access and a heterologous access, the homologous access comprises a homologous access and a heterologous access, and the step of determining the type of the target data table based on the access information, the upstream and downstream table structure and the access record information comprises:
when the heterogeneous access degree is greater than a first threshold value, the homogeneous access degree is equal to the first threshold value, and the table structure of the target data table is the same as that of the upstream table, the target data table is a direct source-attached table;
when the heterogeneous income number is a first threshold value, the same-base income number is equal to a second threshold value or the heterogeneous-base income number is equal to a second threshold value, and the table structure of the target data table is the same as the upstream table structure, the target data table is a secondary source-pasting table;
when the heterogeneous income degree is greater than a first threshold value, the sum of the same-base income degree and the heterogeneous-base income degree is greater than the first threshold value, the sum of the same-base export degree and the heterogeneous-base export degree is greater than the first threshold value, and the heterogeneous export degree is equal to the first threshold value, the target data table is a middle table;
when the sum of the same-warehouse entry degree and the different-warehouse entry degree is greater than a first threshold value, the same-warehouse exit degree is a first threshold value or the sum of the different-warehouse exit degree and the different-source exit degree is greater than a first threshold value, and access record information exists in a preset first time period, the target data table is a result table;
and when the different-base output degree is a first threshold value, the different-source output degree is a first threshold value, and no access record information exists in a preset second time period, the target data table is a temporary table.
8. An apparatus for data table management, comprising:
the instruction receiving module is used for receiving a data table type identification instruction and determining a target data table corresponding to the data table type identification instruction;
the information acquisition module is used for acquiring the blood relationship information and the access record information of the target data table;
the relationship conversion module is used for converting the blood relationship information of the target data table into a blood relationship characteristic map by using a preset map processing frame;
the atlas calculation module is used for acquiring the entrance and exit degree information and the upstream and downstream table structures of the target data table based on the blood relationship characteristic atlas;
a type determination module for determining the type of the target data table based on the access degree information, the upstream and downstream table structure and the access record information.
9. A computer device comprising a memory having computer readable instructions stored therein and a processor that when executed performs the steps of the method of data table management according to any one of claims 1 to 7.
10. A computer-readable storage medium, having computer-readable instructions stored thereon, which, when executed by a processor, implement the steps of a method of data table management according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110925863.XA CN113626438B (en) | 2021-08-12 | 2021-08-12 | Data table management method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110925863.XA CN113626438B (en) | 2021-08-12 | 2021-08-12 | Data table management method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113626438A true CN113626438A (en) | 2021-11-09 |
| CN113626438B CN113626438B (en) | 2024-04-30 |
Family
ID=78385049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110925863.XA Active CN113626438B (en) | 2021-08-12 | 2021-08-12 | Data table management method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626438B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114896584A (en) * | 2022-07-13 | 2022-08-12 | 杭州比智科技有限公司 | Hive data authority control agent layer method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739894A (en) * | 2019-01-04 | 2019-05-10 | 深圳前海微众银行股份有限公司 | Supplement method, apparatus, equipment and the storage medium of metadata description |
| CN112365244A (en) * | 2020-11-27 | 2021-02-12 | 深圳前海微众银行股份有限公司 | Data life cycle management method and device |
| CN112732925A (en) * | 2020-12-31 | 2021-04-30 | 平安科技（深圳）有限公司 | Method for determining investment data based on atlas, storage medium and related equipment |
-
2021
- 2021-08-12 CN CN202110925863.XA patent/CN113626438B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739894A (en) * | 2019-01-04 | 2019-05-10 | 深圳前海微众银行股份有限公司 | Supplement method, apparatus, equipment and the storage medium of metadata description |
| CN112365244A (en) * | 2020-11-27 | 2021-02-12 | 深圳前海微众银行股份有限公司 | Data life cycle management method and device |
| CN112732925A (en) * | 2020-12-31 | 2021-04-30 | 平安科技（深圳）有限公司 | Method for determining investment data based on atlas, storage medium and related equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114896584A (en) * | 2022-07-13 | 2022-08-12 | 杭州比智科技有限公司 | Hive data authority control agent layer method and system |
| CN114896584B (en) * | 2022-07-13 | 2022-10-11 | 杭州比智科技有限公司 | Hive data authority control agent layer method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113626438B (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111124917B (en) | Method, device, equipment and storage medium for managing and controlling public test cases | |
| WO2022095518A1 (en) | Automatic interface test method and apparatus, and computer device and storage medium | |
| CN113326991A (en) | Automatic authorization method, device, computer equipment and storage medium | |
| CN113254445A (en) | Real-time data storage method and device, computer equipment and storage medium | |
| CN111680477A (en) | Method and device for exporting spreadsheet file, computer equipment and storage medium | |
| CN112085087A (en) | Method and device for generating business rules, computer equipment and storage medium | |
| CN112199442A (en) | Distributed batch file downloading method and device, computer equipment and storage medium | |
| CN113377372A (en) | Business rule analysis method and device, computer equipment and storage medium | |
| CN114564294A (en) | Intelligent service arranging method and device, computer equipment and storage medium | |
| CN112182004A (en) | Method and device for viewing data in real time, computer equipment and storage medium | |
| CN113010542B (en) | Service data processing method, device, computer equipment and storage medium | |
| CN113626438A (en) | Data table management method and device, computer equipment and storage medium | |
| WO2022156087A1 (en) | Data blood relationship establishing method and apparatus, computer device, and storage medium | |
| CN116956326A (en) | Authority data processing method and device, computer equipment and storage medium | |
| CN111782649A (en) | Data acquisition format updating method and device, computer equipment and storage medium | |
| CN113886332B (en) | Large file difference comparison method and device, computer equipment and storage medium | |
| CN113254106B (en) | Task execution method and device based on Flink, computer equipment and storage medium | |
| CN115543428A (en) | Simulated data generation method and device based on strategy template | |
| CN115292580A (en) | Data query method and device, computer equipment and storage medium | |
| CN115203672A (en) | Information access control method and device, computer equipment and medium | |
| CN114615325A (en) | Message pushing method and device, computer equipment and storage medium | |
| CN112732925A (en) | Method for determining investment data based on atlas, storage medium and related equipment | |
| CN111143328A (en) | Agile business intelligent data construction method, system, equipment and storage medium | |
| CN109409922A (en) | Data aggregate modeling method, device, computer equipment and storage medium | |
| CN115936909A (en) | Data processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |