CN117827988A - Data warehouse optimization method, device, equipment and storage medium thereof - Google Patents
Data warehouse optimization method, device, equipment and storage medium thereof Download PDFInfo
- Publication number
- CN117827988A CN117827988A CN202410017285.3A CN202410017285A CN117827988A CN 117827988 A CN117827988 A CN 117827988A CN 202410017285 A CN202410017285 A CN 202410017285A CN 117827988 A CN117827988 A CN 117827988A
- Authority
- CN
- China
- Prior art keywords
- target
- frequency call
- frequency
- low
- data warehouse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000005457 optimization Methods 0.000 title claims abstract description 53
- 238000004364 calculation method Methods 0.000 claims abstract description 26
- 239000008280 blood Substances 0.000 claims abstract description 18
- 210000004369 blood Anatomy 0.000 claims abstract description 18
- 238000004458 analytical method Methods 0.000 claims description 34
- 238000012545 processing Methods 0.000 claims description 25
- 238000011084 recovery Methods 0.000 claims description 18
- 238000005192 partition Methods 0.000 claims description 9
- 238000012423 maintenance Methods 0.000 abstract description 26
- 238000010586 diagram Methods 0.000 abstract description 15
- 238000005516 engineering process Methods 0.000 abstract description 13
- 230000002035 prolonged effect Effects 0.000 abstract description 2
- 238000013473 artificial intelligence Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application belongs to the technical field of financial science and technology, is applied to a data warehouse optimization scene in the financial industry, and relates to a data warehouse optimization method, a device, equipment and a storage medium thereof, wherein the occupancy rate of threads Chi Shishi of a target data warehouse is calculated; if the threshold value is not exceeded, repeating real-time calculation and identification; if the threshold value is exceeded, acquiring a data call log; parsing to obtain SQL execution statements; analyzing SQL execution statements, and according to the dependency relationship among the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and all called tables; and updating and optimizing the data in the target data warehouse to obtain the latest blood relationship diagram. The low-frequency call data in the financial business class data warehouse is prevented from occupying too much storage space, and the high-frequency call data is prevented from being scattered to a plurality of forms. The maintenance personnel can conveniently combine the latest blood-edge relation diagram to carry out data warehouse maintenance management, so that the overlarge pressure of the data warehouse is avoided, and the service life of the data warehouse is prolonged.
Description
Technical Field
The application relates to the technical field of financial science and technology, and is applied to a data warehouse optimization scene in the financial industry, in particular to a data warehouse optimization method, a data warehouse optimization device, data warehouse optimization equipment and a storage medium thereof.
Background
With the rapid development of the internet, various industries seek industry breakthrough points by relying on the internet, and in recent years, the financial industry is expanding online business around the internet. The business platform has higher and higher requirements on the financial data warehouse due to larger traffic and data volume involved in the financial industry. Particularly in the field of financial services such as financial investment, financial financing or stock analysis.
The conventional optimization mode of the financial data warehouse is mostly a timing optimization mode, namely, the data warehouse is optimized at a certain fixed time point, such as fixed three-point to five-point at night, namely, the optimization mechanism is triggered to optimize the financial data warehouse when the data warehouse reaches the three-point at night, the mode avoids the high-rise period of service, but is extremely easy to cause useless optimization, moreover, the financial data warehouse cannot be optimized in real time in the real-time process of the financial service, the target data warehouse is easy to store excessive useless data, the form files cannot be adjusted in real time, the pressure of the data warehouse is overlarge, the service life of the data warehouse is reduced, and maintenance personnel are inconvenient to maintain and manage.
Disclosure of Invention
An aim of the embodiment of the application is to provide a data warehouse optimization method, a device, equipment and a storage medium thereof, so as to solve the problems that in the prior art, a financial data warehouse cannot be optimized in real time in the real-time process of financial business, too much useless data cannot be stored in a target data warehouse, and form files cannot be adjusted in real time, so that the pressure of the data warehouse is too high, the service life of the data warehouse is reduced, and maintenance personnel are inconvenient to maintain and manage.
In order to solve the above technical problems, the embodiment of the present application provides a data warehouse optimization method, which adopts the following technical scheme:
a data warehouse optimization method, comprising the steps of:
step 201, calculating the occupancy rate of the thread Chi Shishi when the target data warehouse performs data processing according to a preset real-time calculation component;
step 202, identifying whether the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold by a comparison mode;
step 203, if the occupancy rate of the thread Chi Shishi does not exceed the real-time occupancy rate threshold, executing step 201;
step 204, if the occupancy rate of the thread Chi Shishi exceeds the real-time occupancy rate threshold, acquiring a data call log corresponding to the target data warehouse;
Step 205, obtaining SQL execution sentences in a target time period by analyzing the data call log;
step 206, analyzing the SQL execution statement according to a preset SQL analysis component, and identifying a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field and the dependency relationship among all called tables in the target time period;
and step 207, updating and optimizing the data in the target data warehouse based on the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables, and obtaining the latest blood-edge relationship graph according to the updating and optimizing result.
Further, the step of analyzing the SQL execution statement according to a preset SQL analysis component to identify a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field and a dependency relationship among all called tables in the target time period specifically includes:
analyzing the SQL execution statement by using an SQL analysis tool of the Spark component, and identifying all tables, all fields and dependency relations among all called tables in the target time period through analysis results, or,
Analyzing the SQL execution statement by using an SQL analysis tool of the Hive component, and identifying all tables, all fields and dependency relations among all called tables in the target time period through analysis results, or,
analyzing the SQL execution statement by adopting a MapReduce component in the Hadoop tool, and identifying all tables, all fields and dependency relations among all called tables which are called in the target time period through analysis results;
and counting the calling times of all the tables and all the fields which are called in the target time period, and determining the high-frequency calling table, the low-frequency calling table, the high-frequency calling field and the low-frequency calling field according to the counting result of the calling times.
Further, the step of updating and optimizing the data in the target data warehouse based on the dependency relationship among the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and all called tables, and obtaining the latest blood-edge relationship graph according to the updating and optimizing result specifically includes:
according to the calling times respectively corresponding to the high-frequency calling table, the low-frequency calling table, the high-frequency calling field and the low-frequency calling field;
Performing form reconstruction on the high-frequency call table and the low-frequency call table according to the call times, the high-frequency call field and the low-frequency call field to obtain a reconstructed target form;
and updating a preset original blood edge relation graph according to the dependency relationship among all the called tables and the reconstructed target table to obtain the latest blood edge relation graph.
Further, the step of reconstructing the high-frequency call table and the low-frequency call table according to the call times, the high-frequency call field and the low-frequency call field to obtain a reconstructed target table specifically includes:
identifying a high-frequency call field and a low-frequency call field in the high-frequency call table and a high-frequency call field and a low-frequency call field in the low-frequency call table according to the call times;
acquiring data corresponding to a high-frequency call field and a low-frequency call field in the high-frequency call table respectively, and acquiring data corresponding to the high-frequency call field and the low-frequency call field in the low-frequency call table respectively;
adding data corresponding to a high-frequency call field in the high-frequency call table and data corresponding to the high-frequency call field in the low-frequency call table into the same table to obtain a first target table;
And adding the data corresponding to the low-frequency call field in the high-frequency call table and the data corresponding to the low-frequency call field in the low-frequency call table into the same table to obtain a second target table.
Further, after the step of adding the data corresponding to the high-frequency call field in the high-frequency call table and the data corresponding to the high-frequency call field in the low-frequency call table to the same table to obtain the first target table is executed, the method further includes:
identifying whether the row information of the first target form exceeds a preset first maximum row value and whether the column information of the first target form exceeds a preset first maximum column value according to the row and column storage requirements of a single form in the target data warehouse;
if the row information of the first target form does not exceed the first maximum row value and the column information of the first target form does not exceed the first maximum column value, naming the first target form, and obtaining a naming result as a target table name;
if the row information of the first target form exceeds the first maximum row value or the column information of the first target form exceeds the first maximum column value, splitting the data in the first target form to obtain a split form, wherein the split form does not exceed the first maximum row value and the column information does not exceed the first maximum column value;
Storing the split forms into the same partition preset in the target data warehouse, and renaming the split forms respectively by combining partition identifiers to obtain the table name information of the split forms as target table names.
Further, after the step of adding the data corresponding to the low frequency call field in the high frequency call table and the data corresponding to the low frequency call field in the low frequency call table to the same table to obtain the second target table is executed, the method further includes:
identifying whether the row information of the second target form exceeds a preset second maximum row value and whether the column information of the second target form exceeds a preset second maximum column value according to the row and column storage requirements of the single form in the target data warehouse;
if the row information of the second target form does not exceed the second maximum row value and the column information of the second target form does not exceed the second maximum column value, continuing to cache the second target form in the target data warehouse;
if the row information of the second target form exceeds the second maximum row value or the column information of the second target form exceeds the second maximum column value, deleting the data in the second target form, and adding the deleted second target form into a preset recovery component, wherein the preset recovery component is a recovery component with a preset recovery period, and if the time interval between the current system time and the deleting time point exceeds the recovery period, permanently deleting the second target form in the recovery component.
Further, the step of updating the preset original blood-edge relationship graph according to the dependency relationship among all called tables and the reconstructed target table to obtain the latest blood-edge relationship graph specifically includes:
identifying a high-frequency call field with a dependency relationship according to the dependency relationship among all called tables;
and updating the original blood edge relation graph based on the target table names corresponding to the tables where all the high-frequency calling fields are respectively located and the dependency relations among all the high-frequency calling fields to obtain the latest blood edge relation graph.
In order to solve the above technical problems, the embodiments of the present application further provide a data warehouse optimization device, which adopts the following technical scheme:
a data warehouse optimization device, comprising:
the real-time calculation module is used for calculating the occupancy rate of the thread Chi Shishi when the target data warehouse performs data processing according to a preset real-time calculation component;
the identifying and judging module is used for identifying whether the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold value in a comparison mode;
a repeated execution module, configured to execute the step 201 if the occupancy rate of the thread Chi Shishi does not exceed the real-time occupancy rate threshold;
The data call log acquisition module is used for acquiring a data call log corresponding to the target data warehouse if the occupancy rate of the thread Chi Shishi exceeds the real-time occupancy rate threshold value;
the data call log analysis module is used for obtaining SQL execution sentences in a target time period by analyzing the data call log;
the SQL execution statement analysis module is used for analyzing the SQL execution statement according to a preset SQL analysis component and identifying a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field and the dependency relationship among all called tables in the target time period;
and the data warehouse updating and optimizing module is used for updating and optimizing the data in the target data warehouse based on the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables, and obtaining the latest blood edge relationship graph according to the updating and optimizing result.
In order to solve the above technical problems, the embodiments of the present application further provide a computer device, which adopts the following technical schemes:
a computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data warehouse optimization method described above.
In order to solve the above technical problems, embodiments of the present application further provide a computer readable storage medium, which adopts the following technical solutions:
a computer readable storage medium having stored thereon computer readable instructions which when executed by a processor perform the steps of a data warehouse optimization method as described above.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
according to the data warehouse optimization method, the occupancy rate of threads Chi Shishi when the target data warehouse is subjected to data processing is calculated according to the real-time calculation component; identifying whether the occupancy of the thread Chi Shishi exceeds a preset real-time occupancy threshold; if not, repeating the real-time calculation and the identification; if yes, acquiring a data call log corresponding to the target data warehouse; the SQL execution statement in the target time period is obtained by analyzing the data call log; analyzing SQL execution sentences, identifying and according to the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables in the target time period; and updating and optimizing the data in the target data warehouse, and obtaining the latest blood-margin relation graph according to the updating and optimizing result. The method and the device avoid that the low-frequency call data occupy too much storage space, and also prevent the high-frequency call data from being scattered into a plurality of forms so as to be unfavorable for maintenance and management. The data warehouse optimization method is convenient for assisting data warehouse maintenance personnel to carry out target data warehouse maintenance management by combining with the latest blood relationship diagram, avoids overlarge data warehouse pressure, particularly relates to a financial business type data warehouse, has complicated data and large data volume, and can further improve the service life of the corresponding financial business type data warehouse.
Drawings
For a clearer description of the solution in the present application, a brief description will be given below of the drawings that are needed in the description of the embodiments of the present application, it being obvious that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a data warehouse optimization method according to the present application;
FIG. 3 is a flow chart of one embodiment of step 207 shown in FIG. 2;
FIG. 4 is a flow chart of one embodiment of step 302 shown in FIG. 3;
FIG. 5 is a flow chart of one embodiment of step 303 shown in FIG. 3;
FIG. 6 is a schematic diagram of the architecture of one embodiment of a data warehouse optimization device in accordance with the present application;
FIG. 7 is a schematic structural diagram of one embodiment of a computer device according to the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to better understand the technical solutions of the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture ExpertsGroup Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving PictureExperts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the data warehouse optimization method provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the data warehouse optimization device is generally disposed in the server/terminal device.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow chart of one embodiment of a data warehouse optimization method in accordance with the present application is shown. The data warehouse optimization method comprises the following steps:
In step 201, the occupancy rate of the thread Chi Shishi when the target data warehouse performs data processing is calculated according to a preset real-time calculation component, wherein the preset real-time calculation component comprises an online calculation component based on a flank stream data processing form and an offline calculation component based on a Spark stream data processing form.
The occupancy rate of the thread Chi Shishi when the target data warehouse is subjected to data processing is calculated through the real-time calculation component, so that when the occupancy rate of the thread Chi Shishi exceeds the threshold value of the budgeted real-time occupancy rate, the target data warehouse is updated and optimized in time, and excessive data processing resources occupied by the target data warehouse are avoided. Specifically, the target data warehouse may be an insurance business data warehouse, a financial analysis business data warehouse or a stock analysis business data warehouse.
The online computing component based on the Flink stream data processing form and the offline computing component based on the Spark stream data processing form can calculate the occupancy rate of the thread Chi Shishi in real time in the transmission process of data in a stream form, so that the occupancy rate of the thread Chi Shishi can be conveniently obtained when the data is circulated based on the target data warehouse, and excessive data processing resources occupied by the target data warehouse are avoided.
In step 202, by comparison, it is identified whether the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold.
Step 203, if the occupancy rate of the thread Chi Shishi does not exceed the real-time occupancy rate threshold, then step 201 is performed.
And 204, if the occupancy rate of the thread Chi Shishi exceeds the real-time occupancy rate threshold, acquiring a data call log corresponding to the target data warehouse.
And step 205, obtaining SQL execution sentences in a target time period by analyzing the data call log.
And 206, analyzing the SQL execution statement according to a preset SQL analysis component, and identifying the dependency relationship among the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and all called tables in the target time period.
In this embodiment, the step of analyzing the SQL execution statement according to a preset SQL analysis component, and identifying a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field, and a dependency relationship among all called tables in the target time period specifically includes: analyzing the SQL execution statement by using an SQL analysis tool of a Spark component, identifying all tables, all fields and all dependency relations among the called tables in the target time period through analysis results, or analyzing the SQL execution statement by using an SQL analysis tool of a Hive component, identifying all tables, all fields and all dependency relations among the called tables in the target time period through analysis results, or analyzing the SQL execution statement by using a MapReduce component in a Hadoop tool, and identifying all tables, all fields and all dependency relations among the called tables in the target time period through analysis results; and counting the calling times of all the tables and all the fields which are called in the target time period, and determining the high-frequency calling table, the low-frequency calling table, the high-frequency calling field and the low-frequency calling field according to the counting result of the calling times.
Analyzing the SQL execution statement through a preset SQL analysis component, identifying the dependency relationship among the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and all called tables in the target time period, facilitating updating and optimizing the target data warehouse, avoiding that the low-frequency call data occupy too much storage space and preventing the high-frequency call data from being scattered into a plurality of forms, and being unfavorable for maintenance and management. Specifically, the target time period refers to a time period between a time point when the updating and optimizing of the target data warehouse are completed last time and a time point when the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold value when the target data warehouse performs data processing at this time.
And step 207, updating and optimizing the data in the target data warehouse based on the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables, and obtaining the latest blood-edge relationship graph according to the updating and optimizing result.
With continued reference to fig. 3, fig. 3 is a flow chart illustrating one embodiment of step 207 of fig. 2, including:
Step 301, according to the call times respectively corresponding to the high-frequency call table, the low-frequency call table, the high-frequency call field and the low-frequency call field;
step 302, performing form reconstruction on the high-frequency call table and the low-frequency call table according to the call times, the high-frequency call field and the low-frequency call field, and obtaining a reconstructed target form;
with continued reference to FIG. 4, FIG. 4 is a flow chart of one embodiment of step 302 of FIG. 3, including:
step 401, identifying a high-frequency call field and a low-frequency call field in the high-frequency call table, and a high-frequency call field and a low-frequency call field in the low-frequency call table according to the call times;
step 402, obtaining data corresponding to a high-frequency call field and a low-frequency call field in the high-frequency call table respectively, and obtaining data corresponding to a high-frequency call field and a low-frequency call field in the low-frequency call table respectively;
step 403, adding the data corresponding to the high-frequency call field in the high-frequency call table and the data corresponding to the high-frequency call field in the low-frequency call table to the same table to obtain a first target table;
in this embodiment, after the step of adding the data corresponding to the high-frequency call field in the high-frequency call table and the data corresponding to the high-frequency call field in the low-frequency call table to the same table to obtain the first target table is performed, the method further includes: identifying whether the row information of the first target form exceeds a preset first maximum row value and whether the column information of the first target form exceeds a preset first maximum column value according to the row and column storage requirements of a single form in the target data warehouse; if the row information of the first target form does not exceed the first maximum row value and the column information of the first target form does not exceed the first maximum column value, naming the first target form, and obtaining a naming result as a target table name; if the row information of the first target form exceeds the first maximum row value or the column information of the first target form exceeds the first maximum column value, splitting the data in the first target form to obtain a split form, wherein the split form does not exceed the first maximum row value and the column information does not exceed the first maximum column value; storing the split forms into the same partition preset in the target data warehouse, and renaming the split forms respectively by combining partition identifiers to obtain the table name information of the split forms as target table names.
The method comprises the steps of adding data corresponding to all high-frequency calling fields into the same form to generate a first target form, splitting the first target form according to row and column storage requirements, storing the split form into the same partition preset in a target data warehouse, renaming the split form respectively by combining partition identifiers to obtain the table name information of the split form as a target table name, integrating the high-frequency calling fields in the target time period into the same partition as much as possible, ensuring that each split form meets the row and column storage requirements of the target data warehouse, avoiding excessive or too little data quantity in the single form, and facilitating maintenance and management of the target data warehouse.
And step 404, adding the data corresponding to the low frequency call field in the high frequency call table and the data corresponding to the low frequency call field in the low frequency call table into the same table to obtain a second target table.
In this embodiment, after the step of adding the data corresponding to the low frequency call field in the high frequency call table and the data corresponding to the low frequency call field in the low frequency call table to the same table to obtain the second target table, the method further includes: identifying whether the row information of the second target form exceeds a preset second maximum row value and whether the column information of the second target form exceeds a preset second maximum column value according to the row and column storage requirements of the single form in the target data warehouse; if the row information of the second target form does not exceed the second maximum row value and the column information of the second target form does not exceed the second maximum column value, continuing to cache the second target form in the target data warehouse; if the row information of the second target form exceeds the second maximum row value or the column information of the second target form exceeds the second maximum column value, deleting the data in the second target form, and adding the deleted second target form into a preset recovery component, wherein the preset recovery component is a recovery component with a preset recovery period, and if the time interval between the current system time and the deleting time point exceeds the recovery period, permanently deleting the second target form in the recovery component.
And adding data corresponding to all the low-frequency calling fields into the same form to generate a second target form, identifying whether the rank information in the second target form reaches a preset maximum rank value, and deleting the second target form if the rank information reaches the preset maximum rank value, so that the data corresponding to too many low-frequency calling fields are prevented from being stored in the target data warehouse, the storage resource consumption of the target data warehouse is saved to a certain extent, and the maintenance and the management of the target data warehouse are facilitated.
And step 303, updating a preset original blood-edge relationship graph according to the dependency relationship among all called tables and the reconstructed target table, and obtaining the latest blood-edge relationship graph.
With continued reference to fig. 5, fig. 5 is a flow chart of one embodiment of step 303 of fig. 3, including:
step 501, identifying a high-frequency call field with a dependency relationship according to the dependency relationship among all called tables;
step 502, updating the original blood-margin relation graph based on the target table names corresponding to the tables where all the high-frequency call fields are respectively located and the dependency relations among all the high-frequency call fields, so as to obtain the latest blood-margin relation graph.
In this embodiment, after performing the step of obtaining the latest blood-edge relationship map, the method further includes: steps 201 to 207 are executed in an iterative manner, so that continuous updating and optimization of the target data warehouse are facilitated according to the occupancy rate of threads Chi Shishi corresponding to the target data warehouse, the service life of the target data warehouse is guaranteed, and the target data warehouse is maintained in time.
After the target data warehouse is updated and adjusted, the original blood-edge relation diagram corresponding to the target data warehouse is updated according to a new form (the split form or the first target form) generated after updating and optimizing, so that the latest blood-edge relation diagram is obtained, maintenance personnel of the auxiliary data warehouse can conveniently combine the latest blood-edge relation diagram to maintain and manage the target data warehouse, excessive pressure of the data warehouse, particularly the financial business data warehouse, is avoided, the involved data are complex, the data amount is large, and the service life of the corresponding financial business data warehouse can be further prolonged by adopting the data warehouse optimizing method.
The method comprises the steps that the occupancy rate of threads Chi Shishi when a target data warehouse is subjected to data processing is calculated according to a real-time calculation component; identifying whether the occupancy of the thread Chi Shishi exceeds a preset real-time occupancy threshold; if not, repeating the real-time calculation and the identification; if yes, acquiring a data call log corresponding to the target data warehouse; the SQL execution statement in the target time period is obtained by analyzing the data call log; analyzing SQL execution sentences, identifying and according to the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables in the target time period; and updating and optimizing the data in the target data warehouse, and obtaining the latest blood-margin relation graph according to the updating and optimizing result. The method and the device avoid that the low-frequency call data occupy too much storage space, and also prevent the high-frequency call data from being scattered into a plurality of forms so as to be unfavorable for maintenance and management. The data warehouse optimization method is convenient for assisting data warehouse maintenance personnel to carry out target data warehouse maintenance management by combining with the latest blood relationship diagram, avoids overlarge data warehouse pressure, particularly relates to a financial business type data warehouse, has complicated data and large data volume, and can further improve the service life of the corresponding financial business type data warehouse.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, large data warehouse optimization technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
In the embodiment of the application, the occupancy rate of the thread Chi Shishi when the target data warehouse is subjected to data processing is calculated according to the real-time calculation component; identifying whether the occupancy of the thread Chi Shishi exceeds a preset real-time occupancy threshold; if not, repeating the real-time calculation and the identification; if yes, acquiring a data call log corresponding to the target data warehouse; the SQL execution statement in the target time period is obtained by analyzing the data call log; analyzing SQL execution sentences, identifying and according to the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables in the target time period; and updating and optimizing the data in the target data warehouse, and obtaining the latest blood-margin relation graph according to the updating and optimizing result. The method and the device avoid that the low-frequency call data occupy too much storage space, and also prevent the high-frequency call data from being scattered into a plurality of forms so as to be unfavorable for maintenance and management. The data warehouse optimization method is convenient for assisting data warehouse maintenance personnel to carry out target data warehouse maintenance management by combining with the latest blood relationship diagram, avoids overlarge data warehouse pressure, particularly relates to a financial business type data warehouse, has complicated data and large data volume, and can further improve the service life of the corresponding financial business type data warehouse.
With further reference to fig. 6, as an implementation of the method shown in fig. 2 described above, the present application provides an embodiment of a data warehouse optimization apparatus, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic devices.
As shown in fig. 6, the data warehouse optimization device 600 according to the present embodiment includes: the system comprises a real-time calculation module 601, an identification judgment module 602, a repeated execution module 603, a data call log acquisition module 604, a data call log analysis module 605, an SQL execution statement analysis module 606 and a data warehouse update optimization module 607. Wherein:
the real-time computing module 601 is configured to compute, according to a preset real-time computing component, an occupancy rate of a thread Chi Shishi when a target data warehouse performs data processing, where the preset real-time computing component includes an online computing component based on a Flink stream data processing form and an offline computing component based on a Spark stream data processing form;
an identifying and judging module 602, configured to identify whether the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold;
a repeated execution module 603, configured to execute the step 201 if the occupancy rate of the thread Chi Shishi does not exceed the real-time occupancy rate threshold;
A data call log obtaining module 604, configured to obtain a data call log corresponding to the target data repository if the occupancy rate of the thread Chi Shishi exceeds the real-time occupancy rate threshold;
the data call log analysis module 605 is configured to obtain an SQL execution statement in a target time period by analyzing the data call log;
the SQL execution statement parsing module 606 is configured to parse the SQL execution statement according to a preset SQL parsing component, and identify a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field, and dependencies among all called tables in the target time period;
the data warehouse update optimization module 607 is configured to update and optimize data in the target data warehouse based on the high frequency call table, the low frequency call table, the high frequency call field, the low frequency call field, and the dependency relationships among all called tables, and obtain the latest blood edge relationship graph according to the update optimization result.
The method comprises the steps that the occupancy rate of threads Chi Shishi when a target data warehouse is subjected to data processing is calculated according to a real-time calculation component; identifying whether the occupancy of the thread Chi Shishi exceeds a preset real-time occupancy threshold; if not, repeating the real-time calculation and the identification; if yes, acquiring a data call log corresponding to the target data warehouse; the SQL execution statement in the target time period is obtained by analyzing the data call log; analyzing SQL execution sentences, identifying and according to the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables in the target time period; and updating and optimizing the data in the target data warehouse, and obtaining the latest blood-margin relation graph according to the updating and optimizing result. The method and the device avoid that the low-frequency call data occupy too much storage space, and also prevent the high-frequency call data from being scattered into a plurality of forms so as to be unfavorable for maintenance and management. The data warehouse optimization method is convenient for assisting data warehouse maintenance personnel to carry out target data warehouse maintenance management by combining with the latest blood relationship diagram, avoids overlarge data warehouse pressure, particularly relates to a financial business type data warehouse, has complicated data and large data volume, and can further improve the service life of the corresponding financial business type data warehouse.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by computer readable instructions, stored on a computer readable storage medium, that the program when executed may comprise the steps of embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 7, fig. 7 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 7 comprises a memory 7a, a processor 7b, a network interface 7c communicatively connected to each other via a system bus. It should be noted that only a computer device 7 having components 7a-7c is shown in the figures, but it should be understood that not all of the illustrated components need be implemented, and that more or fewer components may alternatively be implemented. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.
The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 7a includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 7a may be an internal storage unit of the computer device 7, such as a hard disk or a memory of the computer device 7. In other embodiments, the memory 7a may also be an external storage device of the computer device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 7. Of course, the memory 7a may also comprise both an internal memory unit of the computer device 7 and an external memory device. In this embodiment, the memory 7a is typically used to store an operating system and various application software installed on the computer device 7, such as computer readable instructions of a data warehouse optimization method. Further, the memory 7a may be used to temporarily store various types of data that have been output or are to be output.
The processor 7b may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data warehouse optimization chip in some embodiments. The processor 7b is typically used to control the overall operation of the computer device 7. In this embodiment, the processor 7b is configured to execute computer readable instructions stored in the memory 7a or process data, such as computer readable instructions for executing the data warehouse optimization method.
The network interface 7c may comprise a wireless network interface or a wired network interface, which network interface 7c is typically used for establishing a communication connection between the computer device 7 and other electronic devices.
The computer equipment provided by the embodiment belongs to the technical field of financial science and technology, and is applied to a data warehouse optimization scene in the financial industry. The method comprises the steps that the occupancy rate of threads Chi Shishi when a target data warehouse is subjected to data processing is calculated according to a real-time calculation component; identifying whether the occupancy of the thread Chi Shishi exceeds a preset real-time occupancy threshold; if not, repeating the real-time calculation and the identification; if yes, acquiring a data call log corresponding to the target data warehouse; the SQL execution statement in the target time period is obtained by analyzing the data call log; analyzing SQL execution sentences, identifying and according to the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables in the target time period; and updating and optimizing the data in the target data warehouse, and obtaining the latest blood-margin relation graph according to the updating and optimizing result. The method and the device avoid that the low-frequency call data occupy too much storage space, and also prevent the high-frequency call data from being scattered into a plurality of forms so as to be unfavorable for maintenance and management. The data warehouse optimization method is convenient for assisting data warehouse maintenance personnel to carry out target data warehouse maintenance management by combining with the latest blood relationship diagram, avoids overlarge data warehouse pressure, particularly relates to a financial business type data warehouse, has complicated data and large data volume, and can further improve the service life of the corresponding financial business type data warehouse.
The present application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions executable by a processor to cause the processor to perform the steps of a data warehouse optimization method as described above.
The computer readable storage medium provided by the embodiment belongs to the technical field of financial science and technology, and is applied to a data warehouse optimization scene in the financial industry. The method comprises the steps that the occupancy rate of threads Chi Shishi when a target data warehouse is subjected to data processing is calculated according to a real-time calculation component; identifying whether the occupancy of the thread Chi Shishi exceeds a preset real-time occupancy threshold; if not, repeating the real-time calculation and the identification; if yes, acquiring a data call log corresponding to the target data warehouse; the SQL execution statement in the target time period is obtained by analyzing the data call log; analyzing SQL execution sentences, identifying and according to the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables in the target time period; and updating and optimizing the data in the target data warehouse, and obtaining the latest blood-margin relation graph according to the updating and optimizing result. The method and the device avoid that the low-frequency call data occupy too much storage space, and also prevent the high-frequency call data from being scattered into a plurality of forms so as to be unfavorable for maintenance and management. The data warehouse optimization method is convenient for assisting data warehouse maintenance personnel to carry out target data warehouse maintenance management by combining with the latest blood relationship diagram, avoids overlarge data warehouse pressure, particularly relates to a financial business type data warehouse, has complicated data and large data volume, and can further improve the service life of the corresponding financial business type data warehouse.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.
It is apparent that the embodiments described above are only some embodiments of the present application, but not all embodiments, the preferred embodiments of the present application are given in the drawings, but not limiting the patent scope of the present application. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a more thorough understanding of the present disclosure. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing, or equivalents may be substituted for elements thereof. All equivalent structures made by the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the protection scope of the application.
Claims (10)
1. A method of data warehouse optimization, comprising the steps of:
step 201, calculating the occupancy rate of the thread Chi Shishi when the target data warehouse performs data processing according to a preset real-time calculation component;
step 202, identifying whether the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold by a comparison mode;
step 203, if the occupancy rate of the thread Chi Shishi does not exceed the real-time occupancy rate threshold, executing step 201;
step 204, if the occupancy rate of the thread Chi Shishi exceeds the real-time occupancy rate threshold, acquiring a data call log corresponding to the target data warehouse;
step 205, obtaining SQL execution sentences in a target time period by analyzing the data call log;
step 206, analyzing the SQL execution statement according to a preset SQL analysis component, and identifying a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field and the dependency relationship among all called tables in the target time period;
and step 207, updating and optimizing the data in the target data warehouse based on the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables, and obtaining the latest blood-edge relationship graph according to the updating and optimizing result.
2. The method for optimizing a data warehouse according to claim 1, wherein the step of analyzing the SQL execution statement according to a preset SQL analysis component, and identifying the dependency relationship among the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and all called tables in the target time period specifically comprises:
analyzing the SQL execution statement by using an SQL analysis tool of the Spark component, and identifying all tables, all fields and dependency relations among all called tables in the target time period through analysis results, or,
analyzing the SQL execution statement by using an SQL analysis tool of the Hive component, and identifying all tables, all fields and dependency relations among all called tables in the target time period through analysis results, or,
analyzing the SQL execution statement by adopting a MapReduce component in the Hadoop tool, and identifying all tables, all fields and dependency relations among all called tables which are called in the target time period through analysis results;
and counting the calling times of all the tables and all the fields which are called in the target time period, and determining the high-frequency calling table, the low-frequency calling table, the high-frequency calling field and the low-frequency calling field according to the counting result of the calling times.
3. The data warehouse optimization method as claimed in claim 1, wherein the step of updating and optimizing the data in the target data warehouse based on the dependency relationship among the high frequency call table, the low frequency call table, the high frequency call field, the low frequency call field, and all called tables, and obtaining the latest blood-edge relationship graph according to the updated and optimized result specifically comprises:
according to the calling times respectively corresponding to the high-frequency calling table, the low-frequency calling table, the high-frequency calling field and the low-frequency calling field;
performing form reconstruction on the high-frequency call table and the low-frequency call table according to the call times, the high-frequency call field and the low-frequency call field to obtain a reconstructed target form;
and updating a preset original blood edge relation graph according to the dependency relationship among all the called tables and the reconstructed target table to obtain the latest blood edge relation graph.
4. The data warehouse optimization method as claimed in claim 3, wherein the step of reconstructing the high frequency call table and the low frequency call table according to the call times, the high frequency call fields and the low frequency call fields to obtain the reconstructed target table specifically comprises:
Identifying a high-frequency call field and a low-frequency call field in the high-frequency call table and a high-frequency call field and a low-frequency call field in the low-frequency call table according to the call times;
acquiring data corresponding to a high-frequency call field and a low-frequency call field in the high-frequency call table respectively, and acquiring data corresponding to the high-frequency call field and the low-frequency call field in the low-frequency call table respectively;
adding data corresponding to a high-frequency call field in the high-frequency call table and data corresponding to the high-frequency call field in the low-frequency call table into the same table to obtain a first target table;
and adding the data corresponding to the low-frequency call field in the high-frequency call table and the data corresponding to the low-frequency call field in the low-frequency call table into the same table to obtain a second target table.
5. The data warehouse optimization method as claimed in claim 4, wherein after performing the step of adding the data corresponding to the high frequency call field in the high frequency call table and the data corresponding to the high frequency call field in the low frequency call table to the same table to obtain the first target table, the method further comprises:
Identifying whether the row information of the first target form exceeds a preset first maximum row value and whether the column information of the first target form exceeds a preset first maximum column value according to the row and column storage requirements of a single form in the target data warehouse;
if the row information of the first target form does not exceed the first maximum row value and the column information of the first target form does not exceed the first maximum column value, naming the first target form, and obtaining a naming result as a target table name;
if the row information of the first target form exceeds the first maximum row value or the column information of the first target form exceeds the first maximum column value, splitting the data in the first target form to obtain a split form, wherein the split form does not exceed the first maximum row value and the column information does not exceed the first maximum column value;
storing the split forms into the same partition preset in the target data warehouse, and renaming the split forms respectively by combining partition identifiers to obtain the table name information of the split forms as target table names.
6. The data warehouse optimization method as claimed in claim 4, wherein after performing the step of adding the data corresponding to the low frequency call field in the high frequency call table and the data corresponding to the low frequency call field in the low frequency call table to the same table to obtain the second target table, the method further comprises:
identifying whether the row information of the second target form exceeds a preset second maximum row value and whether the column information of the second target form exceeds a preset second maximum column value according to the row and column storage requirements of the single form in the target data warehouse;
if the row information of the second target form does not exceed the second maximum row value and the column information of the second target form does not exceed the second maximum column value, continuing to cache the second target form in the target data warehouse;
if the row information of the second target form exceeds the second maximum row value or the column information of the second target form exceeds the second maximum column value, deleting the data in the second target form, and adding the deleted second target form into a preset recovery component, wherein the preset recovery component is a recovery component with a preset recovery period, and if the time interval between the current system time and the deleting time point exceeds the recovery period, permanently deleting the second target form in the recovery component.
7. The data warehouse optimization method according to claim 5, wherein the step of updating a preset original blood-margin relationship graph according to the dependency relationship among all called tables and the reconstructed target table to obtain the latest blood-margin relationship graph specifically comprises the following steps:
identifying a high-frequency call field with a dependency relationship according to the dependency relationship among all called tables;
and updating the original blood edge relation graph based on the target table names corresponding to the tables where all the high-frequency calling fields are respectively located and the dependency relations among all the high-frequency calling fields to obtain the latest blood edge relation graph.
8. A data warehouse optimization apparatus, comprising:
the real-time calculation module is used for calculating the occupancy rate of the thread Chi Shishi when the target data warehouse performs data processing according to a preset real-time calculation component;
the identifying and judging module is used for identifying whether the occupancy rate of the thread Chi Shishi exceeds a preset real-time occupancy rate threshold value in a comparison mode;
a repeated execution module, configured to execute the step 201 if the occupancy rate of the thread Chi Shishi does not exceed the real-time occupancy rate threshold;
The data call log acquisition module is used for acquiring a data call log corresponding to the target data warehouse if the occupancy rate of the thread Chi Shishi exceeds the real-time occupancy rate threshold value;
the data call log analysis module is used for obtaining SQL execution sentences in a target time period by analyzing the data call log;
the SQL execution statement analysis module is used for analyzing the SQL execution statement according to a preset SQL analysis component and identifying a high-frequency call table, a low-frequency call table, a high-frequency call field, a low-frequency call field and the dependency relationship among all called tables in the target time period;
and the data warehouse updating and optimizing module is used for updating and optimizing the data in the target data warehouse based on the high-frequency call table, the low-frequency call table, the high-frequency call field, the low-frequency call field and the dependency relationship among all called tables, and obtaining the latest blood edge relationship graph according to the updating and optimizing result.
9. A computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data warehouse optimisation method as claimed in any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the data warehouse optimization method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410017285.3A CN117827988A (en) | 2024-01-04 | 2024-01-04 | Data warehouse optimization method, device, equipment and storage medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410017285.3A CN117827988A (en) | 2024-01-04 | 2024-01-04 | Data warehouse optimization method, device, equipment and storage medium thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117827988A true CN117827988A (en) | 2024-04-05 |
Family
ID=90511337
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410017285.3A Pending CN117827988A (en) | 2024-01-04 | 2024-01-04 | Data warehouse optimization method, device, equipment and storage medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117827988A (en) |
-
2024
- 2024-01-04 CN CN202410017285.3A patent/CN117827988A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113282611B (en) | Method, device, computer equipment and storage medium for synchronizing stream data | |
| CN116431878A (en) | Vector retrieval service method, device, equipment and storage medium thereof | |
| CN113010542B (en) | Service data processing method, device, computer equipment and storage medium | |
| CN117094729A (en) | Request processing method, device, computer equipment and storage medium | |
| CN116450723A (en) | Data extraction method, device, computer equipment and storage medium | |
| CN115016735A (en) | Control method, device and medium for distributed cache system | |
| CN117827988A (en) | Data warehouse optimization method, device, equipment and storage medium thereof | |
| CN114496139A (en) | Quality control method, device, equipment and system of electronic medical record and readable medium | |
| CN114663073B (en) | Abnormal node discovery method and related equipment thereof | |
| CN116842011A (en) | Blood relationship analysis method, device, computer equipment and storage medium | |
| CN117290452A (en) | Data warehouse management method, device, equipment and storage medium thereof | |
| CN116680263A (en) | Data cleaning method, device, computer equipment and storage medium | |
| CN117272077A (en) | Data processing method, device, computer equipment and storage medium | |
| CN117194032A (en) | Task execution method, device, equipment and storage medium thereof | |
| CN117319400A (en) | Service processing node adjustment method, device, equipment and storage medium thereof | |
| CN116611936A (en) | Data analysis method, device, computer equipment and storage medium | |
| CN116795882A (en) | Data acquisition method, device, computer equipment and storage medium | |
| CN116821493A (en) | Message pushing method, device, computer equipment and storage medium | |
| CN116450687A (en) | Insurance product operation information release notification method and related equipment thereof | |
| CN116842050A (en) | Multi-dimensional state data processing method, device, computer equipment and storage medium | |
| CN117251468A (en) | Query processing method, device, computer equipment and storage medium | |
| CN117874073A (en) | Search optimization method, device, equipment and storage medium thereof | |
| CN116756163A (en) | Data synchronization improvement method, device, equipment and storage medium thereof | |
| CN116402644A (en) | Legal supervision method and system based on big data multi-source data fusion analysis | |
| CN117785832A (en) | Log processing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |