CN115640274A - Method, device and storage medium for dynamically adjusting database model - Google Patents

Method, device and storage medium for dynamically adjusting database model Download PDF

Info

Publication number
CN115640274A
CN115640274A CN202110814308.XA CN202110814308A CN115640274A CN 115640274 A CN115640274 A CN 115640274A CN 202110814308 A CN202110814308 A CN 202110814308A CN 115640274 A CN115640274 A CN 115640274A
Authority
CN
China
Prior art keywords
data
database model
database
model
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110814308.XA
Other languages
Chinese (zh)
Inventor
王爱军
郑星权
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN202110814308.XA priority Critical patent/CN115640274A/en
Priority to PCT/CN2022/091510 priority patent/WO2023000766A1/en
Publication of CN115640274A publication Critical patent/CN115640274A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a method, equipment and a storage medium for dynamically adjusting a database model. Optimizing a data processing process corresponding to managed data stored in a database model to obtain current measurement index data corresponding to the optimized data processing process; performing data analysis on the current measurement index data to obtain an analysis result; driving the database model to perform data adjustment according to the analysis result; and performing the optimization processing on the database model after data adjustment to update the current measurement index data until the current measurement index data is the same as preset target measurement index data, so as to dynamically adjust managed data stored in the database model, and solve the problem that the database cannot exert better performance due to deviation between the data and the model in the operation period in the related technology.

Description

Method, device and storage medium for dynamically adjusting database model
Technical Field
The embodiment of the invention relates to the technical field of databases, in particular to a method, equipment and a storage medium for dynamically adjusting a database model.
Background
Management and analysis for mass data is typically based on a database model. In the related art, the modeling of the database is driven by a rule-based model, however, the database model constructed by the method is generally not applicable to the management of various types of data. For example, in managing performance data of network elements of multiple standards, there may be a deviation between the run-time data and the model, which may result in that the database may not perform well.
Disclosure of Invention
The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
The embodiment of the invention provides a method, equipment and a storage medium for dynamically adjusting a database model, which can solve the problem that the database cannot exert better performance due to deviation between data and the model in the operation period in the related technology.
In a first aspect, an embodiment of the present invention provides a method for dynamically adjusting a database model, including:
optimizing the data processing process corresponding to the managed data stored in the database model to obtain current measurement index data corresponding to the optimized data processing process;
performing data analysis on the current measurement index data to obtain an analysis result;
driving the database model to carry out data adjustment according to the analysis result;
and performing the optimization processing on the database model after data adjustment to update the current measurement index data until the current measurement index data is the same as preset target measurement index data.
In a second aspect, an embodiment of the present invention further provides an apparatus, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method for dynamic adjustment of a database model as described above in the first aspect when executing the computer program.
In a third aspect, the embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for performing the method for dynamically adjusting the database model as described above.
The embodiment of the invention comprises the following steps: optimizing the data processing process corresponding to the managed data stored in the database model so as to restrict the data processing process of the managed data and keep the managed data in a continuously optimized state; after the data processing process of the managed data is optimized, the corresponding current measurement index data is also changed; after the data processing process is optimized, the current measurement index data corresponding to the optimized data processing process can be obtained through calculation; then, performing data analysis on the current measurement index data to obtain an analysis result; driving the database model to perform data adjustment according to the analysis result, namely dynamically adjusting managed data stored in the database model; and then, performing the optimization processing on the database model subjected to data adjustment to update the current measurement index data until the current measurement index data is the same as the preset target measurement index data, so that the optimized and adjusted database model can be obtained. The database model has good performance, can be suitable for the management and analysis of various types of data, and solves the problem that the database cannot exert good performance due to the deviation between the data and the model in the operation period in the related technology.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
FIG. 1 is a flow diagram of a method for dynamic adjustment of a database model provided by one embodiment of the invention;
FIG. 2 is a flowchart illustrating optimization of a data processing procedure in a method for dynamic adjustment of a database model according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating optimization of a data processing procedure in a method for dynamic adjustment of a database model according to another embodiment of the present invention;
FIG. 4 is a flow chart of a time series distribution result obtained in the method for dynamically adjusting a database model according to an embodiment of the present invention;
FIG. 5 is a flow chart of partitioning managed data in a method for dynamic adjustment of a database model according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating data tuning of a database model in a method for dynamic tuning of a database model according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating a resource distribution result obtained in a method for dynamically adjusting a database model according to another embodiment of the present invention;
FIG. 8 is a schematic diagram of a device for dynamically adjusting a database model according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It should be noted that although functional blocks are partitioned in a schematic diagram of an apparatus and a logical order is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the partitioning of blocks in the apparatus or the order in the flowchart. The terms first, second and the like in the description and in the claims, as well as in the drawings described above, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
In the related art, management and analysis for mass data are usually based on a database model, and modeling of the database is based on rule-based model driving, however, the database model constructed by the method is generally not suitable for management of various types of data. It can be understood that a database model built based on a rule-driven model is generally not suitable for use in various scene environments due to differences in the managed data, such as data amount, data type, and the like, which may cause differences in the database model with respect to the data processing procedures in the managed data. For example, in managing network element performance data of multiple standards, if a database model is constructed based on rule-based model drive (that is, the database model can be understood as a database model constructed based on expert experience) and the database model is directly used as a reference model, there is a deviation between the data and the model during operation, which may result in that the database may not perform well, and the application system in the managed data may have low efficiency, and may not be reasonable in resource usage and configuration.
Based on the above, the invention provides a method, a device and a storage medium for dynamically adjusting a database model, which enable the database model after being optimized and adjusted to have better performance by dynamically adjusting managed data stored in the database model, and are suitable for management and analysis of various types of data, thereby solving the problem that the database cannot exert better performance due to deviation existing in the data and the model in the operation period in the related technology.
It will be appreciated that the present invention optimizes managed data stored in the database model primarily from two phases. The first stage is to optimize the data processing process corresponding to the managed data stored in the database model; and in the second stage, the database model is subjected to data adjustment for optimization in the aspect of data processing efficiency of managed data.
Specifically, the embodiment of the invention constructs a database model in a model-driven manner, and the database model can be used as a reference model. It will be appreciated that the database model (i.e. the reference model) may be derived by:
a network element performance model is established which includes model elements (i.e., network element performance data), it being understood that the relationships of the model elements may be established from a business perspective. The model elements may specifically include network element standards, managed objects, measurement types, indexes, counters, table definitions corresponding to a database, and the like, which is not specifically limited in this embodiment; it is to be understood that the above-mentioned index may be a network element performance index. Since the model elements in the network element performance model are not linked to the database model to be built, after the network element performance model is built, the database model needs to be built in a model-driven manner. That is, model elements in the network element performance model are mapped into database objects by analyzing the network element performance model. It is understood that the network element performance model may be generated based on expert experience, or by importing a model trained from the same type of network as the network element performance model.
It is understood that database objects include, but are not limited to: the database may be a database, a logic sub-library, a database table, a data table, a partition, a view, an index, a custom function, or the like, which is not specifically limited in this embodiment. For example, the model elements may be defined by corresponding table definitions of the database to facilitate mapping the model elements to database objects in the database model. For example, in some embodiments, the mapping relationship is established by defining the network element system in the model element in a certain database or a certain data table in the database model. It can also be understood that, since Extract-Transform-Load (ETL), query analysis, and historical Data cleaning of managed Data (such as network element performance Data) will be involved in the process of constructing and maintaining the database model, mapping the managed Data into the database model is a Data Definition Language (DDL) operation on database objects, such as construction and maintenance of a logic sub-library, a Data table, an index, a custom function, a view, and a partition; alternatively, mapping to a database model is a Data Management Language (DML) operation on a database object, such as adding, querying, deleting, or updating a Data table. It can be appreciated that the custom function in the database object can effectively help the database model process data, for example, help managed data to perform conversion operation so as to be stored in a data table. For a view, a data display manner can be understood, for example, a data table includes one thousand fields, and a user only focuses on one hundred fields at a certain time, and display data corresponding to the one hundred fields can be established through the view.
It will be appreciated that managed data of embodiments of the present invention includes database objects. The database model is built according to the database objects by taking the database objects as the starting points of the database model.
Therefore, the invention further constructs a database model through the network element performance model, and after the database model is used as a reference model, the database model is optimized in the data processing process and adjusted by adopting the dynamic database model adjusting method, so that the optimized and adjusted database model is obtained, and better performance can be exerted.
It should be noted that, in the embodiment of the present invention, the database model and the related specific application scenario are further constructed by establishing the network element performance model, which is for more clearly explaining the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided by the embodiment of the present invention. Those skilled in the art will appreciate that there is a wide variety of data types in wireless communication technologies, and that management and analysis of data for different types is typically based on a database model. The present embodiment proposes to construct a database model based on a network element performance model, specifically for the application scenario of management and analysis of network element performance data of multiple systems, but is not limited thereto. The technical scheme provided by the embodiment of the invention is also applicable to similar technical problems.
Based on the database model constructed as the reference model, various embodiments of the method for dynamically adjusting the database model are provided.
The embodiments of the present invention will be further explained with reference to the drawings.
The embodiment of the invention particularly provides a method for dynamically adjusting a database model. As shown in fig. 1, fig. 1 is a flowchart of a method for dynamically adjusting a database model according to an embodiment of the present invention, where the method for dynamically adjusting a database model includes, but is not limited to, the following steps:
s100, optimizing a data processing process corresponding to managed data stored in a database model to obtain current measurement index data corresponding to the optimized data processing process;
step S200, performing data analysis on the current measurement index data to obtain an analysis result;
step S300, driving a database model to adjust data according to the analysis result;
and step S400, optimizing the database model after data adjustment to update the current measurement index data until the current measurement index data is the same as the preset target measurement index data.
The data processing process of the managed data can be restrained by optimizing the data processing process corresponding to the managed data stored in the database model, so that the data processing process of the managed data can be kept in a continuously optimized state.
After the data processing process of the managed data is optimized, the corresponding current measurement index data is also changed; therefore, after the data processing process is optimized, the current measurement index data corresponding to the optimized data processing process can be obtained through calculation. And then, performing data analysis on the current measurement index data to obtain an analysis result. It can be understood that, in this embodiment, on the basis that the constructed database model is used as the reference model, the mechanism for dynamically adjusting the database model is driven by acquiring the current metric index data corresponding to the optimized data processing process and performing continuous machine learning and analysis on the current metric index data to establish an analysis result of the current metric index data corresponding to the data processing process in the database model operation period. The database model can be driven to carry out data adjustment according to the analysis result, so that managed data stored in the database model can be dynamically adjusted. And then, performing the optimization processing on the database model after the data adjustment, namely returning to the step S100 to update the current measurement index data until the current measurement index data is the same as the preset target measurement index data, so that the database model after the data adjustment is optimized and adjusted can be obtained.
It can be understood that the data adjustment of the database model may be set as program automatic adjustment or manual adjustment, so as to drive the database model to perform data adjustment through the analysis result, and verify the quality of the database model by obtaining the current measurement index data corresponding to the data processing process, so that the current measurement index data corresponding to the managed data in the optimized and adjusted database model is optimal.
It should be noted that the database model that is finally optimized and adjusted can be applied to the same type of scene environment and can also be applied to various personalized scenes. For example, for the same type of scene environment, it can be understood that the data types of the managed data are the same, and/or the data amount of the managed data is the same. For the database objects in the managed data, assuming that the data types of the managed data are the same, such as 5G, and the data volumes corresponding to the database objects in the managed data are 2000, the adopted database model is also at the management level corresponding to 2000 data volumes; and the optimized and adjusted database model copy is also applicable to other scene environments with 2000 data volumes and the same data types in other areas. In some embodiments, the management level of the database model may be differentiated by the size of the data volume, for example, in a broad sense, the management level of the database model may be: 2000 data volumes are one management level, and 2 ten thousand data volumes are the other management level; alternatively, in a narrow sense, 2000 data volumes are one management level, and 2001 data volumes are the other management level.
For another example, for various personalized scenes, due to the diversity of the data types and data volumes of the managed data, the database model can be optimally adjusted according to the difference of the data types and/or the difference of the data volumes of the managed data, so as to be suitable for the personalized scenes. For the data types of the managed data, for example, for different types of mobile communication systems such as 2G/3G/4G/5G, etc., there will be differences between the corresponding database models, and after performing corresponding personalized optimization adjustment on the database models, the finally optimized and adjusted database models can be applied to the mobile communication systems. Or, the managed data has the same data type, but the data volumes corresponding to the database objects in the managed data are different, and as for the 4G mobile communication system, the data volumes corresponding to the database objects in the managed data at different local points are different, for example, the number of data tables is different, and therefore, the corresponding database models are also different.
By the method for dynamically adjusting the database model, the database model has good performance, can be suitable for management and analysis of various types of data, and solves the problem that the database cannot exert good performance due to deviation existing in the data and the model in the operation period in the related technology.
It should be noted that the managed data stored in the database model is not limited to the network element performance data, but may be other performance data, including but not limited to: and all the managed system corresponding to the measurement index data.
As shown in fig. 2, fig. 2 is a flowchart of an optimization process for a data processing procedure according to an embodiment of the present invention. It is understood that, the optimization process performed in step S100 on the data processing procedure corresponding to the managed data stored in the database model includes, but is not limited to, the following steps:
step S110, obtaining the algorithm complexity corresponding to the data processing process, and optimizing the algorithm complexity which does not accord with the first rule; the first rule comprises that the algorithm complexity is less than or equal to a preset threshold value.
The embodiment specifically performs optimization processing on the algorithm complexity corresponding to the data processing process based on the first rule.
It is understood that the types of data processing procedures related to managed data stored in the database model are generally more, so in some embodiments, some data processing procedures with higher frequency of use may be extracted for data analysis. Specifically, the data processing procedure includes, but is not limited to: the method comprises the following steps of extraction, conversion, loading, query, summary and supplement summary. Therefore, the algorithm complexity corresponding to the data extraction process, the data conversion process, the data loading process, the data query process, the data summarization process or the data complement summarization process can be optimized based on the first rule, that is, the algorithm complexity which does not conform to the first rule is optimized, and it can be understood that the algorithm complexity after the optimization is less than or equal to the preset threshold. In some embodiments, an O (n) function is generally used to characterize the algorithm complexity, i.e., the algorithm complexity may be smaller than or equal to a predetermined O (n). And continuously optimizing the data processing process corresponding to the algorithm complexity which does not conform to the first rule so as to reduce the algorithm complexity until the algorithm complexity is less than or equal to O (n), wherein the data processing process of the embodiment can be kept in a continuously optimized state.
As shown in fig. 3, fig. 3 is a flowchart of an optimization process for a data processing procedure according to another embodiment of the present invention. It can be understood that, the optimization of the data processing procedure corresponding to the managed data stored in the database model in step S100 further includes, but is not limited to, the following steps:
and step S120, acquiring the structured query language corresponding to the data processing process, and optimizing the structured query language which does not accord with the second rule.
It should be noted that the second rule includes at least one of the following: the data redistribution or broadcast is less than or equal to a set threshold; the index level is less than or equal to the index level threshold; the number of nested loops is 0; the number of times of overflowing the file is 0; querying the partition involved is less than the partition threshold; the screening rate of the partitions involved in the query is smaller than a screening rate threshold value; the number of data skews is 0; the data was filtered using the function for 0 times.
If the data processing process involves Structured Query Language (SQL), the embodiment needs to perform optimization processing on the Structured Query Language corresponding to the data processing process based on the second rule.
It is understood that, based on the above embodiments, the data processing procedure includes but is not limited to: the extraction process, the conversion process, the loading process, the query process, the summarization process, the supplement summarization process and the like, so that the structured query languages corresponding to the extraction process, the conversion process, the loading process, the query process, the summarization process or the supplement summarization process of the data can be optimized based on the second rule respectively, namely the structured query languages which do not accord with the second rule are optimized.
It should be noted that, since the types of structured query languages are usually more, some structured query languages with higher frequency of usage can be extracted for data analysis in some embodiments. The structured query language described above is extracted, for example, by algorithms such as machine learning clustering, classification, and the like.
The execution information corresponding to the structured query language includes but is not limited to: data redistribution or broadcast, index hierarchy, nested loops, overflow files, involved partitions, data skew information, data screening conditions, etc. It will be appreciated that the structured query language described above needs to follow database best practice rules, and therefore needs to be optimized for structured query languages that do not comply with the second rules.
The second rule includes at least one of: the data redistribution or broadcast is less than or equal to a set threshold; the index level is less than or equal to the index level threshold, for example, the index level threshold is 2 levels; the number of nested loops is a threshold number of loops, e.g., a threshold number of loops of 0, which indicates that no nested loops should exist; the number of times of overflowing the file is an overflow number threshold, for example, the overflow number threshold is 0, which indicates that the overflowing file is avoided; for example, for a data table that includes multiple partitions (e.g., 100 partitions), it is usually necessary to set the partition involved in the query to be smaller than the partition threshold in order to avoid the second rule being not met due to too many partitions involved in the query. If all the partitions (for example, 100 partitions) are directly subjected to full-table scanning or 80% of the partitions (for example, 80 partitions) are subjected to scanning in the query process, it indicates that the partitions involved in the query are too many and do not meet the second rule, and the query needs to be optimized until the second rule is met; the screening rate of the partition related to the query is smaller than the screening rate threshold, for example, the screening rate of the partition related to the query should be low, and when the screening rate is too high, the current measurement index data corresponding to the managed data is poor, and the data processing efficiency is affected; the data inclination number is an inclination number threshold, for example, the inclination number threshold is 0, which indicates that there is no inclination in the data, and the data inclination information of the embodiment includes the data inclination number; and the number of times of filtering the data by using the function is a filtering number threshold, for example, the filtering number threshold is 0, which indicates that the data is not filtered by using the function, and the data filtering condition of the embodiment includes the number of times of filtering the data by using the function, and so on.
In this embodiment, data analysis is performed on the structured query language corresponding to the data processing process, so as to perform optimization processing on the structured query language that does not conform to the second rule, and it can be ensured that the corresponding current measurement index data can be optimized in the same data processing process.
It is understood that, in some embodiments, the algorithm complexity and the structured query language corresponding to the data processing process may also be obtained separately, so as to optimize the algorithm complexity and the structured query language separately.
It will also be appreciated that, in some embodiments, the structured Query Language generally includes a Data Query Language (DQL), a Data Manipulation Language (DML), a Transaction Processing Language (TPL), a Data Control Language (DCL), a Data Definition Language (DDL), and a pointer Control Language (CCL). The embodiment specifically performs data analysis on a data definition language and a data operation language related to the database model. For example, a data definition language and a data operation language corresponding to managed data stored in the database model are respectively obtained, so as to optimize the data definition language and the data operation language respectively.
It should be noted that the managed data stored in the database model is optimized mainly from two stages. The above embodiment specifically describes the first stage, that is, the optimization processing is performed on the algorithm complexity and the structured query language corresponding to the data processing process corresponding to the managed data.
The second phase of the embodiment is described in detail below, and is to perform data adjustment on the database model for optimization in terms of data processing efficiency of managed data. Specifically, the database model is data-adjusted for optimization in terms of the metric data corresponding to the managed data.
It will be appreciated that the current metric data may include current efficiency index data, current resource usage index data. Namely, the measurement index data includes efficiency index data and resource use index data.
When the current metric index data includes the current efficiency index data, the embodiment of the present invention is specifically described as follows:
as shown in fig. 4, fig. 4 is a flowchart for obtaining a time-series distribution result according to an embodiment of the present invention. In step S200, the data analysis is performed on the current measurement index data to obtain an analysis result, which includes, but is not limited to, the following steps:
step S210, performing data analysis on the current efficiency index data in a time dimension to obtain a time series distribution result.
It is understood that, in some embodiments, the current efficiency index data corresponding to the data processing procedure can be characterized in a time series manner. Therefore, in the embodiment, a time-series distribution result can be obtained by performing data analysis on the current efficiency index data in the time dimension. The purpose of this setting is to facilitate further analysis of the time series distribution result to obtain the regular change corresponding to the time series distribution result. It is understood that, in some embodiments, the regular change corresponding to the time-series distribution result is associated with the preset partition range corresponding to the managed data stored in the database model. It can be understood that, by performing data analysis on the current efficiency index data in the time dimension, different machine learning algorithms can be adopted to obtain time series distribution results conveniently.
As shown in fig. 5, it can be understood that, in an embodiment, before performing the optimization process on the data processing procedure corresponding to the managed data stored in the database model in step S100, the method for dynamically adjusting the database model includes, but is not limited to, the following steps:
and S101, partitioning managed data stored in the database model according to the corresponding preset partition range.
It can be understood that, before step S100, a database model needs to be constructed in a rule model driven manner, and the database model can be used as a reference model, so that it can perform corresponding partitioning on managed data stored in the database model according to a preset partition range. So set up for the benchmark model can be based on predetermineeing the subregion scope and carry out the subregion to managed data, so that raise the efficiency. Then, the invention can specifically optimize the database model by performing data adjustment through the following embodiments:
optimizing algorithm complexity and/or structured query language in a data processing process corresponding to managed data stored in a database model; then, collecting current measurement index data corresponding to the optimized data processing process, wherein the current measurement index data corresponds to the optimized data processing process of the database model in the operation period; then, performing data analysis on the current efficiency index data on a time dimension to obtain a time series distribution result, and driving a database model to perform data adjustment according to the time series distribution result; and optimizing the database model after data adjustment to update the current efficiency index data until the current efficiency index data is the same as the preset target efficiency index data. And performing data adjustment on the database model according to the time series distribution result corresponding to the current efficiency index data so as to realize dynamic adjustment on managed data in the database model, and facilitating training to obtain an optimal database model.
Specifically, in one embodiment, as shown in fig. 6, the database model is driven to perform data adjustment according to the analysis result in step S300, which includes but is not limited to the following steps:
step S310, according to the time sequence distribution result, partition adjustment is carried out on a preset partition range to obtain an adjustment range;
and step S320, performing data adjustment according to managed data stored in the adjustment range driving database model.
It is understood that the time-series distribution result of the present embodiment is mainly influenced by the long-term tendency factor T, the seasonal variation factor S, the periodic variation factor (also referred to as cyclic variation factor) C, the irregular variation factor I, and the like. The long-term trend factor T represents the general variation trend formed by the action of certain fundamental factors in a longer period; the seasonal variation factor S represents a periodic fluctuation of a fixed length and amplitude formed under the influence of seasonal variation; the periodic variation factor C represents fluctuation in which fluctuation is variable due to various factors; the irregular variation factor I represents an irregularly following variation.
It should be noted that, in this embodiment, the irregular variation factor I needs to be eliminated to extract the long-term trend factor T, the seasonal variation factor S, and the periodic variation factor C; performing partition adjustment on a preset partition range according to a time sequence distribution result reflected by the long-term trend factor T, the seasonal variation factor S and the periodic variation factor C to obtain an adjustment range; and then, driving the managed data stored in the database model to perform data adjustment according to the adjustment range so as to dynamically adjust the managed data in the database model, such as dynamically adjusting the number of logical sub-libraries in the database model, the range of data table partitions and the like according to the adjustment range. And finally, carrying out optimization processing on the algorithm complexity and/or the structured query language in the data processing process corresponding to the managed data stored in the database model to the database model after data adjustment so as to update the current efficiency index data until the current efficiency index data is the same as the preset target efficiency index data.
When the current metric index data includes the current resource usage index data, the embodiment of the present invention is described specifically as follows:
as shown in fig. 7, the data analysis of the current metric index data in step S200 to obtain an analysis result specifically includes, but is not limited to, the following steps:
step S220, performing data analysis on the current resource use index data to obtain a resource distribution result.
It is to be appreciated that the resource usage indicator data includes at least one of: application resource usage, database resource usage, storage device input/output resource usage. It will be appreciated that the resource usage data may include: memory occupation data, central Processing Unit (CPU) utilization data, input/Output (I/O) response time data, input/Output read-write traffic data, network traffic data, and the like.
In the embodiment, the data processing process corresponding to managed data stored in a database model is optimized, and current resource use index data corresponding to the optimized data processing process is collected; and finally, optimizing the database model after data adjustment to update the current resource use index data until the current resource use index data is the same as the preset target resource use index data. In this embodiment, the quality of the database model is verified by collecting the current resource usage index data, so that the current resource usage index data corresponding to the managed data in the database model that is finally optimized and adjusted is optimal, and if the minimum resource usage is obtained, it indicates that the database model completes optimization and adjustment at this time.
It should be noted that, in an embodiment, when the current measurement index data includes current efficiency index data and current resource usage index data, the current measurement index data is optimized by performing optimization processing on algorithm complexity and/or structured query language in a data processing process corresponding to managed data stored in a database model, and respectively acquiring current efficiency index data and current resource usage index data corresponding to the optimized data processing process; in the second stage of optimization, on one hand, data analysis is performed on the current efficiency index data in the time dimension to obtain a time series distribution result; on the other hand, performing data analysis on the current resource use index data to obtain a resource distribution result; then, according to the time sequence distribution result, carrying out partition adjustment on a preset partition range corresponding to managed data stored in the database model to obtain an adjustment range; driving managed data stored in the database model to perform data adjustment according to the adjustment range and the resource distribution result; and carrying out the optimization processing again on the database model after the data adjustment so as to respectively update the current efficiency index data and the current resource use index data until the current efficiency index data is the same as the preset target efficiency index data and the current resource use index data is the same as the preset target resource use index data. It can be understood that, in this embodiment, the quality of the database model is verified by collecting current efficiency index data, current resource usage index data, and the like corresponding to the data processing process, so as to obtain an optimal database model through training.
The method for dynamically adjusting the database model of the present invention is described in detail below by taking the example that the managed data includes network element performance data.
In this embodiment, the database model is further constructed by establishing a network element performance model, and the constructed database model may be used as a reference model. Specifically, aiming at the network element performance data, the invention adjusts and optimizes the data of the database model (namely, the reference model) from the aspect of the measurement index data corresponding to the network element performance data. It is understood that the network element performance data includes, but is not limited to: network element system of management, and indexes and counters of managed objects under different systems.
Specifically, optimization processing is performed on algorithm complexity and a structured query language in a data processing process corresponding to network element performance data stored in a database model, and current measurement index data corresponding to the data processing process after the optimization processing is acquired, that is, current efficiency index data and current resource usage index data corresponding to the data processing process in the operation period of the database model are acquired; the efficiency index data may further include a network element performance index, a frequency of use of a counter, and the like. By the arrangement, the current efficiency index data and the current resource use index data corresponding to the same data processing process can be optimized. For example, in some embodiments, the current efficiency index data may be represented by a current time-consuming duration (or a current execution time) corresponding to the data processing, and when the current efficiency index data is optimized, the data processing efficiency may be effectively improved; and if the current resource usage index data is optimized (for example, the optimized current resource usage rate is reduced), the resource usage can be guaranteed to be in the corresponding target range.
Then, performing data analysis on the current efficiency index data on a time dimension to obtain a time series distribution result; and performing data analysis on the current resource use index data to obtain a resource distribution result. Machine learning can be respectively carried out on the current efficiency index data and the current resource use index data corresponding to the data processing process so as to obtain corresponding analysis results. It is understood that the present embodiment drives the database model to perform data adjustment through the time-series distribution result and the resource distribution result, for example, drives the number of logical sub-libraries in the database model, the definition of the data table, the definition of the range or index of the data table partition, and so on. And returning the database model after data adjustment to the optimization processing to update the current efficiency index data and the current resource use index data until the current efficiency index data is the same as the preset target efficiency index data and the current resource use index data is the same as the preset target resource use index data.
Therefore, through continuous optimization and adjustment of the database model, the optimal database model corresponding to the managed data, such as the network element performance data, can be obtained through training.
Referring to fig. 8, it can be understood that the method for dynamically adjusting a database model according to the embodiment of the present invention specifically corresponds to an apparatus for dynamically adjusting a database model. The device includes but is not limited to: a network element performance data extraction, conversion and loading module 100, a network element performance data analysis module 200, a data processing process optimization module 300, a database model optimization module 400, a database management and maintenance module 500 and an analysis database module 600.
The network element performance data extraction, conversion and loading module 100 is configured to acquire network element performance data, perform data cleaning and data conversion on the network element performance data, and send the network element performance data after the data cleaning and data conversion to the analysis type database module 600;
the network element performance data analysis module 200 is configured to perform data statistics and data analysis on the network element performance data, and send the network element performance data after the data statistics and data analysis to the analysis-type database module 600;
the network element performance data extraction, conversion and loading module 100 and the network element performance data analysis module 200 are further configured to collect algorithm complexity and structured query language in the data processing process, and send the algorithm complexity and the structured query language to the data processing process optimization module 300; the database model optimization module 400 is further used for acquiring current measurement index data corresponding to the optimized data processing process and sending the current measurement index data to the database model optimization module; the current measurement index data comprises current efficiency index data and current resource use index data;
the data processing process optimizing module 300 is configured to obtain the algorithm complexity and the structured query language sent by the network element performance data extracting, converting and loading module 100 and the network element performance data analyzing module 200, optimize the algorithm complexity that does not comply with the first rule, and optimize the structured query language that does not comply with the second rule; the database model is also used for maintaining database objects in the database model in a model driving mode, wherein the database objects comprise a database, a data table, partitions, views, indexes and custom functions;
the database model optimization module 400 is configured to obtain current metric index data corresponding to the data processing process sent by the network element performance data extraction, conversion and loading module 100 and the network element performance data analysis module 200, perform data analysis on the current metric index data to obtain an analysis result, and send the analysis result to the database management and maintenance module 500;
the database management and maintenance module 500 is used for acquiring the analysis result sent by the database model optimization module 400 and driving the database model to perform data adjustment according to the analysis result;
the analysis type database module 600 is configured to obtain the network element performance data after the data cleaning and the data conversion sent by the network element performance data extraction, conversion and loading module 100, obtain the network element performance data after the data statistics and the data analysis sent by the network element performance data analysis module 200, and perform data storage and data management on the network element performance data; the analysis type database module 600 includes a GBase database module and a greenplus database module.
It can be understood that, the network element performance data extraction, conversion and loading module 100 can acquire network element performance data corresponding to a network element, for example, network element performance data from a File Transfer Protocol (FTP) server is acquired; or acquiring Network element performance data from a Simple Network Management Protocol (SNMP) server; or network element performance data from a Socket server, etc.
It can be understood that the device of this embodiment may further include an application running module, where the application running module is configured to support the corresponding application running when the database model performs data adjustment; the application running module comprises a blade server module and a rack server module.
It can be understood that the network element performance data extraction, conversion, and loading module 100 and the network element performance data analysis module 200 can collect current efficiency index data and current resource usage index data corresponding to the data processing process from the aspect of the data processing efficiency of the network element performance data. For example, the data processing process includes: the extraction process, the conversion process, the loading process, the query process, the summarization process and the supplement summarization process, and then the current efficiency index data corresponding to the data processing process can be expressed as: the current execution time corresponding to the extraction process, the conversion process, the loading process, the query process, the summarization process and the supplement summarization process; the current resource usage index data corresponding to the data processing procedure can be expressed as: the current application resource is, for example, occupied by the current central processing unit or occupied by the current memory. In some embodiments, the network element performance data extraction, conversion, and loading module 100 and the network element performance data analysis module 200 can also collect, from the aspect of data analysis, the frequency of queries corresponding to the database tables and columns, or the distribution of performance data.
It can be understood that the database model optimization module 400 performs data analysis on the current measurement index data, specifically, performs data analysis on the current efficiency index data through a machine learning algorithm in a time dimension to obtain a time series distribution result, and the time series distribution result is mainly affected by factors such as a long-term trend factor T, a seasonal variation factor S, a periodic variation factor C, and an irregular variation factor I. For example, the current execution time corresponding to the summarizing process, the loading process or the querying process of the network element performance data may be analyzed. In some embodiments, after the irregular variation factor I needs to be eliminated, a correlation between a time series distribution result (for example, long-term trend factor T/seasonal variation factor S/periodic variation factor C) reflected by current efficiency index data (for example, current execution time) corresponding to a data processing process (for example, a summarizing process) corresponding to managed data (for example, network element performance data) and managed objects (for example, a logic sub-library and a data table partition) in the database model is established.
For example, for a 2G/3G/4G/5G co-managed mobile communication system, the number of indicators (e.g., network element performance indicators) and counters included in the system is usually tens of thousands, and different users can obtain and use the indicators and counters concerned by themselves through the query analysis function. For example, for a query of network element performance indicators and counters, mapping into a database model is a query analysis of database objects (e.g., database tables, etc.). The usage frequency of the database object (for example, a database table) in the database model is also characterized in a time series manner. Therefore, by analyzing the current measurement index data corresponding to the network element performance data to obtain an analysis result, the association between the data storage and the data query of the network element performance data can be obtained, so as to provide a basis for data adjustment of the database model, for example, a basis for structure adjustment of a data table, operation frequency adjustment of data summarization, creation adjustment of an intermediate table, and the like.
It can also be understood that, the database model optimization module 400 can also perform optimization processing on the database model after data adjustment to update the current efficiency index data until the current efficiency index data is the same as the preset target efficiency index data, that is, by continuous data analysis and data adjustment, the current efficiency index data corresponding to the data processing process after the optimization processing can be ensured to continuously judge whether the current efficiency index data is better or not, that is, whether the current efficiency index data is the same as the preset target efficiency index data, under the condition that the data processing function meets the service requirement and the current resource usage index data is the target resource usage index data (if the resource usage is not increased). And (5) training to obtain an optimal database model through iterative adjustment. It is understood that the database model optimization module 400 verifies the quality of the database model through the current efficiency index data and the current resource usage index data, for example, the final current resource usage index data corresponding to the optimal database model reflects the least resource usage, the current efficiency index data reflects the optimal execution time or time duration, and so on.
The database management and maintenance module 500 performs data adjustment by driving the database model according to the analysis result; specifically, continuous data adjustment is carried out on the database model based on rules corresponding to machine learning; such as adjusting the number of logical sub-libraries, the extent of the data table partitions, the structure of the data table, the frequency of operations of data summarization, the creation of intermediate tables, and so forth.
The method for dynamic adjustment of the database model according to the invention is described below in a specific embodiment.
Example 1: the present embodiment describes a process of adjusting data of a preset partition range corresponding to a database table, which includes but is not limited to:
(1) The current measurement index data corresponding to the optimized data processing process is collected through the network element performance data extraction, conversion and loading module 100 and the network element performance data analysis module 200, and the current measurement index data is reported to the database model optimization module 400. The current measurement index data of this embodiment includes current efficiency index data and current resource usage index data, and both the current efficiency index data and the current resource usage index data have time characteristics. It can be understood that the efficiency index data includes summary time-consuming duration, loading time-consuming duration, query time-consuming duration, and the like; the resource usage index data includes application resource usage (e.g., cpu occupancy/memory occupancy), database resource usage (e.g., cpu occupancy/memory occupancy), storage device Input/Output resource usage (e.g., input/Output Per Second (IOPS)/Input/Output wait (IOWait)), and so on.
(2) The database model optimization module 400 performs data analysis on the current efficiency index data in the time dimension to obtain a time series distribution result, the time series distribution result is mainly influenced by factors such as a long-term trend factor T, a seasonal variation factor S, a periodic variation factor C and an irregular variation factor I, and after the irregular variation factor I is eliminated, a time series distribution result corresponding to the long-term trend factor T/the seasonal variation factor S/the periodic variation factor C is obtained. It can be understood that the long-term trend factor T specifically shows an average trend, while the seasonal variation factor S is kept consistent with the first preset partition range corresponding to the original data table, and the periodic variation factor C is kept consistent with the second preset partition range corresponding to the summary table.
(3) The summary time duration is specifically defined as a variable w; it will be appreciated that the maximum value Max (w) of w is less than the service-defined time-consuming threshold λ; the range of the preset partition is defined as a variable P, and the value range of P is [1,30 ]](ii) a Wherein, the first preset partition range corresponding to the original data table is defined as a variable P 1 The first adjustment range corresponding to the original data table is defined as a variable p 1 Defining the second preset partition range corresponding to the summary table as a variable P 2 The second adjustment range corresponding to the summary table is defined as the variable p 2 。P、P 1 、P 2 、p 1 、p 2 The units of (A) are days. In order to obtain the optimal w value corresponding to the database modelAnd (3) driving the database management and maintenance module 500 to perform data adjustment on the database model by the database model optimization module 400 based on the time series distribution result obtained in the step (2), so as to perform optimization adjustment on the database model. It can be understood that the original data table corresponds to a first preset partition range P 1 The setting is usually done based on experience, and is usually large. Therefore, if the original data table corresponds to the first predetermined partition range P 1 >1, a first preset partition range P corresponding to the original data table can be obtained 1 Performing first partition adjustment to obtain a proper first adjustment range p corresponding to the original data table 1 . Specifically, the first partition adjustment may be set to: p is a radical of 1 ＝P 1 -1; wherein p is 1 Greater than or equal to 1.
(4) Optimizing the database model after data adjustment through the database model optimization module 400 to update the current efficiency index data, and simultaneously assisting to refer to the current resource usage index data, if the usage of the application resources or the total usage of the database resources is not increased, the input/output resources of the storage device are used in the hardware capacity range, or the time-consuming duration corresponding to other data processing processes is not deteriorated, the first adjustment range p corresponding to the original data table can be stored 1 . It will be appreciated that the summary table correspondence may perform the partition adjustment as described above to obtain the appropriate second adjustment range p for the summary table correspondence 2 。
(5) Example 1 illustrates that:
for example, the raw data sheet is partitioned by day and holds managed data at 15 minute granularity, and the summary sheet is partitioned by week and holds managed data at hour granularity. Specifically, managed data in the summary table is obtained by aggregating managed data in the original data table and then loading the managed data in the original data table into the summary table. And performing data analysis on the current efficiency index data corresponding to the summary table in the time dimension to obtain a time series distribution result, wherein the seasonal variation factor S corresponding to the summary table is 1 day, and the periodic variation factor C is 1 week.
Summarizing time sequence distribution results corresponding to time-consuming duration, and mainly presenting two rules:
rule one: when the granularity is observed by days, the summary time duration gradually increases along with the time;
the reason for the occurrence of rule one is: since the original data table is partitioned according to the day, for managed data on the same day, the corresponding data volume of the managed data is gradually increased in the time dimension, so that when the managed data are summarized in hours, the data range corresponding to the read managed data is gradually increased, the reading time is increased, and the summarizing time duration is increased.
Rule two: the summary time duration of the same time period on different days is in a gradual rising trend.
The reason why rule two occurs is: when managed data are summarized and loaded into an hour table, repeated data deletion and insertion operations exist, and the data range related to the deletion operations becomes larger as the data volume corresponding to the managed data is continuously increased when the managed data are summarized, so that the time consumed for deletion is increased; the insertion operation may involve an operation of an index page, and the insertion takes a long time because the index range increases as the amount of data corresponding to the managed data is increased when the managed data is summarized. Therefore, the total time consumption of the same period of different days is in a gradual rising trend.
It is understood that, in the original data table of the embodiment, if the first predetermined partition range P is included in the original data table 1 If =1, the first partition adjustment is not required for the original data table. And for the second preset partition range P corresponding to the summary table 2 If =7, the second preset partition range P may be defined 2 A second partition adjustment is made, which may be set to: p is a radical of formula 2 ＝P 2 -1, to obtain a suitable second adjustment range p corresponding to the summary table 2 In order to ensure that the resource usage index data is within the corresponding target range.
Example 2: this embodiment describes a process of adjusting data of a data table structure and a data summary, which includes but is not limited to:
(1) The current measurement index data corresponding to the data processing process is obtained through the network element performance data analysis module 200, and the current measurement index data is reported to the database model optimization module 400. The measurement index data includes the use frequency of the index (such as the network element performance index) and the counter, the summary time range corresponding to the index (such as the network element performance index) and the counter, the index (such as the network element performance index) and the database table to which the counter belongs.
(2) The database model optimization module 400 performs data analysis on the current measurement index data to obtain analysis results, where the analysis results include hot data distribution results, such as index distribution results for high frequency use and index distribution results for low frequency use; and/or a counter distribution result used at high frequency, a counter distribution result used at low frequency, and the like, so that a corresponding database table and column used at high frequency, a corresponding database table and column used at low frequency, and the like can be further obtained according to the indexes (such as network element performance indexes) and the database table to which the counter belongs.
(3) The database management and maintenance module 500 performs data adjustment on the database model to perform optimization adjustment on the database model. For example, the hour summary of the performance data is applied to the analysis of the performance hour data by the user. For performance data which needs to be subjected to data analysis in the last hour every hour, summarizing the performance data in time according to 1 hour interval, for example, summarizing the performance data every hour and storing the granularity of 15 minutes, namely, the performance data every hour corresponds to the current measurement index data; for a current time (e.g. 16); for performance data that involves little or no hourly data analysis, the frequency of usage of summaries may be reduced, while the summary time range is expanded (e.g., summaries at 8-hour intervals) to improve overall hourly summary efficiency. Performance data that is not of interest to the application may be left out of storage and the database table structure adjusted.
(4) Verifying database model effect, this embodiment is through reducing the use frequency to the storage of performance data and reduction gathering to reduce the use to application system calculation and storage resource, thereby promote the whole efficiency of database model.
Example 3: this embodiment describes a process of performing data adjustment on a database model in the field of database applications, which includes but is not limited to:
(1) And acquiring current measurement index data corresponding to the data processing process, wherein the current measurement index data comprises current efficiency index data and current resource use index data. For example, efficiency index data (such as time-consuming duration) corresponding to data warehousing and data query, database resource usage (such as central processor occupation/memory occupation), storage device input/output resource usage (such as IOPS/IOWait), etc., and the current metric index data has a temporal characteristic.
(2) Because the analytical database is a cluster database constructed by a plurality of logical sub-databases, the number of the logical sub-databases is in a class-direct proportion relation with the IOPS. And performing data analysis on current resource use index data corresponding to the input/output resource use of the storage device in a time dimension to judge whether the IOPS and the IOWait are in a hardware capability range. If the IOPS generated in the data processing process does not reach or exceed the hardware capability range, the IOPS can be increased or decreased by increasing or decreasing the number of the logical sub-libraries, for example, when the IOPS generated in the data processing process does not reach the hardware capability range, the number of the logical sub-libraries can be increased to increase the IOPS; while IOWait should be within the corresponding target range, i.e. within a reasonable range. The IOPS and IOWait of the present embodiment are within the corresponding target range, and may play a positive role in the efficiency of the data processing process.
(3) The current efficiency index data corresponding to the data query process is obtained, and the database table partition/sub-table in the database model is driven to perform data adjustment, so that the database model can exert the maximum performance, and the implementation principle of the method can refer to embodiment 1.
It can be understood that the method for dynamically adjusting the database model of the embodiment is active system maintenance, rather than problem-driven or manual maintenance, and can effectively improve the operation and maintenance efficiency of the system.
By carrying out data analysis on the network element performance index and the use frequency of the counter, a hotspot data distribution result can be obtained, the structure of a database table and the storage of data can be adjusted, the data summarizing and analyzing efficiency is improved, and the resource use index data can be ensured to be in a corresponding target range.
The database model is constructed and maintained in a model driving mode, so that the maintainability of the database model can be improved, the version upgrading efficiency is improved, and the waiting time is reduced.
Through the model sharing property, the trained database model can be exported and copied to the same type of system, and the training cost of the database model is reduced.
In addition, an embodiment of the present invention also provides an apparatus, including: a memory, a processor, and a computer program stored on the memory and executable on the processor.
The processor and memory may be connected by a bus or other means.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and these remote memories may be connected to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The non-transitory software programs and instructions required to implement the method for dynamic adjustment of database models of the above-described embodiments are stored in the memory, and when executed by the processor, perform the method for dynamic adjustment of database models of the above-described embodiments, for example, performing method steps S100 to S400 in fig. 1, method step S110 in fig. 2, method step S120 in fig. 3, method step S210 in fig. 4, method step S101 in fig. 5, method steps S310 to S320 in fig. 6, and method step S220 in fig. 7 described above.
The above described embodiments of the device are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may also be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Furthermore, an embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions, which are executed by a processor or controller, for example, by a processor in the above-mentioned device embodiment, and may cause the above-mentioned processor to execute the method for dynamically adjusting the database model in the above-mentioned embodiment, for example, to execute the above-mentioned method steps S100 to S400 in fig. 1, method step S110 in fig. 2, method step S120 in fig. 3, method step S210 in fig. 4, method step S101 in fig. 5, method steps S310 to S320 in fig. 6, and method step S220 in fig. 7.
It will be understood by those of ordinary skill in the art that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, or suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
While the preferred embodiments of the present invention have been described, the present invention is not limited to the above embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and such equivalent modifications or substitutions are to be included within the scope of the present invention defined by the appended claims.

Claims (11)

1. A method for dynamically adjusting a database model, comprising:
optimizing the data processing process corresponding to the managed data stored in the database model to obtain current measurement index data corresponding to the optimized data processing process;
performing data analysis on the current measurement index data to obtain an analysis result;
driving the database model to carry out data adjustment according to the analysis result;
and performing the optimization processing on the database model after data adjustment to update the current measurement index data until the current measurement index data is the same as preset target measurement index data.
2. The method of claim 1, wherein the database model is derived by:
establishing a network element performance model, wherein the network element performance model comprises model elements;
parsing the network element performance model to map the model elements to database objects, wherein the managed data comprises the database objects;
and constructing the database model according to the database object.
3. The method according to claim 1, wherein the optimizing the data processing procedure corresponding to the managed data stored in the database model comprises:
acquiring algorithm complexity corresponding to a data processing process, and optimizing the algorithm complexity which does not accord with a first rule; wherein the first rule comprises that the algorithm complexity is less than or equal to a preset threshold.
4. The method according to claim 1, wherein the optimizing the data processing procedure corresponding to the managed data stored in the database model comprises:
acquiring a structured query language corresponding to the data processing process, and optimizing the structured query language which does not accord with a second rule;
wherein the second rule comprises at least one of:
the data redistribution or broadcast is less than or equal to a set threshold;
the index level is less than or equal to the index level threshold;
the number of nested loops is 0;
the number of times of overflowing the file is 0;
querying the partition involved is less than the partition threshold;
the screening rate of the partitions involved in the query is smaller than a screening rate threshold value;
the number of data skews is 0;
the data was filtered using a function for 0 times.
5. The method of any of claims 1 to 4, wherein the current metric data comprises current efficiency index data;
the data analysis of the current measurement index data to obtain an analysis result includes:
and carrying out data analysis on the current efficiency index data on a time dimension to obtain a time series distribution result.
6. The method of claim 5, wherein before performing optimization processing on the data processing procedure corresponding to the managed data stored in the database model, the method further comprises:
and partitioning the managed data stored in the database model according to the corresponding preset partition range.
7. The method of claim 6, wherein driving the database model for data adjustment based on the analysis result comprises:
according to the time sequence distribution result, carrying out partition adjustment on the preset partition range to obtain an adjustment range;
and driving managed data stored in the database model to perform data adjustment according to the adjustment range.
8. The method of claim 7, wherein the optimizing the data-adjusted database model to update the current metric data until the current metric data is the same as a preset target metric data comprises:
and performing the optimization processing on the database model after data adjustment to update the current efficiency index data until the current efficiency index data is the same as the preset target efficiency index data.
9. The method of any of claims 1 to 4, wherein the current metric data comprises current resource usage metric data;
the data analysis of the current measurement index data to obtain an analysis result includes:
and performing data analysis on the current resource use index data to obtain a resource distribution result.
10. An apparatus, comprising: memory, processor and computer program stored on the memory and executable on the processor, the processor implementing a method for dynamic adjustment of a database model according to any one of claims 1 to 9 when executing the computer program.
11. A computer-readable storage medium storing computer-executable instructions for performing the method for dynamic adjustment of a database model of any one of claims 1 to 9.
CN202110814308.XA 2021-07-19 2021-07-19 Method, device and storage medium for dynamically adjusting database model Pending CN115640274A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110814308.XA CN115640274A (en) 2021-07-19 2021-07-19 Method, device and storage medium for dynamically adjusting database model
PCT/CN2022/091510 WO2023000766A1 (en) 2021-07-19 2022-05-07 Method for dynamically adjusting database model, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110814308.XA CN115640274A (en) 2021-07-19 2021-07-19 Method, device and storage medium for dynamically adjusting database model

Publications (1)

Publication Number Publication Date
CN115640274A true CN115640274A (en) 2023-01-24

Family

ID=84939426

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110814308.XA Pending CN115640274A (en) 2021-07-19 2021-07-19 Method, device and storage medium for dynamically adjusting database model

Country Status (2)

Country Link
CN (1) CN115640274A (en)
WO (1) WO2023000766A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115858699A (en) * 2023-02-28 2023-03-28 北京仁科互动网络技术有限公司 Data warehouse construction method and device, electronic equipment and readable storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116842223B (en) * 2023-08-29 2023-11-10 天津鑫宝龙电梯集团有限公司 Working condition data management method, device, equipment and medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060085484A1 (en) * 2004-10-15 2006-04-20 Microsoft Corporation Database tuning advisor
US8180762B2 (en) * 2005-12-13 2012-05-15 International Business Machines Corporation Database tuning methods
CN100543746C (en) * 2007-03-16 2009-09-23 华为技术有限公司 The system and method for a kind of fulfillment database system Automatic Optimal
JP6669571B2 (en) * 2016-04-19 2020-03-18 株式会社シスバンク Tuning apparatus and method for relational database
US11327988B2 (en) * 2019-06-20 2022-05-10 International Business Machines Corporation Dynamically self-indexing database-management system
CN111339066B (en) * 2020-05-20 2020-09-04 腾讯科技（深圳）有限公司 Database optimization method and device, electronic equipment and computer-readable storage medium
CN112231236A (en) * 2020-12-14 2021-01-15 广东睿江云计算股份有限公司 Method for testing database performance

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115858699A (en) * 2023-02-28 2023-03-28 北京仁科互动网络技术有限公司 Data warehouse construction method and device, electronic equipment and readable storage medium

Also Published As

Publication number Publication date
WO2023000766A1 (en) 2023-01-26

Similar Documents

Publication Publication Date Title
US10977234B2 (en) Combining compressed and uncompressed data at query time for efficient database analytics
US10073888B1 (en) Adjusting partitioning policies of a database system in view of storage reconfiguration
US10725995B2 (en) Automatically revising synopsis table structure
US11132341B2 (en) Automated information lifecycle management using low access patterns
US9892150B2 (en) Unified data management for database systems
US20190278783A1 (en) Compaction policy
EP2811792B1 (en) A method for operating a mobile telecommunication device
EP3757815A1 (en) Data processing method and device, storage medium, and electronic device
WO2023000766A1 (en) Method for dynamically adjusting database model, device and storage medium
EP3401807B1 (en) Synopsis based advanced partition elimination
CN107004016B (en) Efficient data manipulation support
US20160328445A1 (en) Data Query Method and Apparatus
US20180260439A1 (en) Grouping in analytical databases
JP2008225575A (en) Computer load estimation system and method
CN103336792A (en) Method and device for data partition
CN111309251A (en) Data storage method, system, electronic device and readable storage medium
CN116089414A (en) Time sequence database writing performance optimization method and device based on mass data scene
CN114020713A (en) File merging method and device of log structure merging tree, electronic equipment and storage medium
US20170235779A1 (en) Processing query of database and data stream
CN111125158B (en) Data table processing method, device, medium and electronic equipment
CN111723089B (en) Method and device for processing data based on column type storage format
US20160253591A1 (en) Method and apparatus for managing performance of database
CN117194426A (en) Data partition archiving method, system, equipment and medium
CN117290352A (en) Index establishing method, electronic equipment and computer storage equipment
EP4409425A1 (en) Dataset optimization framework

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination