CN116126859A - Data management method and device, electronic equipment and storage medium - Google Patents
Data management method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116126859A CN116126859A CN202211732680.7A CN202211732680A CN116126859A CN 116126859 A CN116126859 A CN 116126859A CN 202211732680 A CN202211732680 A CN 202211732680A CN 116126859 A CN116126859 A CN 116126859A
- Authority
- CN
- China
- Prior art keywords
- data
- probability
- monitoring information
- target
- adopting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013523 data management Methods 0.000 title claims abstract description 25
- 238000012544 monitoring process Methods 0.000 claims abstract description 73
- 238000012545 processing Methods 0.000 claims abstract description 54
- 230000004044 response Effects 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 30
- 230000006378 damage Effects 0.000 claims description 29
- 230000015654 memory Effects 0.000 claims description 23
- 238000007726 management method Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 238000004064 recycling Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 14
- 238000004891 communication Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 7
- 238000012217 deletion Methods 0.000 description 4
- 230000037430 deletion Effects 0.000 description 4
- 238000009825 accumulation Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data management method, a device, electronic equipment and a storage medium, wherein the method is used for acquiring monitoring information of data in response to execution time of a set task, judging whether the data meets preset operation conditions according to the monitoring information, and processing the data by adopting target operations associated with the operation conditions when the data meets the operation conditions.
Description
Technical Field
The present invention relates to the field of computer software technologies, and in particular, to a data management method, a data management device, an electronic device, and a storage medium.
Background
With the advent of the big data age, the data generation amount and the data storage amount of each industry are greatly improved, and the problem of data accumulation is easily caused, so that how to efficiently manage the accumulated data becomes an important problem to be solved currently.
In the related art, the processing strategy for the related data set under each active duration is generally determined according to a plurality of set active durations, however, when the number of data to be processed in the data set is large under the same active duration, data accumulation in the set is easy to generate based on the mode, so that the processing efficiency is affected.
Disclosure of Invention
The embodiment of the application provides a data management method, a data management device, electronic equipment and a storage medium, which are used for avoiding data accumulation and improving data management efficiency.
In a first aspect, an embodiment of the present application provides a data management method, including:
and responding to the execution time of the set task, and acquiring monitoring information of the data, wherein the monitoring information is used for representing the current attribute of the data.
Judging whether the data meets preset operation conditions or not according to the monitoring information, wherein the operation conditions relate to target operations set for the data.
And processing the data with the target operation in response to the data meeting the operating condition.
And monitoring and recording the processing result of the data, and carrying out life cycle management on the data according to the processing result.
In a second aspect, an embodiment of the present application provides a data management apparatus, including:
and the task execution module is used for responding to the execution time of the set task and acquiring the monitoring information of the data, wherein the monitoring information is used for representing the current attribute of the data.
And the condition judging module is used for judging whether the data meets preset operation conditions according to the monitoring information, wherein the operation conditions are related to target operations set for the data.
And the operation processing module is used for processing the data by adopting the target operation in response to the data meeting the operation condition.
And the monitoring management module is used for monitoring and recording the processing result of the data and managing the life cycle of the data according to the processing result.
In an optional embodiment, the determining, according to the monitoring information, whether the data meets a preset operation condition, where the condition determining module is configured to:
and responding to the monitoring information to meet a set automatic triggering rule, calculating the operation probability of the data corresponding to the target operation, and determining that the data meets a preset operation condition when the operation probability is not smaller than a preset threshold value.
And/or the number of the groups of groups,
and responding to the monitoring information to meet a set custom trigger rule, acquiring at least one custom trigger field set corresponding to the custom trigger rule, and determining that the data meets a preset operation condition when the monitoring information is matched with the at least one custom trigger field, wherein the custom trigger field is used for indicating a target operation environment of the data.
In an alternative embodiment, the calculating the operation probability of the data corresponding to the target operation, the condition judging module is configured to:
and calculating a first operation probability of the data corresponding to the target operation by adopting a first parameter according to the calling date of the data contained in the monitoring information, wherein the first parameter represents the maximum storage time of the data.
And calculating a second operation probability of the data corresponding to the target operation by adopting a second parameter according to the calling times of the data contained in the monitoring information, wherein the second parameter represents the sum of the calling times of the data.
And respectively adopting a first weight and a second weight to carry out weighted summation on the first operation probability and the second operation probability to obtain a weighted summation value, and adopting a preset function and a third operation probability to calculate the operation probability of the data corresponding to the target operation according to the weighted summation value, wherein the sum value of the first weight and the second weight is one, and the third operation probability is calculated according to the number of storage bars of the data.
In an optional embodiment, the calculating, according to the weighted sum, a preset function and a third operation probability, calculates an operation probability of the data corresponding to the target operation, where the condition determining module is configured to:
and calculating a third operation probability of the data corresponding to the target operation by adopting a third parameter according to the number of the data, wherein the third parameter represents the maximum storage record quantity of the data.
And calculating an objective function value corresponding to the weighted sum value according to a preset function, and calculating the objective function value by adopting the third operation probability to obtain the operation probability of the data corresponding to the objective operation.
In an alternative embodiment, the target operation is data archiving, and the processing is performed on the data by adopting the target operation, and the operation processing module is configured to:
filing the file, and storing the data in a file form;
or,
and (5) adopting database archiving, and storing the data into a specified database table.
In an optional embodiment, if the target operation is data destruction, the processing of the data with the target operation is performed, and the operation processing module is configured to:
and adopting a recycle bin to destroy, and migrating the data to a target recycling space.
Or,
and adopting permanent destruction to directly delete the data.
In an alternative embodiment, after the monitoring and recording the processing result of the data, the monitoring management module is further configured to:
and according to the processing result, updating the current data quantity of the data and updating the processed data quantity of the data corresponding to the target operation.
Analyzing the running condition of the data by adopting the current data quantity and the processed data quantity, and displaying the running condition in a preset display interface.
In a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of the data management method of the first aspect described above.
In a fourth aspect, a computer readable storage medium is proposed, comprising program code for causing an electronic device to perform the steps of the data management method of the first aspect described above, when said program code is run on the electronic device.
The technical effects of the embodiment of the application are as follows:
the embodiment of the application provides a data management method, a device, electronic equipment and a storage medium, wherein the method is used for acquiring monitoring information of data in response to execution time of a set task, judging whether the data meets preset operation conditions according to the monitoring information, and processing the data by adopting target operations associated with the operation conditions when the data meets the operation conditions.
Drawings
Fig. 1 is a schematic diagram of a possible application scenario provided in an embodiment of the present application;
fig. 2 is a schematic diagram of a management system according to an embodiment of the present application;
FIG. 3 is a flowchart of a data management method according to an embodiment of the present application;
FIG. 4 is a timing task schematic provided in an embodiment of the present application;
FIG. 5 is a schematic diagram of data archiving provided in an embodiment of the present application;
fig. 6 is a schematic diagram of data destruction according to an embodiment of the present application;
FIG. 7 is a schematic diagram of life cycle monitoring according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a data management device according to an embodiment of the present application;
fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present invention based on the embodiments herein.
It should be noted that "a plurality of" is understood as "at least two" in the description of the present application. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. A is connected with B, and can be represented as follows: both cases of direct connection of A and B and connection of A and B through C. In addition, in the description of the present application, the words "first," "second," and the like are used merely for distinguishing between the descriptions and not be construed as indicating or implying a relative importance or order.
In addition, in the technical scheme, the data are collected, transmitted, used and the like, and all meet the requirements of national related laws and regulations.
A data management method provided in the embodiments of the present application will be described and illustrated in detail below with reference to the accompanying drawings.
Referring to fig. 1, a schematic diagram of a possible application scenario provided in an embodiment of the present application includes: the platform 11 and the server 12 may perform information interaction between the platform 11 and the server 12 through a communication network, where a communication manner adopted by the communication network may include: wireless communication and wired communication.
Illustratively, the platform 11 may access the network for communication with the server 12 via cellular mobile communication technology, including fifth generation mobile communication (5th Generation Mobile Networks,5G) technology; alternatively, the platform 11 may also access the network for communication with the server 12 via short-range wireless communication means, including wireless fidelity (Wireless Fidelity, wi-Fi) technology.
Alternatively, the server 12 may be a separate physical server or may be a server cluster formed by a plurality of physical servers, which is not limited in this embodiment, and for ease of understanding, a server is taken as an example to describe each device and its respective functions in the following description.
The platform 11 is a device that can provide an interface presentation and/or data connectivity function to a user, such as a handheld terminal device, a vehicle mounted terminal device, or any electronic device having a data visualization function, etc.
Illustratively, the platform 11 includes, but is not limited to: android devices, IOS devices, mobile internet devices (Mobile Internet Device, MID), or wireless terminal devices in smart cities, etc.
Further, the target terminal 11 may be provided with a client related to data management, and the client may be software (e.g., APP, browser, short video software, etc.), web page, applet, etc. Alternatively, in the embodiment of the present application, the platform 11 may use the above-mentioned client to provide a display interface for a manager and dynamically display the running status of the related data.
Further, the server 12 may generate and/or store a large amount of data, such as user access data, system log data, etc., and referring to fig. 2, in a specific embodiment, the server 102 is disposed with a management system 20, where the management system 20 may be configured to manage all (or a specified plurality of) data in the server 12 according to the respective set operating conditions, and the management system 20 includes an archiving/destruction condition module 201, an archiving module 202, a destruction module 203, a timing task module 204, and a lifecycle monitoring module 205.
An archiving/destruction condition module 201, configured to set an operation condition for one or more data, where the operation condition is associated with a target operation, and the target operation is an archiving/destruction related operation.
An archiving module 202 is configured to archive at least part of the data that satisfies the operation condition associated with archiving, that is, at least part of the data that satisfies the operation condition when the target operation is data archiving.
The destruction module 203 is configured to perform destruction processing on at least part of the data that satisfies the operation condition associated with the destruction, that is, destroy at least part of the data that satisfies the operation condition when the target operation is data destruction.
The timing task module 204 is configured to set execution time of the task, that is, to set execution time for each data, for example, including daily/monthly/specified date execution, and the like.
The lifecycle monitoring module 205 is configured to monitor and record the whole lifecycle processing result of the data and analyze the operation status thereof, and optionally, the lifecycle monitoring module 205 may further perform statistics on the execution status of each target operation and/or the operation status of the data, and dynamically display the relevant result (e.g. generating, archiving, destroying records, etc.) in the display interface of the platform 11 through the client and the like.
Based on the above application scenario, the data management method provided in the embodiment of the present application will be further described and illustrated with reference to the accompanying drawings, and referring to fig. 3, the method includes:
s301: and responding to the execution time of the set task, and acquiring the monitoring information of the data.
S302: judging whether the data meets preset operation conditions according to the monitoring information.
Specifically, a set task, i.e., a timing task set for data, is used to indicate the execution time of the method, e.g., set to execute daily/monthly/specified date, etc., i.e., information is acquired on the data when the specified execution time is reached, and exemplary, by setting a plurality of tasks (task 1-task n) by the timing task and setting the respective execution times of the plurality of tasks, the execution process of each task is as shown in fig. 4.
Specifically, the monitoring information is used to characterize the current attribute of the data, such as a specified data field including, but not limited to, a call date field, a call number field, or a partial field containing a specified type of character.
Further, the operating conditions are associated with target operations set for the data, such as, but not limited to, data archiving, data destruction, and the like.
In an optional embodiment, in S302, it is determined, according to the monitoring information, whether the data meets a preset operation condition, including any one or a combination of the following:
1) And responding to the monitoring information to meet the set automatic triggering rule, calculating the operation probability of the data corresponding to the target operation, and determining that the data meets the preset operation condition when the operation probability is not smaller than a preset threshold value.
Specifically, the operation condition may be an automatic triggering condition set for the data by an automation algorithm, and in the above case, when it is determined that the monitoring information meets the set automatic triggering rule, it may be determined whether the data meets a preset operation condition according to the calculated operation probability of the data corresponding to the target operation.
In an alternative embodiment, the automatic triggering rule characterizes that the monitoring information indicates that the data includes a specified field, for example, a calling date and/or calling number field, and calculates an operation probability of the data corresponding to the target operation according to the field, where the method includes:
step 11: and calculating a first operation probability of the data corresponding to the target operation by adopting the first parameter according to the call date of the data contained in the monitoring information.
Step 12: and calculating a second operation probability of the data corresponding to the target operation by adopting a second parameter according to the calling times of the data contained in the monitoring information.
Step 13: and respectively adopting a first weight and a second weight to carry out weighted summation on the first operation probability and the second operation probability to obtain a weighted summation value, and adopting a preset function and a third operation probability according to the weighted summation value to calculate the operation probability of the data corresponding to the target operation.
Specifically, the first parameter characterizes the maximum storage time of the data, and the first operation probability of the data corresponding to the target operation in the storage time is calculated according to the first parameter and the calling date of the data, as shown in the following formula:
wherein D is the stored time of the data determined by the calling date, D is the set first parameter,
a first operational probability of the data is characterized.
Further, the second parameter characterizes the sum of the calling times of the data, and the second operation probability of the data corresponding to the target operation on the calling times is calculated according to the second parameter and the calling times of the data, as shown in the following formula:
where c is the number of calls of the data, sum (c 1 ，c 2 ，…c n ) For the sum of call times of the data, c1, c2, … cn is the historical call times of the data,
a second operational probability of the data is characterized.
Further, based on the first operation probability and the second operation probability obtained by the calculation, the set first weight and second weight are adopted to carry out weighted summation, and the operation probability of the target operation corresponding to the data is calculated according to a preset function and third operation probability, wherein the sum of the first weight and the second weight is one, and the following formula is shown:
w 1 +w 2 ＝1
wherein w is 1 、w 2 Respectively a first weight and a second weight which are set,
for a preset function->
For the third operation probability->
And the operation probability corresponding to the current target operation is given to the data.
Specifically, the third operation probability may be obtained by calculating, according to the number of storage strings of the data, a third parameter representing the maximum storage record amount of the data, and the preset function may be used to normalize the calculated weighted sum value, as shown in the following formula:
wherein L is the current storage number of the data, L is the maximum storage record amount of the data,
and p is a function independent variable for a preset function.
Further, based on the operation probability of the data obtained by the calculation corresponding to the target operation (such as data archiving/data destruction), when the operation probability is not smaller than a preset threshold (such as 0.8), determining that the data meets a preset operation condition.
2) And responding to the monitoring information meeting the set custom trigger rules, acquiring at least one custom trigger field set corresponding to the custom trigger rules, and determining that the data meets the preset operation condition when the monitoring information is matched with the at least one custom trigger field.
Specifically, the operation condition may be a data custom operation condition set by a user, and in the above case, when it is determined that the monitoring information meets a set custom trigger rule, whether the data meets a preset operation condition may be determined according to at least one corresponding custom trigger field.
In a specific embodiment, the custom trigger field may be used to indicate a target operating environment of the data, that is, when the data matches the target operating environment indicated by at least one custom trigger field, and the target operation is used to process the data, where the custom trigger field may be composed of fields/characters under any type of date type, character type, integer type, etc., for example, a custom trigger field characterized by "(day () -date_ +1) >10" may be used to determine that the data matches the target operating environment to process the data using the target operation when the call time (date) of the data is greater than 10 days from the current time.
It should be noted that, the number of the custom trigger fields may be one or more, and when the monitoring information matches each custom trigger field, it is determined that the monitoring information meets the preset operation condition.
S303: and processing the data with the target operation in response to the data meeting the operation condition.
In an alternative embodiment, the target operation includes data archiving, which is used to reduce data backlog, facilitate data arrangement, and store data in a standardized manner, specifically, for one data, when the data meets the operation condition corresponding to the data archiving, the target operation is used to process the data, including: filing the file, and storing the data in a file form; alternatively, database archiving is employed to save the data to a specified database table.
For example, referring to FIG. 5, when the target operation is data archiving, both file archiving and database archiving may be included, where a database archiving characterization maintains a database table containing the data, refers to maintaining the data satisfying the operating condition to an existing specified database table, or maintaining the data and multiple data under the same operating condition as a new database table; the file archive characterizes a data file that holds the data, i.e., the data that satisfies the operating conditions is stored in a file format, as shown, as a csv, db, excel, xml, md, sql, etc. format file.
In an alternative embodiment, the target operation includes data destruction, which is used to reduce data backlog, delete redundant data, and reduce storage space occupied by data, specifically, when the data meets an operation condition corresponding to data destruction, the target operation is used to process the data, including: adopting a recycle bin to destroy, and transferring the data to a target recycling space; or, the data is directly deleted by adopting permanent destruction.
For example, referring to fig. 6, when the target operation is data archiving, two manners of recycle bin destruction and permanent destruction may be included, where the recycle bin destruction represents that the data meeting the operation condition is moved to the target recycling space (e.g., a system recycle bin), and then the data can be recovered or permanently deleted according to actual needs; the permanent destruction characterization deletes the data directly in the storage space.
S304: and monitoring and recording the processing result of the data, and carrying out life cycle management on the data according to the processing result.
Specifically, after monitoring and recording the processing result of the data, updating the operation condition of the data in real time according to the processing result, including: the method comprises the steps of current data volume, archive data volume (database archive data volume and file archive data volume), destruction data volume (recycle bin destruction data volume and permanent deletion data volume), so that the operation condition of the data is analyzed based on the analysis, and management staff can conveniently know the operation condition analyzed by the data in a display interface.
For example, referring to fig. 7, taking a plurality of data (data 1-data n) as an example, in the life cycle monitoring, for each data, the processing result under the corresponding target operation is monitored and recorded, and the current data amount, the database archiving data amount, the file archiving data amount, the recycle bin destruction data amount and the permanent deletion data amount of the data are updated and displayed at the same time, further, the upcoming database archiving data amount, the upcoming file archiving data amount, the upcoming recycle bin destruction data amount and the upcoming permanent deletion data amount of the data are displayed, and optionally, the relevant information of the current data amount, the archiving data amount and the destruction data amount of the data in the specified period (for example, monthly/daily/specified time range) is dynamically displayed to reflect the running condition of the data, so that the manager can know the relevant information, and it can be understood that the data amount of the plurality of the data corresponding to each target operation, that is, the current data amount, the database archiving data amount, the total amount, the file data amount …, the upcoming recycle bin total amount and the permanent deletion data amount and the like are displayed, which are not repeated.
Further, based on the same technical concept, the embodiment of the application also provides a data management device, which is used for realizing the above method flow of the embodiment of the application. Referring to fig. 8, the apparatus includes: a task execution module 801, a condition judgment module 802, an operation processing module 803, and a monitoring management module 804, wherein:
the task execution module 801 is configured to obtain monitoring information of data in response to a set execution time of a task, where the monitoring information is used to characterize a current attribute of the data.
A condition judging module 802, configured to judge whether the data meets a preset operation condition according to the monitoring information, where the operation condition is related to a target operation set for the data.
An operation processing module 803, configured to process the data with the target operation in response to the data meeting the operation condition.
The monitoring management module 804 is configured to monitor and record a processing result of the data, and manage a life cycle of the data according to the processing result.
In an alternative embodiment, the determining whether the data meets a preset operation condition according to the monitoring information, where the condition determining module 802 is configured to:
and responding to the monitoring information to meet a set automatic triggering rule, calculating the operation probability of the data corresponding to the target operation, and determining that the data meets a preset operation condition when the operation probability is not smaller than a preset threshold value.
And/or the number of the groups of groups,
and responding to the monitoring information to meet a set custom trigger rule, acquiring at least one custom trigger field set corresponding to the custom trigger rule, and determining that the data meets a preset operation condition when the monitoring information is matched with the at least one custom trigger field, wherein the custom trigger field is used for indicating a target operation environment of the data.
In an alternative embodiment, the calculating the operation probability of the data corresponding to the target operation, the condition determining module 802 is configured to:
and calculating a first operation probability of the data corresponding to the target operation by adopting a first parameter according to the calling date of the data contained in the monitoring information, wherein the first parameter represents the maximum storage time of the data.
And calculating a second operation probability of the data corresponding to the target operation by adopting a second parameter according to the calling times of the data contained in the monitoring information, wherein the second parameter represents the sum of the calling times of the data.
And respectively adopting a first weight and a second weight to carry out weighted summation on the first operation probability and the second operation probability to obtain a weighted summation value, and adopting a preset function and a third operation probability to calculate the operation probability of the data corresponding to the target operation according to the weighted summation value, wherein the sum value of the first weight and the second weight is one, and the third operation probability is calculated according to the number of storage bars of the data.
In an optional embodiment, the calculating the operation probability of the data corresponding to the target operation according to the weighted sum value and using a preset function and a third operation probability, the condition determining module 802 is configured to:
and calculating a third operation probability of the data corresponding to the target operation by adopting a third parameter according to the number of the data, wherein the third parameter represents the maximum storage record quantity of the data.
And calculating an objective function value corresponding to the weighted sum value according to a preset function, and calculating the objective function value by adopting the third operation probability to obtain the operation probability of the data corresponding to the objective operation.
In an alternative embodiment, the target operation is data archiving, and the processing is performed on the data by adopting the target operation, and the operation processing module 803 is configured to:
and (5) adopting file archiving, and storing the data in a file form.
Or,
and (5) adopting database archiving, and storing the data into a specified database table.
In an alternative embodiment, if the target operation is data destruction, the processing of the data with the target operation is performed, and the operation processing module 803 is configured to:
and adopting a recycle bin to destroy, and migrating the data to a target recycling space.
Or,
and adopting permanent destruction to directly delete the data.
In an alternative embodiment, after the monitoring and recording the processing result of the data, the monitoring management module 804 is further configured to:
and according to the processing result, updating the current data quantity of the data and updating the processed data quantity of the data corresponding to the target operation.
Analyzing the running condition of the data by adopting the current data quantity and the processed data quantity, and displaying the running condition in a preset display interface.
Based on the same inventive concept as the above-mentioned application embodiments, an electronic device is also provided in the application embodiments, and the electronic device may be used for data management. In one embodiment, the electronic device may be a server, a terminal device, or other electronic device. In this embodiment, the electronic device may be configured as shown in fig. 9, including a memory 901, a communication interface 903, and one or more processors 902.
A memory 901 for storing a computer program executed by the processor 902. The memory 901 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, a program required for running an instant communication function, and the like; the storage data area can store various instant messaging information, operation instruction sets and the like.
The memory 901 may be a volatile memory (RAM) such as a random-access memory (RAM); the memory 901 may also be a nonvolatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk (HDD) or a Solid State Drive (SSD), or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 901 may be a combination of the above memories.
The processor 902 may include one or more central processing units (Central Processing Unit, CPU) or digital processing units, etc. A processor 902 for implementing the above-described data management method when calling the computer program stored in the memory 901.
The communication interface 903 is used to communicate with terminal devices and other servers.
The specific connection medium between the memory 901, the communication interface 903, and the processor 902 is not limited in the embodiments of the present application. In the embodiment of the present application, the memory 901 and the processor 902 are connected through the bus 904 in fig. 9, the bus 904 is indicated by a thick line in fig. 9, and the connection manner between other components is only schematically illustrated, and is not limited to the embodiment. The bus 904 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 9, but not only one bus or one type of bus.
Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a data management method as previously discussed.
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more of the elements described above may be embodied in one element in accordance with embodiments of the present application. Conversely, the features and functions of one unit described above may be further divided into a plurality of units to be embodied.
Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required to or suggested that these operations must be performed in this particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
The embodiment of the application provides a data management method, a device, electronic equipment and a storage medium, wherein the method is used for acquiring monitoring information of data in response to execution time of a set task, judging whether the data meets preset operation conditions according to the monitoring information, and processing the data by adopting target operations associated with the operation conditions when the data meets the operation conditions.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's equipment, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.
In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected over the Internet using an Internet service provider).
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.
Claims (10)
1. A method of data management, comprising:
responding to the execution time of a set task, and acquiring monitoring information of data, wherein the monitoring information is used for representing the current attribute of the data;
judging whether the data meets preset operation conditions according to the monitoring information, wherein the operation conditions relate to target operations set for the data;
processing the data with the target operation in response to the data meeting the operating condition;
and monitoring and recording the processing result of the data, and carrying out life cycle management on the data according to the processing result.
2. The method of claim 1, wherein determining whether the data satisfies a preset operating condition based on the monitoring information comprises:
responding to the monitoring information to meet a set automatic triggering rule, calculating the operation probability of the data corresponding to the target operation, and determining that the data meets a preset operation condition when the operation probability is not smaller than a preset threshold;
and/or the number of the groups of groups,
and responding to the monitoring information to meet a set custom trigger rule, acquiring at least one custom trigger field set corresponding to the custom trigger rule, and determining that the data meets a preset operation condition when the monitoring information is matched with the at least one custom trigger field, wherein the custom trigger field is used for indicating a target operation environment of the data.
3. The method of claim 2, wherein the calculating the operational probability that the data corresponds to the target operation comprises:
calculating a first operation probability of the data corresponding to the target operation by adopting a first parameter according to the call date of the data contained in the monitoring information, wherein the first parameter represents the maximum storage time of the data;
calculating a second operation probability of the data corresponding to the target operation by adopting a second parameter according to the calling times of the data contained in the monitoring information, wherein the second parameter represents the sum of the calling times of the data;
and respectively adopting a first weight and a second weight to carry out weighted summation on the first operation probability and the second operation probability to obtain a weighted summation value, and adopting a preset function and a third operation probability to calculate the operation probability of the data corresponding to the target operation according to the weighted summation value, wherein the sum value of the first weight and the second weight is one, and the third operation probability is calculated according to the number of storage bars of the data.
4. The method of claim 3, wherein calculating the operational probability of the data corresponding to the target operation using a predetermined function and a third operational probability based on the weighted sum value comprises:
calculating a third operation probability of the data corresponding to the target operation by adopting a third parameter according to the number of the data, wherein the third parameter represents the maximum storage record quantity of the data;
and calculating an objective function value corresponding to the weighted sum value according to a preset function, and calculating the objective function value by adopting the third operation probability to obtain the operation probability of the data corresponding to the objective operation.
5. The method of any of claims 1-4, wherein the target operation is a data archive, and the processing the data with the target operation comprises:
filing the file, and storing the data in a file form;
or,
and (5) adopting database archiving, and storing the data into a specified database table.
6. The method of any of claims 1-4, wherein the target operation is data destruction, and the processing the data with the target operation comprises:
adopting a recycle bin to destroy, and transferring the data to a target recycling space;
or,
and adopting permanent destruction to directly delete the data.
7. The method of any of claims 1-4, wherein after monitoring and recording the processing results of the data, further comprising:
updating the current data volume of the data according to the processing result, and updating the processed data volume of the data corresponding to the target operation;
analyzing the running condition of the data by adopting the current data quantity and the processed data quantity, and displaying the running condition in a preset display interface.
8. A data management apparatus, comprising:
the task execution module is used for responding to the execution time of the set task and obtaining the monitoring information of the data, wherein the monitoring information is used for representing the current attribute of the data;
the condition judging module is used for judging whether the data meets preset operation conditions according to the monitoring information, wherein the operation conditions relate to target operations set for the data;
an operation processing module for processing the data with the target operation in response to the data satisfying the operation condition;
and the monitoring management module is used for monitoring and recording the processing result of the data and managing the life cycle of the data according to the processing result.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-7 when executing the computer program.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211732680.7A CN116126859A (en) | 2022-12-30 | 2022-12-30 | Data management method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211732680.7A CN116126859A (en) | 2022-12-30 | 2022-12-30 | Data management method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116126859A true CN116126859A (en) | 2023-05-16 |
Family
ID=86311202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211732680.7A Pending CN116126859A (en) | 2022-12-30 | 2022-12-30 | Data management method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116126859A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117131036A (en) * | 2023-10-26 | 2023-11-28 | 环球数科集团有限公司 | Data maintenance system based on big data and artificial intelligence |
-
2022
- 2022-12-30 CN CN202211732680.7A patent/CN116126859A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117131036A (en) * | 2023-10-26 | 2023-11-28 | 环球数科集团有限公司 | Data maintenance system based on big data and artificial intelligence |
| CN117131036B (en) * | 2023-10-26 | 2023-12-22 | 环球数科集团有限公司 | Data maintenance system based on big data and artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021164465A1 (en) | Intelligent early warning method and system | |
| CN108363657B (en) | Method, equipment and medium for monitoring integrity of embedded data acquisition of APP client | |
| US9602340B2 (en) | Performance monitoring | |
| US9372734B2 (en) | Outage window scheduler tool | |
| CN112445583B (en) | Task management method, task management system, electronic device, and storage medium | |
| CN109684320B (en) | Method and equipment for online cleaning of monitoring data | |
| CN109670091B (en) | Metadata intelligent maintenance method and device based on data standard | |
| CN109992473A (en) | Monitoring method, device, equipment and the storage medium of application system | |
| CN116126859A (en) | Data management method and device, electronic equipment and storage medium | |
| CN102271054A (en) | Bookmarks and performance history for network software deployment evaluation | |
| CN112564951A (en) | Method, device, computer equipment and storage medium for avoiding alarm storm | |
| CN115905863A (en) | Machine learning model training method and quantum network equipment performance value prediction method | |
| CN112260858A (en) | Alarm method capable of automatic detection and terminal | |
| CN111884853A (en) | Cloud environment automatic resource management method and system | |
| CN111324583B (en) | Service log classification method and device | |
| CN111737233A (en) | Data monitoring method and device | |
| CN107577433B (en) | Storage medium and file data migration method, device and equipment | |
| CN113824590B (en) | Method for predicting problem in micro service network, computer device, and storage medium | |
| CN110968993A (en) | Information processing method and device, storage medium and processor | |
| US9459939B2 (en) | In-memory approach to extend semantic event processing with domain insights | |
| WO2021218626A1 (en) | Data storage method and apparatus, device, and storage medium | |
| CN114385705A (en) | Data importance identification method, device, equipment and medium | |
| CN110148011B (en) | Method, device, equipment and medium for analyzing active amount drop based on big data | |
| CN110096518A (en) | Knowledge base metadata sending method and device, readable storage medium storing program for executing | |
| CN113127056B (en) | Information processing method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |