CN112486966A - Expired data cleaning method and device and electronic equipment - Google Patents
Expired data cleaning method and device and electronic equipment Download PDFInfo
- Publication number
- CN112486966A CN112486966A CN202011356503.4A CN202011356503A CN112486966A CN 112486966 A CN112486966 A CN 112486966A CN 202011356503 A CN202011356503 A CN 202011356503A CN 112486966 A CN112486966 A CN 112486966A
- Authority
- CN
- China
- Prior art keywords
- data
- service
- cleaned
- days
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 124
- 238000004140 cleaning Methods 0.000 title claims abstract description 77
- 230000008569 process Effects 0.000 claims abstract description 82
- 238000003860 storage Methods 0.000 claims abstract description 21
- 230000004044 response Effects 0.000 claims abstract description 11
- 230000014759 maintenance of location Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000717 retained effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides an expired data cleaning method and device and electronic equipment, and relates to the field of data processing. The outdated data cleaning method comprises the following steps: in response to the received task start information, a plurality of work processes are started. And the working process determines a corresponding service database according to the service navigation information. And then, the work process determines a service data table to be operated from the corresponding service database. Further, the work process determines the data to be cleaned from the service data table to be operated according to the data cleaning rule. And finally, deleting the data to be cleaned after the backup storage of the data to be cleaned by the working process. Therefore, the ticketing service can be provided uninterruptedly when data is cleaned.
Description
[ technical field ] A method for producing a semiconductor device
The present application relates to the field of data processing, and in particular, to a method and an apparatus for cleaning stale data, and an electronic device.
[ background of the invention ]
The cleaning of stale data is crucial to any information system. The overdue data in the information system is cleaned, the residual space of the system storage can be kept to a certain extent, and the stable operation of the system is guaranteed. For a huge information system such as 12306 internet ticketing system having billions of registered users, there is a large amount of overdue data such as order information, stub information, and transaction information. The cleaning of these expired data is a very important link in the system maintenance process.
At present, the cleaning of the background overdue data of the 12306 system can affect the normal development of foreground business. Therefore, the 12306 system cannot provide the ticketing service when the expired data is cleared.
[ summary of the invention ]
The embodiment of the application provides an expired data cleaning method, an expired data cleaning device and electronic equipment, so that ticketing services can be provided uninterruptedly during data cleaning.
In a first aspect, an embodiment of the present application provides an expired data cleaning method, including: starting a plurality of work processes in response to the received task starting information; the working process determines a corresponding service database according to the service navigation information; the work process determines a service data table to be operated from the corresponding service database; the work process determines data to be cleaned from the service data table to be operated according to a data cleaning rule; and deleting the data to be cleaned after the working process stores the data to be cleaned in a backup mode.
In one possible implementation manner, the service navigation information includes a corresponding relationship between a process identifier and a database identifier; the working process determines a corresponding service database according to the service navigation information, and the method comprises the following steps: and the working process determines the service database according to the process identification of the working process and the corresponding relation.
In one possible implementation manner, the determining, by the work process, a to-be-operated service data table from the corresponding service database includes: the work process determines whether each service data table in the service database is associated with a cleaning task identifier; and the work process determines the service data table associated with the cleaning task identifier as a service data table to be operated.
In one possible implementation manner, the data cleansing rule includes a threshold number of retention days; the working process determines the data to be cleaned from the service data table to be operated according to the data cleaning rule, and the method comprises the following steps: determining whether the reserved days of each piece of service data in the service data table to be operated are greater than a reserved days threshold value; and determining the first service data with the retention days larger than the threshold value of the retention days as the data to be cleaned.
In one possible implementation manner, the data cleaning rule further includes a number of deletable days; determining the first service data with the retention days larger than the retention days threshold as the data to be cleaned, wherein the method comprises the following steps: determining a date of generation of the first service data; if the first service data is generated on different dates, determining whether the number of the different dates exceeds the deletable number of days; if the number of the different dates does not exceed the deletable number of days, determining the first service data as data to be cleaned; and if the number of the different dates exceeds the deletable number of days, selecting the data meeting the deletable number of days from the first service data as the data to be cleaned.
In one possible implementation manner, the data cleaning rule further includes cleaning time; before the backup storage of the data to be cleaned by the working process, the method further comprises the following steps: determining that a current time is within the allowed clean time.
In one possible implementation manner, before the backup storage of the to-be-cleaned data by the work process, the method further includes: determining the size of the data to be cleaned; and determining the residual space of the backup database to store the data to be cleaned according to the size of the data to be cleaned.
In a second aspect, an embodiment of the present application provides an expired data cleaning apparatus, including: the starting module is used for responding to the received task starting information and starting a plurality of working processes; the first determining module is used for enabling the working process to determine a corresponding service database according to the service navigation information; the second determining module is used for enabling the working process to determine a service data table to be operated from the corresponding service database; the third determining module is used for enabling the working process to determine the data to be cleaned from the service data table to be operated according to a data cleaning rule; and the cleaning module is used for deleting the data to be cleaned after the working process stores the data to be cleaned in a backup mode.
In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, which when called by the processor are capable of performing the method as described above.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the method as described above.
In the above technical solution, a plurality of work processes are started in response to the received task start information. And the working process determines a corresponding service database according to the service navigation information. And then, the work process determines a service data table to be operated from the corresponding service database. Further, the work process determines the data to be cleaned from the service data table to be operated according to the data cleaning rule. And finally, deleting the data to be cleaned after the backup storage of the data to be cleaned by the working process. Therefore, the ticketing service can be provided uninterruptedly when data is cleaned.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of an expired data cleaning method according to an embodiment of the present application;
FIG. 2 is a flowchart of another stale data cleaning method according to an embodiment of the present application;
FIG. 3 is a flowchart of another stale data cleaning method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an expired data cleaning apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
[ detailed description ] embodiments
For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.
It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
Fig. 1 is a flowchart of an expired data cleaning method according to an embodiment of the present application. As shown in fig. 1, the above-mentioned outdated data cleaning method may include:
In the embodiment of the application, after the task starting information sent by the system is received, a plurality of working processes can execute the data cleaning task at the same time. The system may send the task start information by setting a timing task and triggering the task start information at a preset time node at a fixed time. And triggering task starting information through the operation of an administrator on the system according to actual requirements.
And 102, determining a corresponding service database by the working process according to the service navigation information.
In the embodiment of the application, a plurality of service databases can be set. And each service library can store a service data table.
In some embodiments, when the service data table is stored in the multiple databases, a hash algorithm may be performed on the user name corresponding to the service data. For example, a hashing algorithm may be performed on the first letter of the username. And storing the service data tables with the same hash value in the same service database according to the difference of the obtained hash values.
In other embodiments, when storing the service data tables in the plurality of databases, the service data tables may be classified first. And storing the service data tables of the same category in the same service database according to the classification result.
In the embodiment of the application, the corresponding relation between the process identifier and the database identifier can be set in the service navigation information. After the plurality of working processes are started, the corresponding service databases can be determined according to the process identifiers of the working processes and the corresponding relation in the service navigation information. And a plurality of working processes can simultaneously access different service libraries to execute data cleaning.
And 103, determining a service data table to be operated from the corresponding service database by the work process.
In the embodiment of the application, a plurality of service data tables can be stored in each service database. When the work process accesses the service database, the service data table to be operated needs to be determined from the service database.
In some embodiments, the service data table to be operated may be a service data table containing data amount exceeding a preset threshold. The type and value of the preset threshold can be set according to the requirements of actual conditions. For example, the type of the preset threshold may be the number of days corresponding to the data; the value was 50 days. For example, the type of the preset threshold may be the number of pieces of data; the value was 10000.
In other embodiments, the service data table to be operated may be a service data table that does not meet the preset cleaning-free condition. The preset cleaning-free condition may be, for example, that the service data table corresponding to the new service is cleaned-free. It is also possible that the service data table in a particular period is free from cleaning. The special time may be a time when the large-scale and frequent business data change is caused by external inefficacy.
In this embodiment of the application, the service data table to be operated may be associated with the cleaning task identifier. The related manner may be that a cleaning task identifier is set on a service data table to be operated. The association mode can also be that a cleaning task identifier is set in a definition table associated with the data table to be operated.
Based on the above description, in the embodiment of the present application, when the work process determines the service data table to be operated in the service database, it may be determined whether each service data table is associated with a cleaning task identifier. And determining the service data table associated with the cleaning task identifier as a service data table to be operated.
And step 104, determining the data to be cleaned from the service data table to be operated by the work process according to the data cleaning rule.
In the embodiment of the application, the association relationship between the data cleaning rule and the service data table can be established in advance. Specifically, different business data tables may be associated with different data cleansing rules. Different business data tables may also be associated with the same data cleansing rules. And after the business data table to be operated is determined, the work process determines the data cleaning rule corresponding to the business data table to be operated according to the incidence relation. And further determining the data to be cleaned in the service data table according to the data cleaning rule.
In some embodiments, the data cleansing rule may include a threshold number of days to retain. The threshold number of days to retain specifies the longest time period that data can be retained in the traffic database. When the work process determines the data to be cleaned, as shown in fig. 2, the number of reserved days of each service data in the service data table to be operated is judged. And if the reserved days are greater than the reserved days threshold, determining the corresponding first service data as the data to be cleaned. And if the data retention days are not greater than the retention days threshold value in the to-be-operated service data table, recording the operation log, and re-determining the to-be-operated service data table. And the data retention days are not more than the retention days threshold, which shows that the data in the service data table can be retained continuously until the retention days are more than the retention days threshold. In the embodiment of the present invention, the operation log may be a name of a service data table for recording the operation, whether data cleaning is completed, an incomplete reason, and the like.
In other embodiments, the data cleansing rules may include a threshold number of days to retain and a number of days to delete. The threshold number of days to retain specifies the longest time period that data can be retained in the traffic database. The deletable number of days specifies the number of days for each deletable data. When the work process determines the data to be cleaned, as shown in fig. 2, the number of reserved days of each service data in the service data table to be operated is judged. And if the reserved days are greater than the reserved days threshold, determining the corresponding data as the first service data. Then, a generation date of the first service data is determined. If the first service data is generated on different dates, it is determined whether the number of different dates exceeds the deletable number of days. And if the number of the different dates does not exceed the deletable days, determining the first service data as the data to be cleaned. And if the number of different dates exceeds the deletable number of days, selecting the data meeting the deletable number of days from the first service data as the data to be cleaned. When data satisfying the deletable days is selected, data with longer retention days can be selected preferentially.
And 105, deleting the data to be cleaned after the working process stores the data to be cleaned in a backup mode.
In the embodiment of the application, after the data to be cleaned is determined, the data to be cleaned needs to be backed up to ensure the recoverability of the data. Optionally, the data to be cleaned is backed up in the backup database.
Further, in order to ensure the reliability of the backup, the embodiment of the present application may check the backup condition after the backup is completed. Optionally, it is checked whether the data to be cleaned is backed up completely in the backup database. As shown in fig. 2, after it is determined that the data to be cleaned is completely backed up, the data to be cleaned is deleted from the service data table. And if the complete data to be cleaned is not backed up, deleting the incomplete data to be cleaned in the backup database.
In the embodiment of the application, a plurality of working processes are started in response to the received task starting information. And the working process determines a corresponding service database according to the service navigation information. And then, the work process determines a service data table to be operated from the corresponding service database. Further, the work process determines the data to be cleaned from the service data table to be operated according to the data cleaning rule. And finally, deleting the data to be cleaned after the backup storage of the data to be cleaned by the working process. Therefore, the ticketing service can be provided uninterruptedly when data is cleaned.
Fig. 3 is a flowchart of another stale data cleaning method according to an embodiment of the present application. As shown in fig. 3, before step 105 in the embodiment shown in fig. 1 of the present application, the method may further include:
In the embodiment of the present application, the data cleaning rule further includes cleaning time. The clean-up time specifies the time at which data cleaning can be performed. The cleaning time can be set according to the requirements of actual conditions. In some embodiments, in order to ensure the timeliness of data cleaning, the cleaning time may be set to 00: 00-24: 00. In other embodiments, in order to avoid the peak of foreground ticketing services and ensure the fluency of background data cleaning and foreground services, the cleaning time can be set to 00: 00-06: 00.
After the data to be cleaned is determined, the current time needs to be judged first. As shown in fig. 2, it is determined that the current time is within the clearing time, then the following step 1042 is performed. And determining that the current time is not within the clearing time, recording an operation log, and re-determining a to-be-operated service data table.
In the embodiment of the application, after the data to be cleaned is determined, the data to be cleaned needs to be backed up to the backup database. In order to ensure that the backup database has enough space to store the data to be cleaned, the size of the data to be cleaned and the remaining space of the backup database need to be determined.
As shown in fig. 2, after determining that the remaining space of the backup database can store the data to be cleaned, the data to be cleaned is backed up. And after the backup is finished, deleting the data to be cleaned. And if the residual space of the backup database is determined not to store the data to be cleaned, recording an operation log, and re-determining the service data table to be operated.
In the embodiment of the application, the current time and the size of the data to be cleaned are judged before the data to be cleaned are backed up and deleted. On one hand, the data cleaning time can be regulated, and the data cleaning and the fluency of foreground service are ensured to the maximum extent; on the other hand, the storage capacity of the backup database for the data to be cleaned can be determined, and the data to be cleaned can be completely backed up.
Fig. 4 is a schematic structural diagram of an embodiment of an expired data cleaning device according to the present application. The expired data cleaning device in the embodiment can be used as expired data cleaning equipment to realize the expired data cleaning method provided by the embodiment of the application. As shown in fig. 4, the expired data cleaning apparatus may include: an initiating module 41, a first determining module 42, a second determining module 43, a third determining module 44 and a cleaning module 45.
And the starting module 41 is used for responding to the received task starting information and starting a plurality of work processes.
And the first determining module 42 is used for enabling the working process to determine the corresponding service database according to the service navigation information.
And a second determining module 43, configured to enable the work process to determine a service data table to be operated from the corresponding service database.
And the third determining module 44 is configured to enable the work process to determine the data to be cleaned from the service data table to be operated according to the data cleaning rule.
And the cleaning module 45 is used for deleting the data to be cleaned after the backup of the data to be cleaned is stored in the working process.
In the embodiment of the present application, the starting module 41 starts a plurality of work processes in response to the received task starting information. The first determining module 42 determines the corresponding service database according to the service navigation information. Then, the second determining module 43 determines the service data table to be operated from the corresponding service database. Further, the third determining module 44 determines the data to be cleaned from the service data table to be operated according to the data cleaning rule. Finally, the cleaning module 45 deletes the data to be cleaned after storing the data to be cleaned in a backup manner. Therefore, the ticketing service can be provided uninterruptedly when data is cleaned.
Fig. 5 is a schematic structural diagram of an embodiment of an electronic device according to the present application. As shown in fig. 5, the electronic device may include at least one processor; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the expired data cleaning method provided by the embodiment of the application.
The electronic device may be an expired data cleaning device, and the embodiment does not limit the specific form of the electronic device.
FIG. 5 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present application. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 5, the electronic device is in the form of a general purpose computing device. Components of the electronic device may include, but are not limited to: one or more processors 410, a memory 430, and a communication bus 440 that connects the various system components (including the memory 430 and the processing unit 410).
Electronic devices typically include a variety of computer system readable media. Such media may be any available media that is accessible by the electronic device and includes both volatile and nonvolatile media, removable and non-removable media.
A program/utility having a set (at least one) of program modules, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored in memory 430, each of which examples or some combination may include an implementation of a network environment. The program modules generally perform the functions and/or methodologies of the embodiments described herein.
The electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the electronic device, and/or any devices (e.g., network card, modem, etc.) that enable the electronic device to communicate with one or more other computing devices. Such communication may occur via communication interface 420. Furthermore, the electronic device may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public Network such as the Internet) via a Network adapter (not shown in FIG. 5) that may communicate with other modules of the electronic device via the communication bus 440. It should be appreciated that although not shown in FIG. 5, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, Redundant processing units, external disk drive Arrays, disk array (RAID) systems, tape Drives, and data backup storage systems, among others.
The processor 410 executes various functional applications and data processing, such as implementing the expired data cleaning method provided by the embodiment of the present application, by executing the program stored in the memory 430.
The embodiment of the present application further provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the expired data cleaning method provided in the embodiment of the present application.
The non-transitory computer readable storage medium described above may take any combination of one or more computer readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a flash Memory, an optical fiber, a portable compact disc Read Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
It should be noted that the terminal according to the embodiments of the present application may include, but is not limited to, a Personal Computer (Personal Computer; hereinafter, referred to as PC), a Personal Digital Assistant (Personal Digital Assistant; hereinafter, referred to as PDA), a wireless handheld device, a Tablet Computer (Tablet Computer), a mobile phone, an MP3 player, an MP4 player, and the like.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.
Claims (10)
1. A method for cleaning expired data, comprising:
starting a plurality of work processes in response to the received task starting information;
the working process determines a corresponding service database according to the service navigation information;
the work process determines a service data table to be operated from the corresponding service database;
the work process determines data to be cleaned from the service data table to be operated according to a data cleaning rule;
and deleting the data to be cleaned after the working process stores the data to be cleaned in a backup mode.
2. The method according to claim 1, wherein the service navigation information comprises a correspondence between a process identifier and a database identifier;
the working process determines a corresponding service database according to the service navigation information, and the method comprises the following steps:
and the working process determines the service database according to the process identification of the working process and the corresponding relation.
3. The method of claim 1, wherein the determining, by the work process, the service data table to be operated from the corresponding service database comprises:
the work process determines whether each service data table in the service database is associated with a cleaning task identifier;
and the work process determines the service data table associated with the cleaning task identifier as a service data table to be operated.
4. The method of claim 1, wherein the data cleansing rule comprises a threshold number of days to remain;
the working process determines the data to be cleaned from the service data table to be operated according to the data cleaning rule, and the method comprises the following steps:
determining whether the reserved days of each piece of service data in the service data table to be operated are greater than a reserved days threshold value;
and determining the first service data with the retention days larger than the threshold value of the retention days as the data to be cleaned.
5. The method of claim 4, wherein the data cleansing rules further comprise a number of deletable days;
determining the first service data with the retention days larger than the retention days threshold as the data to be cleaned, wherein the method comprises the following steps:
determining a date of generation of the first service data;
if the first service data is generated on different dates, determining whether the number of the different dates exceeds the deletable number of days;
if the number of the different dates does not exceed the deletable number of days, determining the first service data as data to be cleaned;
and if the number of the different dates exceeds the deletable number of days, selecting the data meeting the deletable number of days from the first service data as the data to be cleaned.
6. The method of claim 1, wherein the data cleansing rules further comprise a cleansing time; before the backup storage of the data to be cleaned by the working process, the method further comprises the following steps:
determining that a current time is within the allowed clean time.
7. The method of claim 1, wherein before the work process stores the backup of the data to be cleaned, the method further comprises:
determining the size of the data to be cleaned;
and determining the residual space of the backup database to store the data to be cleaned according to the size of the data to be cleaned.
8. An expired data cleaning apparatus, comprising:
the starting module is used for responding to the received task starting information and starting a plurality of working processes;
the first determining module is used for enabling the working process to determine a corresponding service database according to the service navigation information;
the second determining module is used for enabling the working process to determine a service data table to be operated from the corresponding service database;
the third determining module is used for enabling the working process to determine the data to be cleaned from the service data table to be operated according to a data cleaning rule;
and the cleaning module is used for deleting the data to be cleaned after the working process stores the data to be cleaned in a backup mode.
9. An electronic device, comprising:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 7.
10. A non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011356503.4A CN112486966A (en) | 2020-11-26 | 2020-11-26 | Expired data cleaning method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011356503.4A CN112486966A (en) | 2020-11-26 | 2020-11-26 | Expired data cleaning method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112486966A true CN112486966A (en) | 2021-03-12 |
Family
ID=74936041
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011356503.4A Pending CN112486966A (en) | 2020-11-26 | 2020-11-26 | Expired data cleaning method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112486966A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113535697A (en) * | 2021-07-07 | 2021-10-22 | 广州三叠纪元智能科技有限公司 | Climbing frame data cleaning method, climbing frame control device and storage medium |
| CN113779026A (en) * | 2021-08-18 | 2021-12-10 | 北京健康之家科技有限公司 | Method and device for processing service data table |
| CN113535697B (en) * | 2021-07-07 | 2024-05-24 | 广州三叠纪元智能科技有限公司 | Climbing frame data cleaning method, climbing frame control device and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103064960A (en) * | 2012-12-31 | 2013-04-24 | 华为技术有限公司 | Method and equipment for database query |
| CN103425778A (en) * | 2013-08-16 | 2013-12-04 | 国家电网公司 | Intelligent development platform for database application system |
| CN106570005A (en) * | 2015-10-08 | 2017-04-19 | 阿里巴巴集团控股有限公司 | Database cleaning method and device |
| CN107357883A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107423448A (en) * | 2017-08-11 | 2017-12-01 | 天津同阳科技发展有限公司 | Distributed data processing device and method |
| CN107862031A (en) * | 2017-11-02 | 2018-03-30 | 东软集团股份有限公司 | Method for processing business, device and server |
| CN109344153A (en) * | 2018-08-22 | 2019-02-15 | 中国平安人寿保险股份有限公司 | The processing method and terminal device of business datum |
| CN110868313A (en) * | 2018-08-28 | 2020-03-06 | 网宿科技股份有限公司 | Inspection method, related device and readable storage medium |
| CN111291023A (en) * | 2020-02-09 | 2020-06-16 | 苏州浪潮智能科技有限公司 | Data migration method, system, device and medium |
| CN111400012A (en) * | 2020-03-20 | 2020-07-10 | 中国建设银行股份有限公司 | Data parallel processing method, device, equipment and storage medium |
-
2020
- 2020-11-26 CN CN202011356503.4A patent/CN112486966A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103064960A (en) * | 2012-12-31 | 2013-04-24 | 华为技术有限公司 | Method and equipment for database query |
| CN103425778A (en) * | 2013-08-16 | 2013-12-04 | 国家电网公司 | Intelligent development platform for database application system |
| CN106570005A (en) * | 2015-10-08 | 2017-04-19 | 阿里巴巴集团控股有限公司 | Database cleaning method and device |
| CN107357883A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107423448A (en) * | 2017-08-11 | 2017-12-01 | 天津同阳科技发展有限公司 | Distributed data processing device and method |
| CN107862031A (en) * | 2017-11-02 | 2018-03-30 | 东软集团股份有限公司 | Method for processing business, device and server |
| CN109344153A (en) * | 2018-08-22 | 2019-02-15 | 中国平安人寿保险股份有限公司 | The processing method and terminal device of business datum |
| CN110868313A (en) * | 2018-08-28 | 2020-03-06 | 网宿科技股份有限公司 | Inspection method, related device and readable storage medium |
| CN111291023A (en) * | 2020-02-09 | 2020-06-16 | 苏州浪潮智能科技有限公司 | Data migration method, system, device and medium |
| CN111400012A (en) * | 2020-03-20 | 2020-07-10 | 中国建设银行股份有限公司 | Data parallel processing method, device, equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113535697A (en) * | 2021-07-07 | 2021-10-22 | 广州三叠纪元智能科技有限公司 | Climbing frame data cleaning method, climbing frame control device and storage medium |
| CN113535697B (en) * | 2021-07-07 | 2024-05-24 | 广州三叠纪元智能科技有限公司 | Climbing frame data cleaning method, climbing frame control device and storage medium |
| CN113779026A (en) * | 2021-08-18 | 2021-12-10 | 北京健康之家科技有限公司 | Method and device for processing service data table |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114416667B (en) | Method and device for rapidly sharing network disk file, network disk and storage medium | |
| CN109656885B (en) | Storage space monitoring method and device, electronic terminal and storage medium | |
| CN111414362A (en) | Data reading method, device, equipment and storage medium | |
| US20160248724A1 (en) | Social Message Monitoring Method and Apparatus | |
| CN112486966A (en) | Expired data cleaning method and device and electronic equipment | |
| CN105574026A (en) | Method and device for service supporting by using non-relational database | |
| CN108205559B (en) | Data management method and equipment thereof | |
| CN108228611B (en) | Document information copying method and device | |
| CN112800134A (en) | Data synchronization method and system | |
| US11750549B2 (en) | File-related task management device | |
| US9424256B2 (en) | Method and apparatus for performing type-aware change tracking in a document | |
| US10360234B2 (en) | Recursive extractor framework for forensics and electronic discovery | |
| CN115543918A (en) | File snapshot method, system, electronic equipment and storage medium | |
| CN111352917B (en) | Information input method and device, electronic equipment and storage medium | |
| CN113986471A (en) | Method, device, equipment and storage medium for safely deleting mirror image file of virtual machine | |
| CN113806556A (en) | Method, device, equipment and medium for constructing knowledge graph based on power grid data | |
| CN113110846A (en) | Method and device for acquiring environment variable | |
| WO2017045493A1 (en) | Method and device for setting files to be removed | |
| CN112818204A (en) | Service processing method, device, equipment and storage medium | |
| CN112148712A (en) | Data processing method, device, equipment and medium | |
| CN111625500A (en) | File snapshot method and device, electronic equipment and storage medium | |
| CN111966645A (en) | Supercomputer data storage method, device, system and storage medium | |
| CN109857519A (en) | The processing method and relevant apparatus of virtual disk | |
| CN113806107B (en) | Object copying method, device, equipment and storage medium | |
| KR102644076B1 (en) | Apparatus and method for recovering deleted chat messages based on fts index data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |