CN109710596A - Data clearing method, device, equipment and computer readable storage medium - Google Patents

Data clearing method, device, equipment and computer readable storage medium Download PDF

Info

Publication number
CN109710596A
CN109710596A CN201811468867.4A CN201811468867A CN109710596A CN 109710596 A CN109710596 A CN 109710596A CN 201811468867 A CN201811468867 A CN 201811468867A CN 109710596 A CN109710596 A CN 109710596A
Authority
CN
China
Prior art keywords
information
cleaning
code
data
backup
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811468867.4A
Other languages
Chinese (zh)
Other versions
CN109710596B (en
Inventor
文玎玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN201811468867.4A priority Critical patent/CN109710596B/en
Publication of CN109710596A publication Critical patent/CN109710596A/en
Application granted granted Critical
Publication of CN109710596B publication Critical patent/CN109710596B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention discloses a kind of data clearing method, device, equipment and computer readable storage medium, the described method includes: reading table name information in cleaning file, owner's information, field information, column name information and cleaning conditional information when receiving cleaning file;Table name information, owner's information and field information are formed into table information for clearance, and backup code is generated according to table information for clearance;Table name information, column name information and cleaning conditional information are formed into cleaning information, and thrashing code is generated according to cleaning information;Thrashing code operation is controlled, is cleared up in table for clearance corresponding to table name information with the cleaning matched target data of conditional information, and control backup code operation, target data is backed up.Writing for developer is avoided according to cleaning file generated for the code of data scrubbing in this programme, so that code specification generated is unified, error rate is reduced, saves exploitation and the testing time of developer and tester.

Description

Data clearing method, device, equipment and computer readable storage medium
Technical field
The invention mainly relates to database technical fields, specifically, being related to a kind of data clearing method, device, equipment And computer readable storage medium.
Background technique
Tables of data storage in database has a large amount of data, and various data have the characteristic updated, data with the time After being updated to new data, the data before updating are cleared up, and there is the number for largely needing to clear up in database According to;The data that such needs is cleared up are cleared up by thrashing code, and are backed up by backup code, to prevent from accidentally clearing up.
Currently, being write for thrashing code and backup code by developer, different developers have difference Write style so that the thrashing code and backup code thousand for data in each tables of data to be cleared up and backed up are poor Ten thousand not, are not easy to the standardized administration of code, and need tester that the more time is spent to test.
Summary of the invention
The main object of the present invention is to provide a kind of data clearing method, device, equipment and computer readable storage medium, Aim to solve the problem that the data stored in the prior art to tables of data in database, the thrashing code write according to developer with And backup code is cleared up and is backed up, and the standardized management of code is unfavorable for, and cause tester spend more time into The problem of row test.
To achieve the above object, the present invention provides a kind of data clearing method, and the data clearing method includes following step It is rapid:
When receiving cleaning file, table name information, owner's information, field information, column name in the cleaning file are read Information and cleaning conditional information;
The table name information, owner's information and field information are formed into table information for clearance, and according to described for clearance Table information generates backup code;
The table name information, column name information and cleaning conditional information are formed into cleaning information, and believed according to the cleaning Breath generates thrashing code;
Control thrashing code operation, in the corresponding table for clearance of the table name information with the cleaning conditional information Matched target data is cleared up, and controls the backup code operation, is backed up to the target data.
Preferably, described the step of generating backup code according to the table information for clearance, includes:
By in the table information for clearance owner's information and table name information be added in default gauge outfit sentence, generate gauge outfit Code;
Field information in the information for clearance is added in preset content sentence, generation table content code, and After the gauge outfit code and the table content code are spliced, default end of identification is added, backup code is generated;
The routing information in cleaning file is read, creates backup file in the path guided by the routing information, And by backup code storage into the backup file.
Preferably, include: after described the step of generating thrashing code according to the cleaning information
Read table name mark corresponding with the table name information, column name information and cleaning conditional information in the cleaning information Know, column name identifies and condition identity;
According to table name mark, column name mark and condition identity by the table name information, column name information and cleaning Conditional information is added in default rollback sentence, generates rollback code;
Rollback file is created in the path guided by the routing information, and the rollback code is stored to described Rollback file.
Preferably, described the step of controlling the backup code operation, backing up to the target data, includes:
According to path guided by the routing information, the backup code is called from the backup file;
The backup code operation is controlled, generates backup table corresponding with the table name information, and read the number of targets According to being transferred in the backup table, to be backed up to the target data;
The mode of operation of cleaning and the backup of the target data is detected, and is target data distribution and the operation The corresponding identifier of state.
Preferably, include: after described the step of being backed up to the target data
When receiving rollback request, according to path guided by the routing information, adjusted from the rollback file With the rollback code;
The rollback code operation is controlled, searches the backup table, and according to the target data each in the backup table Entrained identifier, determine in the target data to rollback data;
After rollback data carry out rolling back action, the identifier is being updated to described.
Preferably, described to include: later by rollback code storage to the step of rollback file
Backup name information corresponding with the backup file, rollback file and thrashing code is read respectively, is returned Rolling name information and cleaning name information;
It names information, rollback name information and cleaning name information to be added in default document the backup, generates Dispose document.
Preferably, include: after described the step of generating backup code according to the table information for clearance
It is identified according to preset authorization, determines the authorization unit lattice in the cleaning file, and read the authorization unit lattice In character string information, the character string information is added in preset authorization sentence, generate authorization sentence;
The authorization sentence is added in the backup code, the backup code is updated.
In addition, to achieve the above object, the present invention also proposes a kind of data scrubbing device, the data scrubbing device packet It includes:
Read module, for when receiving cleaning file, read table name information in the cleaning file, owner's information, Field information, column name information and cleaning conditional information;
First generation module, for the table name information, owner's information and field information to be formed table information for clearance, And backup code is generated according to the table information for clearance;
Second generation module, for the table name information, column name information and cleaning conditional information to be formed cleaning information, And thrashing code is generated according to the cleaning information;
Control module, for controlling thrashing code operation, in the corresponding table for clearance of the table name information with institute State the cleaning matched target data of conditional information to be cleared up, and control backup code operation, to the target data into Row backup.
In addition, to achieve the above object, the present invention also proposes a kind of data cleanser device, the data cleanser device packet It includes: memory, processor, communication bus and the data scrubbing program being stored on the memory;
The communication bus is for realizing the connection communication between processor and memory;
The processor is for executing the data scrubbing program, to perform the steps of
When receiving cleaning file, table name information, owner's information, field information, column name in the cleaning file are read Information and cleaning conditional information;
The table name information, owner's information and field information are formed into table information for clearance, and according to described for clearance Table information generates backup code;
The table name information, column name information and cleaning conditional information are formed into cleaning information, and believed according to the cleaning Breath generates thrashing code;
Control thrashing code operation, in the corresponding table for clearance of the table name information with the cleaning conditional information Matched target data is cleared up, and controls the backup code operation, is backed up to the target data.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer readable storage medium Storage medium be stored with one perhaps more than one program the one or more programs can by one or one with On processor execute to be used for:
When receiving cleaning file, table name information, owner's information, field information, column name in the cleaning file are read Information and cleaning conditional information;
The table name information, owner's information and field information are formed into table information for clearance, and according to described for clearance Table information generates backup code;
The table name information, column name information and cleaning conditional information are formed into cleaning information, and believed according to the cleaning Breath generates thrashing code;
Control thrashing code operation, in the corresponding table for clearance of the table name information with the cleaning conditional information Matched target data is cleared up, and controls the backup code operation, is backed up to the target data.
The data clearing method of the present embodiment works out the cleaning for clearing up stored data in tables of data by developer File simultaneously uploads, and when receiving the cleaning file of the upload, reads table name information therein, owner's information, field information, column Name information and cleaning conditional information, and table name information, owner's information and field information are formed into table information for clearance, and then foundation The table information for clearance generates backup code;Table name information, column name information and cleaning conditional information are formed into cleaning information simultaneously, And thrashing code is generated according to cleaning information;It controls thrashing code to run, in the table for clearance for characterizing table name information and clearly The matched target data of reason information institute is cleared up, while controlling backup code operation, is carried out to the target data cleared up standby Part, to prevent the mistake cleaning to target data.Thrashing code and backup code pass through in the cleaning file read in this programme Multiclass information generate, wherein table name, column name and the cleaning of tables of data where the data that cleaning information representation needs to clear up Condition so that according to cleaning the generated thrashing code of information can be according to cleaning condition accurately to the data in table name, column name It is cleared up;And the table name of tables of data, the tables of data corresponding owner where the data that information representation for clearance needs to clear up And field, so that the data place tables of data one that can be created according to information for clearance backup code generated and need to clear up The backup table of cause, and then realize the accurate backup to cleaning data;Developer is avoided to thrashing code and backup code It writes, so that thrashing code generated and backup code are unified and standard, reduces error rate, save developer and test The exploitation of personnel and testing time.
Detailed description of the invention
Fig. 1 is the flow diagram of data clearing method first embodiment of the invention;
Fig. 2 is the functional block diagram of data scrubbing device first embodiment of the invention;
Fig. 3 is the device structure schematic diagram for the hardware running environment that present invention method is related to.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The present invention provides a kind of data clearing method.
Fig. 1 is please referred to, Fig. 1 is the flow diagram of data clearing method first embodiment of the present invention.In the present embodiment, The data clearing method includes:
Step S10 reads table name information in the cleaning file, owner's information, field letter when receiving cleaning file Breath, column name information and cleaning conditional information;
Data clearing method of the invention is applied to server, suitable for being deposited by server to tables of data database The data put are cleared up.Specifically, when the data stored in the tables of data to database have cleaning demand, by exploit person Member first writes cleaning file, which is the file present in the form of EXCEL, wherein being set with cleaning required for characterization Data where tables of data every content, the table name information of tables of data, place tables of data as where needing the data cleared up Column name information, owner's information of the place tables of data, field information and the cleaning condition to cleaning data of specific data column Information etc..When receiving the cleaning file that developer is uploaded, table name information, Shu Zhuxin are characterized according to presetting Breath, field information, column name information and the identifier for clearing up conditional information place cell read the table name letter in cleaning file Breath, owner's information, field information, column name information and cleaning conditional information;To determine the number for needing to clear up according to table name information According to the title of place tables of data, and there is the owner user of each operating right by owner's information representation to the tables of data, by word Segment information characterizes field name possessed by the tables of data, the data column where the data that You Lieming information representation needs to clear up, by Clear up the data of required cleaning in conditional information characterize data table.
The table name information, owner's information and field information are formed table information for clearance by step S20, and according to institute It states table information for clearance and generates backup code;
Further, the table name information of reading, owner's information and field information are formed into table information for clearance, it should be to clear The information that reason table information representation needs tables of data where the data cleared up to have, the data mesh for wherein needing to clear up in tables of data Marking data indicates, the tables of data where target data is indicated with table for clearance.It is previously provided with the language for generating backup code Sentence, the table information for clearance of formation is transferred in the pre-set sentence, to generate the sentence for being used to form backup code; Wherein backup code is for creating backup table, and the target data through clearing up is transferred in backup table and is backed up, to avoid Maloperation to target data.Specifically, the step of generating backup code according to table information for clearance includes:
Step S21, by the table information for clearance owner's information and table name information be added in default gauge outfit sentence, Generate gauge outfit code;
Understandably, because backup code is for generating to the backup table that target data is backed up in table for clearance, backup Table structure having the same between table and table for clearance, for the backup code of creation backup table is needed according to table for clearance Table information generate.Specifically, include default gauge outfit sentence in pre-set sentence, the owner in table information for clearance is believed Breath and table name information are added in the default gauge outfit sentence, generate gauge outfit code, the gauge outfit code is for generating backup table Gauge outfit, owner's information and table name information because of gauge outfit code according to table for clearance generate, so that the gauge outfit of backup table generated It is identical with the gauge outfit of table for clearance.Wherein presetting gauge outfit sentence can be with " " CREATE TABLE "+table owner table name+" (" " Form exists, and owner's information and table name information are added in default gauge outfit sentence to the position where " table owner " and " table name ", Being formed by sentence is the gauge outfit code for being used to generate gauge outfit.
Field information in the information for clearance is added in preset content sentence by step S22, generates table content generation Code, and after being spliced the gauge outfit code and the table content code, default end of identification is added, backup generation is generated Code;
Further, backup table is during backing up target data, in addition to need where target data to It clears up except the gauge outfit of table, it is also necessary to have the table structure content of table for clearance;Table structure content characterizes to be had in table for clearance Each field name, embody each data type stored in table for clearance.It further include preset content in pre-set sentence Field information in information for clearance is added in preset content sentence by sentence, generates table content code;The table content code For generating the table structure content of backup table, because table content code is generated according to literary name segment information for clearance, so that generated The table structure content of backup table and the table structure content of table for clearance are identical.Hereafter, by gauge outfit code generated and table content Code is spliced, and adds default end of identification at the end for being formed by splicing code through concatenation, is generated with characterizing For creating the backup code of backup table.
Step S23 reads the routing information in cleaning file, creates backup in the path guided by the routing information File, and by backup code storage into the backup file.
Understandably, backup code generated is stored, and stores and is carried out according to routing information, i.e., will Backup code is stored in path specified by routing information.It is preset in cleaning file specific for placing path letter The cell of breath, the specific cell are set with path identifier;After generating backup code, read according to the path identifier The routing information in cleaning file is taken, and creates backup file, the backup file in the path guided by routing information As store the file of backup code generated.Backup file is named by backup name information simultaneously, the backup Name information can be preset in the default cell of cleaning file, can also be uploaded by developer.I.e. with literary from cleaning The mode that the backup name information that developer is uploaded is read or received in the default cell of part gets backup name Information, and backup file is named with backup name information.It, will be generated after to the name of backup file Backup code, which is added in the backup file, to be stored, in order to which the subsequent backup code according to the storage is to the number of cleaning According to progress backup operation.
In view of in the management process of tables of data each in database, in order to ensure the safety of data in each tables of data Property, it needs to limit operating right of each user to tables of data, i.e., divides the difference of paired data table for different users Operating right.And to the backup table that the data of cleaning are backed up, it is substantially interim table, in the data scrubbing completion to cleaning After a period of time, if the data of each tables of data operate normally in database, it can determine that the data cleared up and completed do not influence number According to the normal operation in library;And delete operation can be carried out to the data backed up in backup table.The operating right of the backup table is built It stands under casual user, to be managed collectively;And the operating right of different tables for clearance belongs to different users and owns, by this Have the user of operating right as cleaning user in table for clearance, the operation for the ease of cleaning user to backup table needs Authorize the operating right of backup table to cleaning user, which carries out after generating backup code.Specifically, according to Include: after the step of cleaning table information generation backup code
Step q1, identifies according to preset authorization, determines the authorization unit lattice in the cleaning file, and read the authorization The character string information is added in preset authorization sentence by the character string information in cell, generates authorization sentence;
Authorization unit lattice are preset in cleaning file, and configure corresponding preset authorization mark for the authorization unit lattice Know;After generating backup code, the mark of each unit lattice in cleaning file is read out, and sets by the mark of reading and in advance Fixed preset authorization identifies comparison, when the mark of reading is consistent with preset authorization mark, then illustrates the list in the mark institute source First lattice are authorization unit lattice.Read the character string information in authorization unit lattice, the use authorized required for character string information characterization Name in an account book and its corresponding authorization type, the user that as character string information is characterized authorize corresponding with authorization type to backup table Operating right.Specifically, Authorized operation is realized by authorization sentence, is previously provided with the preset authorization for generating authorization sentence The character string information of reading is added in the preset authorization sentence by sentence, that is, authorization sentence is produced, to be by authorization sentence User's distribution is cleared up to the operating right of backup table.
The authorization sentence is added in the backup code, is updated to the backup code by step q2.
Further, the authorization sentence of generation is added in backup code, backup code is updated, so that backup Code operation, while generating backup table, to cleaning user addition to the operating right of backup table;It is treated in order to clear up user While the data for needing to clear up in cleaning table carry out cleaning operation, backup table is operated;Wherein to backup table handling packet It includes and the data for needing to clear up in table for clearance are transferred to backup table back up, the number that the needs backed up in backup table are cleared up According to complete deletion, and that the data that the needs backed up in backup table are cleared up are restored to table for clearance is medium.
The table name information, column name information and cleaning conditional information are formed cleaning information by step S30, and according to institute It states cleaning information and generates thrashing code;
Further, the table name information read from cleaning file, list information and cleaning conditional information are formed Information is cleared up, with the title for the data place tables of data for needing to clear up by cleaning information representation, and the number in the tables of data According to the title of column, the target data that the data of cleaning conditional information as need to clear up is met in data column.It presets There is the default cleaning sentence for generating thrashing code, the cleaning information of formation is transferred in the default cleaning sentence Generate the cleaning sentence for being used to form thrashing code.And then the thrashing code of generation and backup code are stored in same paths letter In the guided path of breath, is cleared up and backed up to characterize thrashing code and backup code for the data in identical data table Operation.Specifically, creation cleaning file in the path guided by routing information, the cleaning file are to store to be generated Thrashing code file.Simultaneously by cleaning name information to cleaning file name, which names information can be preparatory It is arranged in the default cell of cleaning file, can also be uploaded by developer.I.e. with from cleaning file default cell The mode for the cleaning name information that middle reading or reception developer are uploaded gets cleaning name information, and with the cleaning Name information is named cleaning file.After to cleaning file name, thrashing code generated is added to It is stored in the cleaning file, in order to which the data that the subsequent thrashing code according to the storage clears up needs are cleared up Operation.
Step S40 controls thrashing code operation, in the corresponding table for clearance of the table name information with the cleaning The matched target data of conditional information is cleared up, and controls the backup code operation, is backed up to the target data.
Further, after generating thrashing code, needs can be cleared up to realize by calling the operation of thrashing code Data carry out cleaning operation.Specifically, after the operation of control thrashing code, table name information searching is first passed through to needing to carry out cleaning behaviour The table for clearance made, and then the data column for needing to carry out cleaning operation are searched from table for clearance according to column name information;To data Data in column are read out, and by the data of reading and cleaning conditional information comparison, judge the data and cleaning condition read Whether information matches;If the data and cleaning conditional information matching that read, illustrate that the data of the reading are to be cleared up Target data, and cleaning operation is carried out to it;And work as the data read and cleaning conditional information mismatch, then illustrate reading Data do not need to be cleared up;When all data in data column are read out, and compared with cleaning conditional information Match, then completes the cleaning operation to target data in table for clearance.
During clearing up target data, it is understood that there may be maloperation will not need cleaning data as Target data is cleared up, and leads to the data exception in tables of data;In order to avoid it is such the occurrence of, be provided with backup machine System, recovery operation can be carried out to the data for not needing cleaning by the data of backup when there is maloperation.Specifically, exist While cleaning target data, control backup code operation generates backup table, the target data of cleaning is transferred to It is backed up in the backup table;Wherein controlling the step of backup code runs, backs up to target data includes:
Step S41 calls the backup generation according to path guided by the routing information from the backup file Code;
Understandably, backup code generated is stored in backup file, and backs up file to back up name letter Breath is named, and the path as guided by routing information embody the backup file where position;To need to call When backup code is run, first according to path guided by routing information, find to back up the backup file of name information name Folder, and then the backup code in backup file is called, it is standby to be carried out by target data of the backup code to cleaning Part.
Step S42 controls the backup code operation, generates backup table corresponding with the table name information, and read institute It states target data to be transferred in the backup table, to back up the target data;
Further, control backup code operation, generates backup table corresponding with table name information, i.e., determines table name information For the title of backup table, so that the title of backup table generated is consistent with the title of target data institute derived data table, with body Backup features of the existing backup table to target data institute derived data table.The target data cleared up is read out simultaneously, and will The target data of reading is transferred in the backup table, carries out backup operation.
Step S43, detects the mode of operation of cleaning and the backup of the target data, and for target data distribution with The corresponding identifier of the mode of operation.
It backs up successfully and clears up successfully, Yi Jibei in view of existing during being cleared up target data and being backed up Part is successful but clears up failed situation, using the two classes situation as the mode of operation to cleaning and backup, i.e. mode of operation packet It includes backup and cleaning is successful, and back up successfully but clear up unsuccessful two states.For two different modes of operation It sets different identifiers to be characterized, such as be characterized for the successful setting identification symbol f1 of backup and cleaning, and for standby Part, which succeeds but clears up unsuccessful setting identification symbol f2, to be characterized.After being backed up to target data, detect its cleaning and The mode of operation of backup, and identifier corresponding with mode of operation is distributed to target data, to characterize the cleaning to target data And backup instances.After all target datas for carrying out cleaning operation are backed up, then terminate the cleaning behaviour to target data Make;While realization by backup operation to target data cleaning, maloperation is prevented.
The data clearing method of the present embodiment works out the cleaning for clearing up stored data in tables of data by developer File simultaneously uploads, and when receiving the cleaning file of the upload, reads table name information therein, owner's information, field information, column Name information and cleaning conditional information, and table name information, owner's information and field information are formed into table information for clearance, and then foundation The table information for clearance generates backup code;Table name information, column name information and cleaning conditional information are formed into cleaning information simultaneously, And thrashing code is generated according to cleaning information;It controls thrashing code to run, in the table for clearance for characterizing table name information and clearly The matched target data of reason information institute is cleared up, while controlling backup code operation, is carried out to the target data cleared up standby Part, to prevent the mistake cleaning to target data.Thrashing code and backup code pass through in the cleaning file read in this programme Multiclass information generate, wherein table name, column name and the cleaning of tables of data where the data that cleaning information representation needs to clear up Condition so that according to cleaning the generated thrashing code of information can be according to cleaning condition accurately to the data in table name, column name It is cleared up;And the table name of tables of data, the tables of data corresponding owner where the data that information representation for clearance needs to clear up And field, so that the data place tables of data one that can be created according to information for clearance backup code generated and need to clear up The backup table of cause, and then realize the accurate backup to cleaning data;Developer is avoided to thrashing code and backup code It writes, so that thrashing code generated and backup code are unified and standard, reduces error rate, save developer and test The exploitation of personnel and testing time.
Further, in another embodiment of data scrubbing of the present invention, described generated according to the cleaning information is cleared up Include: after the step of code
Step a1 is read corresponding with the table name information, column name information and cleaning conditional information in the cleaning information Table name mark, column name mark and condition identity;
Understandably, the data cleared up for maloperation need to be restored in the table for clearance where its script, this is extensive Multiple operation is realized by rollback code.Specifically, the table name information, column name information and cleaning conditional information point of cleaning information are formed Table name mark, column name mark and condition identity are not carried, and three is formed in cleaning information and is distinguished.It is cleared up generating After code, table name mark corresponding with table name information, column name information and cleaning conditional information, column name in cleaning information are read Mark and condition identity, to distinguish table name information, column name information and cleaning conditional information according to three.
Step a2, according to the table name mark, column name mark and condition identity by the table name information, column name information with And cleaning conditional information is added in default rollback sentence, generates rollback code;
Further, it is previously provided with the default rollback sentence for generating rollback code, the table name in information will be cleared up Information, column name information and cleaning conditional information are added in the default rollback sentence, that is, produce and be used to form rollback code Rollback sentence;The cleaning information representation of the generation rollback code will have in table name information backup table, according to cleaning condition The Backup Data that information is cleared up be restored to the data of table for clearance corresponding to table name information and column name information column in.Because clear Reason information be related to table name information, column name information and cleaning conditional information three, during addition, need to three into Row is distinguished;Specifically, it presets in rollback sentence and is provided with the different coding for characterizing different information, such as respectively with coding g1, g2, g3 Characterize table name information, column name information and cleaning conditional information;So as to which different information is added to according to the mark read Default rollback sentence, the table name mark such as read, column name mark and condition identity are respectively w1, w2 and w3, then can will carry There is the table name information of w1 to be added to the position encoded where g1 in default rollback sentence, the column name information for carrying w2 is added to The position where g2 is encoded in default rollback sentence, the cleaning conditional information for carrying w3, which is added in default rollback sentence, to be encoded Position where g3.After table name information, column name information and cleaning conditional information are added in default rollback sentence, i.e., Produce the rollback code for being restored to the data being cleaned in backup table.
Step a3 creates rollback file in the path guided by the routing information, and the rollback code is deposited Store up the rollback file.
Further, it after generating rollback code, needs to carry out storage operation to rollback code generated, and should Rollback code and backup code are stored in path guided by same paths information, to characterize rollback code and backup code needle Cleaning and recovery operation are carried out to identical data.Specifically, rollback file is created in the path guided by routing information, The rollback file is the file for storing rollback code generated.Name information to rollback file by rollback simultaneously Name, rollback name information can be preset in the default cell of cleaning file, can also be uploaded by developer.I.e. It is obtained in a manner of reading or receiving the rollback name information that developer is uploaded in the default cell from cleaning file Information is named to rollback, and rollback file is named with rollback name information.After to the name of rollback file, Rollback code generated is added in the rollback file and is stored, in order to the subsequent rollback code according to the storage Rolling back action is carried out to the data of cleaning.
Understandably, the thrashing code of generation, backup code and rollback code are unable to individualism, need to be deployed to tool In the projects of body, the operation of each program is realized by the scheduling of projects.The deployment operation realized by deployment document, specifically Rollback code storage to the step of rollback file is included: by ground later
Step a4 reads backup name corresponding with the backup file, rollback file and thrashing code respectively Information, rollback name information and cleaning name information;
The backup is named information, rollback name information and cleaning name information to be added to default document by step a5 In, generate deployment document.
Further, in order to generate deployment document, to the cleaning text where thrashing code, backup code and rollback code Cleaning name information, backup name information corresponding to part folder, backup file and rollback file and rollback name letter Breath is read out, and names information, backup name information and rollback name information to be added to default text the cleaning of the reading In shelves, deployment document is generated;Wherein presetting document is the file preset for generating deployment document.Pass through the portion of the generation Thrashing code, backup code and rollback code are deployed in projects by administration's document, and projects are according in deployment document Cleaning name information, backup name information and the corresponding cleaning file of rollback name information searching, backup file with And rollback file;And then call the code in each file run, with realization to the data for needing to clear up in tables of data into Row cleaning, backup, and the data of cleaning are restored;It realizes in tables of data while data scrubbing, to because of maloperation And the data cleared up are restored.
Further, described that the target data is backed up in another embodiment of data scrubbing of the present invention Include: after step
Step b1, when receiving rollback request, according to path guided by the routing information, from the rollback file The rollback code is called in folder;
Understandably, after the completion of the target data cleaning for needing to clear up, other data in tables of data are as basic data It uses in all fields;When in target data exist cleared up by maloperation data when, can cause basic data using different Often, it needs to carry out recovery operation to the target data of cleaning, i.e. control rollback code operation carries out rollback.By developer or Operation maintenance personnel triggers rollback request, and when receiving rollback request, then explanation has restores the data of cleaning from backup table Demand;First from the path that rollback code is stored, i.e., in path guided by routing information, it is right with rollback name information institute to search The file answered, this document folder are the rollback file that storage has rollback code;The calls rollback code from rollback file, To restore to the target data of cleaning.
Step b2 controls the rollback code operation, searches the backup table, and according to the mesh each in the backup table Mark data entrained by identifier, determine in the target data to rollback data;
Further, control rollback code operation, according to table name information searching backup code backup table generated, and it is right Each target data for being backed up in backup table according to the identifier respectively carried, determine wherein restored to rollback number According to.Because in cleaning and backup procedure, for backing up and clearing up successful target data and back up successfully but clear up not Successful target data is set with different identifiers;Unsuccessful target data is wherein backed up successfully but clears up, because clearing up not Success, so that the target data of needs cleaning is still present in table for clearance, without carrying out to such target data Restore, only backup and the successful target data of cleaning are restored.Successful target data will be backed up and clear up to make To read the identifier of backup and the successful mode of operation of cleaning to rollback data, by the number of targets with the class identifier According to be determined as need restore to rollback data.
Step b3 after rollback data carry out rolling back action, is being updated the identifier to described.
Further, it is determining after rollback data, rollback code waits for that rollback data carry out rolling back action to this, reads It is transferred in the data column of table for clearance to rollback data, and by it according to column name information;Data column are to clear up to rollback Data column before cleaning in the table for clearance of place, will be restored to the initial position of table for clearance to rollback data.To rollback The state of operation is detected, and is directed to different rollback state more new identifiers;It, i.e., will be to rollback number when rolling back action success According to the initial position for being successfully restored to table for clearance, then identifier is updated to the state that characterization is successfully recovered;Work as rolling back action Failure, i.e., will not successfully be restored to the initial position of table for clearance to rollback data, then identifier is updated to characterization and restored The state of failure;The different conditions updated by identifier embody the success or failure of rolling back action, are convenient for operation maintenance personnel pair The respectively maintenance to rollback data.
In addition, referring to figure 2., the present invention provides a kind of data scrubbing device, it is real in data scrubbing device first of the present invention It applies in example, the data scrubbing device includes:
Read module 10, for reading table name information, Shu Zhuxin in the cleaning file when receiving cleaning file Breath, field information, column name information and cleaning conditional information;
First generation module 20 is believed for the table name information, owner's information and field information to be formed table for clearance Breath, and backup code is generated according to the table information for clearance;
Second generation module 30, for the table name information, column name information and cleaning conditional information to be formed cleaning letter Breath, and thrashing code is generated according to the cleaning information;
Control module 40, for controlling thrashing code operation, in the corresponding table for clearance of the table name information with The matched target data of the cleaning conditional information is cleared up, and controls the backup code operation, to the target data It is backed up.
The data scrubbing device of the present embodiment works out the cleaning for clearing up stored data in tables of data by developer File simultaneously uploads, when receiving the cleaning file of the upload, read module 10 read table name information therein, owner's information, Field information, column name information and cleaning conditional information, and the first generation module 20 is by believing table name information, owner's information and field Breath forms table information for clearance, and then generates backup code according to the table information for clearance;The second generation module 30 is by table simultaneously Name information, column name information and cleaning conditional information form cleaning information, and generate thrashing code according to cleaning information;Control module 40 control thrashing codes are run, and are carried out in the table for clearance characterized to table name information with the matched target data of cleaning information Cleaning, while backup code operation is controlled, the target data cleared up is backed up, to prevent the mistake to target data clear Reason.The multiclass information that thrashing code and backup code pass through in the cleaning file read in this programme generates, wherein cleaning letter Table name, column name and the condition of cleaning of tables of data where breath characterizes the data for needing to clear up, so that according to cleaning information institute Generating thrashing code accurately can clear up the data in table name, column name according to cleaning condition;And information representation for clearance Table name, the corresponding owner of the tables of data and the field of tables of data where the data for needing to clear up, so that according to letter for clearance The data place consistent backup table of tables of data that backup code generated can create and need to clear up is ceased, and then is realized to cleaning The accurate backup of data;It avoids developer to write thrashing code and backup code, so that thrashing code generated It is unified and standard with backup code, error rate is reduced, exploitation and the testing time of developer and tester are saved.
Further, in another embodiment of data scrubbing device of the present invention, first generation module includes:
Adding unit, for by the table information for clearance owner's information and table name information be added to default gauge outfit language In sentence, gauge outfit code is generated;
Generation unit generates table for the field information in the information for clearance to be added in preset content sentence Content code, and after being spliced the gauge outfit code and the table content code, default end of identification is added, is generated standby Part code;
Storage unit is created in the path guided by the routing information for reading the routing information in cleaning file Backup file is built, and by backup code storage into the backup file.
Further, in another embodiment of data scrubbing device of the present invention, the data scrubbing device further include:
Module is obtained, is believed for reading in the cleaning information with the table name information, column name information and cleaning condition Cease corresponding table name mark, column name mark and condition identity;
Adding module, for according to table name mark, column name mark and condition identity by the table name information, column name Information and cleaning conditional information are added in default rollback sentence, generate rollback code;
Memory module, for creating rollback file in the path guided by the routing information, and by the rollback Code is stored to the rollback file.
Further, in another embodiment of data scrubbing device of the present invention, the control module is also used to:
According to path guided by the routing information, the backup code is called from the backup file;
The backup code operation is controlled, generates backup table corresponding with the table name information, and read the number of targets According to being transferred in the backup table, to be backed up to the target data;
The mode of operation of cleaning and the backup of the target data is detected, and is target data distribution and the operation The corresponding identifier of state.
Further, in another embodiment of data scrubbing device of the present invention, the data scrubbing device further include:
Calling module, for when receive rollback request when, according to path guided by the routing information, from described time The rollback code is called in rolling file;
Searching module searches the backup table, and according to each in the backup table for controlling the rollback code operation Identifier entrained by the target data, determine in the target data to rollback data;
First update module, for after rollback data carry out rolling back action, being carried out more to the identifier to described Newly.
Further, in another embodiment of data scrubbing device of the present invention, the data scrubbing device further include:
Handling module, it is corresponding standby with the backup file, rollback file and thrashing code for reading respectively Part name information, rollback name information and cleaning name information;
Third generation module, for information, rollback name information and cleaning name information addition to be named in the backup Into default document, deployment document is generated.
Further, in another embodiment of data scrubbing device of the present invention, the data scrubbing device further include:
Determining module determines the authorization unit lattice in the cleaning file, and read institute for identifying according to preset authorization The character string information in authorization unit lattice is stated, the character string information is added in preset authorization sentence, generates authorization sentence;
Second update module, for the authorization sentence to be added in the backup code, to the backup code into Row updates.
Wherein, each virtual functions module of above-mentioned data scrubbing device is stored in the storage of data cleanser device shown in Fig. 3 In device 1005, when processor 1001 executes data scrubbing program, the function of modules in embodiment illustrated in fig. 2 is realized.
Referring to Fig. 3, Fig. 3 is the device structure schematic diagram for the hardware running environment that present invention method is related to.
Data cleanser device of the embodiment of the present invention can be PC (personal computer, personal computer), can also be with It is the terminal devices such as smart phone, tablet computer, E-book reader, portable computer.
As shown in figure 3, the data cleanser device may include: processor 1001, such as CPU (Central Processing Unit, central processing unit), memory 1005, communication bus 1002.Wherein, communication bus 1002 for realizing Connection communication between processor 1001 and memory 1005.Memory 1005 can be high-speed RAM (random access Memory, random access memory), it is also possible to stable memory (non-volatile memory), such as disk storage Device.Memory 1005 optionally can also be the storage device independently of aforementioned processor 1001.
Optionally, which can also include user interface, network interface, camera, RF (Radio Frequency, radio frequency) circuit, sensor, voicefrequency circuit, WiFi (Wireless Fidelity, WiMAX) module etc.. User interface may include display screen (Display), input unit such as keyboard (Keyboard), and optional user interface can be with Including standard wireline interface and wireless interface.Network interface optionally may include standard wireline interface and wireless interface (such as WI-FI interface).
It will be understood by those skilled in the art that data cleanser device structure shown in Fig. 3 not clear up by structure paired data The restriction of equipment may include perhaps combining certain components or different component cloth than illustrating more or fewer components It sets.
As shown in figure 3, as may include operating system, net in a kind of memory 1005 of computer readable storage medium Network communication module and data scrubbing program.Operating system is to manage and control the journey of data cleanser device hardware and software resource Sequence supports the operation of data scrubbing program and other softwares and/or program.Network communication module is for realizing memory 1005 Communication between internal each component, and communicated between hardware and softwares other in data cleanser device.
In data cleanser device shown in Fig. 3, processor 1001 is used to execute the data stored in memory 1005 clear Program is managed, realizes the step in each embodiment of above-mentioned data clearing method.
The present invention provides a kind of computer readable storage medium, the computer-readable recording medium storage have one or More than one program of person, the one or more programs can also be executed by one or more than one processor with Step in the above-mentioned each embodiment of data clearing method of realization.
It should also be noted that, herein, the terms "include", "comprise" or its any other variant are intended to non- It is exclusive to include, so that the process, method, article or the device that include a series of elements not only include those elements, It but also including other elements that are not explicitly listed, or further include solid by this process, method, article or device Some elements.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including There is also other identical elements in the process, method of the element, article or device.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art The part contributed out can be embodied in the form of software products, which is stored in one as described above In computer readable storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can To be mobile phone, computer, server or the network equipment etc.) execute method described in each embodiment of the present invention.
The above description is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all at this Under the design of invention, using equivalent structure transformation made by description of the invention and accompanying drawing content, or directly/it is used in it indirectly He is included in scope of patent protection of the invention relevant technical field.

Claims (10)

1. a kind of data clearing method, which is characterized in that the data clearing method the following steps are included:
When receiving cleaning file, table name information, owner's information, field information, column name information in the cleaning file are read And cleaning conditional information;
The table name information, owner's information and field information are formed into table information for clearance, and believed according to the table for clearance Breath generates backup code;
The table name information, column name information and cleaning conditional information are formed into cleaning information, and raw according to the cleaning information At thrashing code;
The thrashing code operation is controlled, is matched in the corresponding table for clearance of the table name information with the cleaning conditional information Target data cleared up, and control backup code operation, the target data backed up.
2. data clearing method as described in claim 1, which is characterized in that described standby according to the table information generation for clearance The step of part code includes:
By in the table information for clearance owner's information and table name information be added in default gauge outfit sentence, generate gauge outfit generation Code;
Field information in the information for clearance is added in preset content sentence, generates table content code, and by institute It states gauge outfit code and after the table content code spliced, adds default end of identification, generate backup code;
The routing information in cleaning file is read, backup file is created in the path guided by the routing information, and will The backup code storage is into the backup file.
3. data clearing method as claimed in claim 2, which is characterized in that described to generate cleaning generation according to the cleaning information Include: after the step of code
Read in the cleaning information with the table name information, column name information and cleaning conditional information corresponding table name mark, Column name mark and condition identity;
By the table name information, column name information and condition is cleared up according to table name mark, column name mark and condition identity Information is added in default rollback sentence, generates rollback code;
Rollback file is created in the path guided by the routing information, and the rollback code is stored to the rollback File.
4. data clearing method as claimed in claim 3, which is characterized in that the control backup code operation, to institute Stating the step of target data is backed up includes:
According to path guided by the routing information, the backup code is called from the backup file;
The backup code operation is controlled, backup table corresponding with the table name information is generated, and reads the target data and passes It is defeated into the backup table, to be backed up to the target data;
The mode of operation of cleaning and the backup of the target data is detected, and is target data distribution and the mode of operation Corresponding identifier.
5. data clearing method as claimed in claim 4, which is characterized in that the step backed up to the target data Include: after rapid
When receiving rollback request, according to path guided by the routing information, institute is called from the rollback file State rollback code;
The rollback code operation is controlled, searches the backup table, and taken according to the target data each in the backup table The identifier of band, determine in the target data to rollback data;
After rollback data carry out rolling back action, the identifier is being updated to described.
6. data clearing method as claimed in claim 3, which is characterized in that described to store the rollback code to described time Include: after the step of rolling file
Backup name information corresponding with the backup file, rollback file and thrashing code, rollback life are read respectively Name information and cleaning name information;
It names information, rollback name information and cleaning name information to be added in default document the backup, generates deployment Document.
7. data clearing method as claimed in any one of claims 1 to 6, which is characterized in that described to be believed according to the table for clearance Include: after the step of breath generation backup code
It is identified according to preset authorization, determines the authorization unit lattice in the cleaning file, and read in the authorization unit lattice The character string information is added in preset authorization sentence by character string information, generates authorization sentence;
The authorization sentence is added in the backup code, the backup code is updated.
8. a kind of data scrubbing device, which is characterized in that the data scrubbing device includes:
Read module, for reading table name information, owner's information, field in the cleaning file when receiving cleaning file Information, column name information and cleaning conditional information;
First generation module, for the table name information, owner's information and field information to be formed table information for clearance, and root Backup code is generated according to the table information for clearance;
Second generation module, for the table name information, column name information and cleaning conditional information to be formed cleaning information, and root Thrashing code is generated according to the cleaning information;
Control module, for controlling thrashing code operation, in the corresponding table for clearance of the table name information with it is described clear The target data of manage bar part information matches is cleared up, and controls the backup code operation, is carried out to the target data standby Part.
9. a kind of data cleanser device, which is characterized in that the data cleanser device includes: memory, processor, communication bus And it is stored in the data scrubbing program on the memory;
The communication bus is for realizing the connection communication between processor and memory;
The processor is for executing the data scrubbing program, to realize such as data of any of claims 1-7 The step of method for cleaning.
10. a kind of computer readable storage medium, which is characterized in that it is clear to be stored with data on the computer readable storage medium Program is managed, such as data scrubbing of any of claims 1-7 is realized when the data scrubbing program is executed by processor The step of method.
CN201811468867.4A 2018-11-30 2018-11-30 Data cleaning method, device, equipment and computer readable storage medium Active CN109710596B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811468867.4A CN109710596B (en) 2018-11-30 2018-11-30 Data cleaning method, device, equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811468867.4A CN109710596B (en) 2018-11-30 2018-11-30 Data cleaning method, device, equipment and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN109710596A true CN109710596A (en) 2019-05-03
CN109710596B CN109710596B (en) 2023-12-19

Family

ID=66254570

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811468867.4A Active CN109710596B (en) 2018-11-30 2018-11-30 Data cleaning method, device, equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN109710596B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222039A (en) * 2019-05-07 2019-09-10 平安科技(深圳)有限公司 Data storage and garbage data cleaning method, device, equipment and storage medium
CN112905386A (en) * 2021-02-08 2021-06-04 中国工商银行股份有限公司 Table data backup cleaning method and device based on life cycle
CN113485999A (en) * 2021-08-04 2021-10-08 中国工商银行股份有限公司 Data cleaning method and device and server
CN114071389A (en) * 2020-07-31 2022-02-18 中国移动通信集团新疆有限公司 Test verification method and device, computer equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000300797A (en) * 1999-04-20 2000-10-31 Sophia Co Ltd Arithmetic processing device for game and arithmetic processing method for game
US20050028046A1 (en) * 2003-07-31 2005-02-03 International Business Machines Corporation Alert flags for data cleaning and data analysis
US20080243954A1 (en) * 2007-04-02 2008-10-02 International Business Machines Corporation Generating and indicating incremental backup copies from virtual copies of a data set
CN103593352A (en) * 2012-08-15 2014-02-19 阿里巴巴集团控股有限公司 Method and device for cleaning mass data
CN104036001A (en) * 2014-06-13 2014-09-10 上海新炬网络技术有限公司 Dynamic hotlist priority scheduling based quick data cleaning method
US20160246687A1 (en) * 2015-02-25 2016-08-25 Lexmark International Technology Sarl System and Methods of Backing Up and Restoring Database Objects

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000300797A (en) * 1999-04-20 2000-10-31 Sophia Co Ltd Arithmetic processing device for game and arithmetic processing method for game
US20050028046A1 (en) * 2003-07-31 2005-02-03 International Business Machines Corporation Alert flags for data cleaning and data analysis
US20080243954A1 (en) * 2007-04-02 2008-10-02 International Business Machines Corporation Generating and indicating incremental backup copies from virtual copies of a data set
CN103593352A (en) * 2012-08-15 2014-02-19 阿里巴巴集团控股有限公司 Method and device for cleaning mass data
CN104036001A (en) * 2014-06-13 2014-09-10 上海新炬网络技术有限公司 Dynamic hotlist priority scheduling based quick data cleaning method
US20160246687A1 (en) * 2015-02-25 2016-08-25 Lexmark International Technology Sarl System and Methods of Backing Up and Restoring Database Objects

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
林达德;: "用Javabean构建数据过滤的Script代码生成器", 宁波职业技术学院学报, no. 05 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222039A (en) * 2019-05-07 2019-09-10 平安科技(深圳)有限公司 Data storage and garbage data cleaning method, device, equipment and storage medium
CN110222039B (en) * 2019-05-07 2023-09-29 平安科技(深圳)有限公司 Data storage and garbage data cleaning method, device, equipment and storage medium
CN114071389A (en) * 2020-07-31 2022-02-18 中国移动通信集团新疆有限公司 Test verification method and device, computer equipment and storage medium
CN112905386A (en) * 2021-02-08 2021-06-04 中国工商银行股份有限公司 Table data backup cleaning method and device based on life cycle
CN113485999A (en) * 2021-08-04 2021-10-08 中国工商银行股份有限公司 Data cleaning method and device and server

Also Published As

Publication number Publication date
CN109710596B (en) 2023-12-19

Similar Documents

Publication Publication Date Title
CN109710596A (en) Data clearing method, device, equipment and computer readable storage medium
CN104346185B (en) Application object attributes update method, device and application platform
CN108959059B (en) Test method and test platform
CN108845930B (en) Interface operation test method and device, storage medium and electronic device
US10146668B1 (en) Modeling code coverage in software life cycle
CN105138347A (en) Difference upgrade patch generating method, software upgrading method and corresponding apparatus
CN110673936B (en) Breakpoint continuous operation method and device for arrangement service, storage medium and electronic equipment
CN105302533A (en) Code synchronizing method and apparatus
CN112214411B (en) Disaster recovery system testing method, device, equipment and storage medium
CN106648994B (en) Method, equipment and system for backing up operation log
CN111581005B (en) Terminal repairing method, terminal and storage medium
CN109740129B (en) Report generation method, device and equipment based on blockchain and readable storage medium
CN105512562B (en) Vulnerability mining method and device and electronic equipment
CN109684326A (en) Tables of data creation method, device, equipment and computer readable storage medium
CN113535135B (en) Software development method and device, computer equipment and storage medium
CN106445529A (en) Backup method and system for configuration information of continuous integration server
CN111104155A (en) Database script management platform and method, electronic equipment and storage medium
CN104881361A (en) Software development process monitoring and managing method and system
CN111367869A (en) Mirror image file processing method and device, storage medium and electronic equipment
US10146823B2 (en) Relevant-information providing method, relevant-information providing apparatus, and relevant-information providing program
EP3514680A1 (en) Identification of changes in functional behavior and runtime behavior of a system during maintenance cycles
CN110471689A (en) A kind of application program updating method and device
CN104077053B (en) Terminal information operating method and device and mobile terminal
KR101588375B1 (en) Method and system for managing database
CN110908644A (en) Configuration method and device of state node, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant