CN104036001B - Dynamic hotlist priority scheduling based quick data cleaning method - Google Patents
Dynamic hotlist priority scheduling based quick data cleaning method Download PDFInfo
- Publication number
- CN104036001B CN104036001B CN201410264994.8A CN201410264994A CN104036001B CN 104036001 B CN104036001 B CN 104036001B CN 201410264994 A CN201410264994 A CN 201410264994A CN 104036001 B CN104036001 B CN 104036001B
- Authority
- CN
- China
- Prior art keywords
- cleaning
- data
- hotlist
- database
- priority
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a dynamic hotlist priority scheduling based quick data cleaning method. The method includes the following steps: a) prolonging a data cleaning cycle of database table objects, and synchronizing the data cleaning cycle to a management parameter table in a database; b) comparing the management parameter table with a database table, if the table is not is within a cleaning range, rejecting the table, and automatically generating cleaning object details and cleaning scripts; c) deploying database table access frequency collection scripts, making statistics on access frequency changes in real time, dynamically determining hotlists in the cleaning object details, and subjecting the hotlists to priority scheduling; d) performing historical data backup, and performing data cleaning if data are compared to be consistent. By prolonging the data cleaning cycle for the database table objects, passive cleaning is changed into active management and cleaning; by the aid of a dynamic priority scheduling algorithm, the hotlists are processed according to priority, risk of misoperation is reduced, cleaning speed is increased, and overall performance of the database is improved effectively.
Description
Technical field
The present invention relates to a kind of database maintenance method, more particularly to a kind of data dispatched based on hotlist dynamic priority are fast
Fast method for cleaning.
Background technology
In current telecommunication service system, with fusion, cutover etc. the project implementation, carry key business it is provincial in
Heart core business system data base becomes increasingly huge.Especially accounting system, charge system, Jing subsystems etc., with when
Between passage and portfolio growth, quantity database is continuously increased, and data volume is also day by day too fat to move, and data base's bearing pressure is got over
Come bigger, such as develop as one pleases, data base will finally can't bear the heavy load.It is how ageing to data to strengthen management, historical data is entered
Row is effectively administered, and becomes a particularly important problem.
For the continuous growth of data volume, alleviated by increasing storage, if table space alarm is not stored
Can distribute, using the manual interim cleaning solve problem of delete modes or truncate modes.So, become data scrubbing
A kind of cleaning of passive type, effect on driving birds is not good.
On the other hand, in data scrubbing, many artificial operation factors, the system misoperation risk of increase are added
Property, maintenance cost can be greatly improved.So for how data scrubbing is effectively solved, needing synthetically to manage consideration.
Therefore, the management of available data cleaning is not often optimum, and is likely present some hidden danger.It is main
The reason for wanting is as follows：1) most of operation system fails thorough quick in view of the data after putting into operation at the beginning of system design
Growing concern, Data lifecycle management is lack of standardization.2) system is more and more, complexity also more and more higher, and the difficulty of management is got over
Come bigger, basic nobody can do must understand whole system, for the table for needing cleaning is difficult to confirm.
Fig. 1 is referred to, available data clean-up process is as follows：1) whether monitoring personnel monitor in real time table space alerts.2) such as
Fruit table space exceeds threshold values, then checked whether extension storage.3) if extension storage, then table space is increased.If 4) do not had
There is extension storage, then assessing object table spatial data object and be confirmed whether to have can clear up object, write clear after the completion of confirmation
Reason scheme, scheme implements cleaning by rear.5) solve problem is verified whether.
Prior art shortcoming is as follows：1) the data scrubbing mode of passive type, it is impossible to effectively solving problem.2) data scrubbing is true
Recognize that the cycle is long, emergency can not be processed in time.3) manual steps are more, there is maloperation risk, increase O＆M cost.4)
Order according to acquiescence is cleared up, it is impossible to enough to hotlist priority treatment.5) backed up using delete modes and clear up when
Wait, there are many deletion data cases.
The content of the invention
The technical problem to be solved is to provide a kind of data dispatched based on hotlist dynamic priority and is quickly cleared up
Method, can make hotlist priority treatment, be effectively improved database performance, reduce maloperation risk, improve cleaning speed.
The technical scheme that the present invention is adopted to solve above-mentioned technical problem is to provide a kind of based on hotlist dynamic priority tune
The quick method for cleaning of data of degree, comprises the steps：A) the data scrubbing cycle of database table object is increased, and data are clear
The management parameters table of reason cycle synchronisation to data base；B) compared according to management parameters table and database table, using Oracle
Regular expression is compared to common table, manual point of table and partition table, if not in cleanup area if table is rejected, and
Automatically generate cleaning object detail and cleaning script；C) database table access frequency collection script is disposed, it is each in real-time statistics SQL
The access frequency change of tables of data；It is ranked up according to the access frequency of each tables of data, in being dynamically determined cleaning object detail
Hotlist, and priority scheduling is carried out to hotlist；D) carry out historical data backup, after backup if comparing unanimously if to carry out data clear
Reason.
The above-mentioned quick method for cleaning of data dispatched based on hotlist dynamic priority, wherein, also include in the step a)
Increase database table object DELETE, TRUNCATE, PARTITION_TABLE manner of cleaning up and retention time, cleaning field,
The cleaning condition of field type, and it is synchronized to the management parameters table of data base.
The above-mentioned quick method for cleaning of data dispatched based on hotlist dynamic priority, wherein, the step b) includes as follows
Process：B1) by task scheduling obtain specify table space, by naming rule check after, according in management parameters table this specify
In the data scrubbing cycle of table space, generation can be cleared up the date；B2) for general data table, directly according to the data of the tables of data
Cleaning cycle determines whether that data can be cleared up, and if then generating cleaning script, terminates if not；B3) for craft
Divide table, then generating by regular expression needs cleaning table, determines whether that data can be cleared up, if then generating cleaning foot
This, terminates if not；B4) output cleaning object detail and cleaning object script.
The above-mentioned quick method for cleaning of data dispatched based on hotlist dynamic priority, wherein, step b1) in default table
Space is full database data table space.
The above-mentioned quick method for cleaning of data dispatched based on hotlist dynamic priority, wherein, the step c) is robbed using non-
Account for formula dispatching algorithm carries out priority scheduling to hotlist.
The above-mentioned quick method for cleaning of data dispatched based on hotlist dynamic priority, wherein, the step d) includes as follows
Process：D1) automatization's backup is carried out according to the table order that hotlist Priority-driven Scheduling Algorithm is generated；D2 the type of cleaning table) is judged, such as
Fruit is manual point of table or date partition table, then import the whole table of history library backup by data pump or need to clear up subregion, data ratio
To clearing up after zero difference；D3) if cleaning table type is DELETE tables, generated after association middle table using rowid modes,
Again history library backup is imported by data pump and associate middle table, cleared up after comparing zero difference.
Present invention contrast prior art has following beneficial effect：The present invention provide based on hotlist dynamic priority dispatch
The quick method for cleaning of data, by increasing the data scrubbing cycle for database table object, by passive cleaning active management is changed into
And cleaning；Collected by daily SQL access frequencys, hotlist priority treatment is made using dynamically optimized scheduling algorithm, formulate significant figure
According to cleaning mechanism, maloperation risk is reduced, improve cleaning speed；Data scrubbing is made from original unicity to systematization and managementization
Direction is developed, and is effectively improved data base's overall performance.
Description of the drawings
Fig. 1 is the data scrubbing schematic flow sheet of existing database；
Fig. 2 is data quick clean-up process schematic diagram of the present invention based on hotlist dynamic priority scheduling；
Fig. 3 is database schema schematic diagram of the present invention based on the quick cleaning of hotlist dynamic priority scheduling；
Fig. 4 is present invention cleaning object production procedure schematic diagram；
Fig. 5 is the quick clean-up process schematic diagram of data object of the present invention；
Fig. 6 clears up DELETE and represents intention for the present invention using rowid modes.
Specific embodiment
With reference to the accompanying drawings and examples the invention will be further described.
Fig. 2 is data quick clean-up process schematic diagram of the present invention based on hotlist dynamic priority scheduling；Fig. 3 is base of the present invention
In the database schema schematic diagram of the quick cleaning of hotlist dynamic priority scheduling.
Fig. 2 and Fig. 3 is referred to, the quick method for cleaning of data dispatched based on hotlist dynamic priority that the present invention is provided is included
Following steps：
A) data scrubbing cycle of database table object, manner of cleaning up and cleaning condition are increased, and by the data scrubbing cycle
It is synchronized to the management parameters table of data base；Manner of cleaning up includes DELETE, TRUNCATE, PARTITION_ of database table object
TABLE manner of cleaning up, cleaning condition includes retention time, clears up field and field type, and by above-mentioned all parameter synchronization extremely
The management parameters table of data base；
B) according to the data scrubbing cycle of management parameters table, using Oracle regular expressions to common table, manual point of table
Compare with all kinds database table such as partition table, if not in cleanup area if table is rejected, and automatically generate
Cleaning object detail and cleaning script；
C) database table access frequency collection script is disposed, the access frequency change of each tables of data in real-time statistics SQL；Number
Can be divided into day, moon periodic cleaning and access frequency without directly contacting according to cleaning cycle, be carried out according to the access frequency of each tables of data
Sequence, for access frequency comes most front table hotlist is, the hotlist being dynamically determined in cleaning object detail, and hotlist is entered
Row major is dispatched.
The quick method for cleaning of data dispatched based on hotlist dynamic priority that the present invention is provided, is broadly divided into data Life Cycle
Period management stage, cleaning object generation phase, hotlist priority scheduling stage and quick clean-up phase four-stage are realizing.Each rank
Section concrete function is described as follows：
1st, the Data lifecycle management stage：
The stage is mainly responsible for the generation of data life period, by managing application of reaching the standard grade, increases the data for creating object
The related datas such as cleaning cycle, manner of cleaning up, cleaning condition are filled in, and through evaluation, it is synchronous by cleaning after the completion of enforcement of reaching the standard grade
It is inner that object information is configured to data base's " Data lifecycle management parameter list ".Manage in this way by passive confirmation,
It is changed into active process mode.
2nd, object generation phase is cleared up：
The stage is responsible for finding cleaning object and generating cleaning script.Flow process is as shown in Figure 4：
1) task scheduling is performed by storing process, supports to specify table space or full storehouse parameter, generates cleaning object.
2) clear up object and support common table and manual point of table mode.So-called craft point table is referred to is advised by the name of rule
Then, table is divided into into several tables according to days, numeral, letter.For example：UR_BCSUCESS_INFO_ [YYYYMM] table is days
Ending, it is exactly manual point of table so to belong to a class.
3) this programme supports the cleaning of opponent's work point table, quotes the regular expression technology of ORACLE, so needing name
Mode will meet ORACLE regular expression requirements.For example：
UR_BCSUCESS_INFO_ [YYYYMM] is changed to after regular expression：
UR_BCSUCESS_INFO_[[:digit:]]{6}
Reference statement mode：
select'da',owner,table_name
from dba_tables
Where owner=' DBCUSTADM '
and regexp_like(table_name,'^'||’UR_BCSUCESS_INFO_[[:digit:]]{6}’||'
$')；
4) after naming rule inspection, generation can be cleared up the date.
5) common table is checked whether, if common table, determines whether that data can be cleared up, if then generating cleaning foot
This, terminates if not.
6) if manual point of table, then generate table by regular expression, determine whether that data can be cleared up, if then
Cleaning script is generated, is terminated if not.
7) export：Cleaning object is detailed, cleaning object script.
3rd, the hotlist priority scheduling stage：
The stage is responsible for hotlist priority treatment.Comprise the following steps that：
Deployment acquisition tables access frequency, constantly generates every table access frequency.
Dynamic priority dispatching algorithm, using non-preemptive, under this scheduling mode, system once gives multi-frame system
In ready queue after the object of highest priority, the object just can be performed down always, until completion system is just by datatron point
The high ready object of another priority of dispensing.
4th, quick clean-up phase：
The stage is responsible for backing up different types of table, comparing and cleaning operation, and flow process is as shown in Figure 5：
1) table order is generated according to hotlist Priority-driven Scheduling Algorithm, script is backed up and cleared up by automatization and is performed.
Backup method of calling example：
Derive data base call as follows:
nohup ksh expdp.sh-n8-d DIR_DUMP-c tname01.txt-f/arch02/wcrma_expdp-
S/lssj3/wcrma_text-H admin/hisdb/dpdump&
History library calls as follows:
nohup ksh impdp.sh-n8-d DATA_PUMP_DIR-w/oracle10/admin/hisdb/dpdump/-
t WCRMABCV-u WCRMA-k TBS_DATA_HIS_01-p15155910722&
# derived parameter explanations:
-n：Represent and line number
-d：Represent derived dmp files place catalogue
-c：Representing needs derived table, and the tname01.txt files need to be placed under the path specified by-f parameters
-f：Represent that export deposit position is consistent with-d parameter lists
-S：Secondary storage catalogue is passed in expression
-H：History library directory is passed in expression
# imports parameter declaration:
-n：Represent and line number
-d：Represent and import DIR_DUMP directory names
-w：Represent and import history library dmp file paths, be consistent with-d
-t：Represent the TNS of dmp files place data base
-u:Represent and import SCHEMA titles
-k:Expression imports to the table space of history library
-p:Represent the telephone number sent after success
Data purge calls example：
# user's names test, the table name user_info_test of cleaning clears up 3+1 month numbers by update times field
According to each cleaning 50w datas
SQL>exec sp_del_data('test','user_info_test','update_time',3,500000)；
2) judge the type of cleaning table, if manual point of table or date partition table, then carry out whole table or need cleaning point
Area is backed up, and is backed up by data pump importing history library.Cleared up after comparing zero difference.
3) table type is cleared up for DELETE modes, cleared up using rowid modes.
Using rowid mode reasons：Back up and clear up in this work in data scrubbing, due to when BACKUP TIME, comparison
Between and clearance time it is all different, there are three time points, if use date directly clears up data as condition, in scale removal process
It is middle to there is the cleaning data risk inconsistent with comparison data, Backup Data.
Using rowid mode advantages：Data backup and cleaning concordance are ensured, comparison process is reduced, can also improve clear
Reason speed.
It is as shown in Figure 6 using rowid mode operating procedure examples：
Data rowid for needing cleaning are backed up to into middle table, is directed into middle table conditions relevant by data pump and is gone through
Shi Ku, by calling storing process after backup success, will need to clear up the cleaning of rowid related datas in batches.
In sum, the present invention provide based on hotlist dynamic priority dispatch the quick method for cleaning of data, by for count
Increase the data scrubbing cycle according to storehouse table object, active management and cleaning are changed into by passive cleaning；By daily SQL access frequencys
Collect, hotlist priority treatment made using dynamically optimized scheduling algorithm, formulate valid data cleaning mechanism, reduce maloperation risk,
Improve cleaning speed；Data scrubbing is developed from original unicity to systematization and managementization direction, be effectively improved data base whole
Body performance.Concrete advantage is as follows：1) four-stage model is creatively proposed, is carried out from the control of frame keyholed back plate to data management and control actively
Data scrubbing, removes interim cleaning or the simple management mode situation that time-consuming, effect is little from.2) by being adjusted using dynamic priority
Degree, hotlist can be processed most soon, make database performance effectively get a promotion.3) for DELETE is cleared up, propose to adopt rowid side
Formula, ensures data backup and cleaning concordance, and improves cleaning speed.4) when can greatly reduce process using the present invention
Between, if compared with the data volume of 300G, the deadline by 3 days before, within falling below 8 hours, it is to avoid excessive people
Work operating procedure, and for the condition such as the familiarity for the treatment of people system, technical merit degree of dependence is reduced.Strengthen number
According to library management, quick cleaning effect is improved, reduce O＆M cost.
Although the present invention is disclosed as above with preferred embodiment, so it is not limited to the present invention, any this area skill
Art personnel, without departing from the spirit and scope of the present invention, when a little modification and perfect, therefore the protection model of the present invention can be made
Enclose when by being defined that claims are defined.
Claims (5)
1. it is a kind of based on hotlist dynamic priority dispatch the quick method for cleaning of data, it is characterised in that comprise the steps：
A) the data scrubbing cycle of database table object is increased, and by the management parameters of data scrubbing cycle synchronisation to data base
Table；
B) all kinds database table is compared according to the data scrubbing cycle of management parameters table, if not in cleanup area
Then table is rejected, and automatically generates cleaning object detail and cleaning script；
C) database table access frequency collection script is disposed, the access frequency change of each tables of data in real-time statistics SQL；According to each
The access frequency of tables of data is ranked up, the hotlist being dynamically determined in cleaning object detail, and carries out priority scheduling to hotlist；
D) carry out historical data backup, after backup if comparing unanimously if carry out data scrubbing；
The step b) includes following process：
B1) obtained by task scheduling and specify table space, after naming rule is checked, according to the specified table in management parameters table
In the data scrubbing cycle in space, generation can be cleared up the date；
B2) for general data table, directly according to the data scrubbing cycle of the tables of data determine whether that data can be cleared up, if
Having then generate cleaning script, terminates if not；
B3) for manual point of table, then being generated by regular expression needs cleaning table, determines whether that data can be cleared up, if
Having then generate cleaning script, terminates if not；
B4) output cleaning object detail and cleaning object script.
2. it is as claimed in claim 1 to be based on the quick method for cleaning of data that hotlist dynamic priority is dispatched, it is characterised in that described
Also include increasing DELETE, TRUNCATE, PARTITION_TABLE manner of cleaning up and the reservation of database table object in step a)
Time, cleaning field, the cleaning condition of field type, and it is synchronized to the management parameters table of data base.
3. it is as claimed in claim 1 to be based on the quick method for cleaning of data that hotlist dynamic priority is dispatched, it is characterised in that described
Step b1) in acquiescence table space be full database data table space.
4. it is as claimed in claim 1 to be based on the quick method for cleaning of data that hotlist dynamic priority is dispatched, it is characterised in that described
Step c) carries out priority scheduling using non-preemptive scheduling algorithm to hotlist.
5. it is as claimed in claim 1 to be based on the quick method for cleaning of data that hotlist dynamic priority is dispatched, it is characterised in that described
Step d) includes following process：
D1) automatization's backup is carried out according to the table order that hotlist Priority-driven Scheduling Algorithm is generated；
D2 the type of cleaning table) is judged, if manual point of table or date partition table, then import history library backup by data pump
Whole table needs to clear up subregion, is cleared up after comparing zero difference；
D3) if cleaning table type is DELETE tables, generated after association middle table using rowid modes, then led by data pump
Enter history library backup association middle table, cleared up after comparing zero difference.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410264994.8A CN104036001B (en) | 2014-06-13 | 2014-06-13 | Dynamic hotlist priority scheduling based quick data cleaning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410264994.8A CN104036001B (en) | 2014-06-13 | 2014-06-13 | Dynamic hotlist priority scheduling based quick data cleaning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104036001A CN104036001A (en) | 2014-09-10 |
| CN104036001B true CN104036001B (en) | 2017-05-10 |
Family
ID=51466771
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410264994.8A Active CN104036001B (en) | 2014-06-13 | 2014-06-13 | Dynamic hotlist priority scheduling based quick data cleaning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104036001B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572867A (en) * | 2014-12-18 | 2015-04-29 | 上海新炬网络信息技术有限公司 | Data processing method based on Oracle |
| CN107301186B (en) * | 2016-04-15 | 2020-10-09 | 中国移动通信集团重庆有限公司 | Invalid data identification method and device |
| CN106020730A (en) * | 2016-05-25 | 2016-10-12 | 青岛海信移动通信技术股份有限公司 | Clearing method and device for multi-media data of mobile device |
| CN106445738B (en) * | 2016-09-13 | 2020-10-16 | 苏州浪潮智能科技有限公司 | Database backup method and device |
| CN107247758B (en) * | 2017-05-31 | 2020-10-27 | 深圳市长亮科技股份有限公司 | Data processing method based on bank core system |
| CN108319694B (en) * | 2018-02-02 | 2023-01-20 | 中国银行股份有限公司 | Automatic historical data cleaning method and device |
| US20190370059A1 (en) * | 2018-05-30 | 2019-12-05 | Advanced Micro Devices, Inc. | Multi-kernel wavefront scheduler |
| CN108829782B (en) * | 2018-05-31 | 2023-08-25 | 平安科技（深圳）有限公司 | Data table cleaning method, server and computer readable storage medium |
| CN108959400B (en) * | 2018-06-05 | 2021-10-01 | 中国银行股份有限公司 | Bank system historical data cleaning method and device |
| CN109344158A (en) * | 2018-10-10 | 2019-02-15 | 武汉思普崚技术有限公司 | The method and apparatus of redundant data is deleted from mass data |
| CN109710596B (en) * | 2018-11-30 | 2023-12-19 | 平安科技（深圳）有限公司 | Data cleaning method, device, equipment and computer readable storage medium |
| CN110457275A (en) * | 2019-08-15 | 2019-11-15 | 中国银行股份有限公司 | The method for cleaning and device of disk history file |
| CN111538720B (en) * | 2020-03-12 | 2023-07-21 | 嘉陵江亭子口水利水电开发有限公司 | Method and system for cleaning basic data of power industry |
| CN112307007B (en) * | 2020-12-02 | 2023-10-31 | 中国联合网络通信集团有限公司 | Database cleaning method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6965888B1 (en) * | 1999-09-21 | 2005-11-15 | International Business Machines Corporation | Method, system, program, and data structure for cleaning a database table using a look-up table |
| CN102567428A (en) * | 2010-12-30 | 2012-07-11 | 中国移动通信集团浙江有限公司 | Method and device for managing life cycle of online data |
| CN102999528A (en) * | 2011-09-16 | 2013-03-27 | 阿里巴巴集团控股有限公司 | Method and device for ETL (Extract Transform and Load) task off-lining and data cleaning in data warehouse |
| CN103390066A (en) * | 2013-08-08 | 2013-11-13 | 上海新炬网络技术有限公司 | Database overall automation optimizing early warning device and processing method thereof |
| CN103793530A (en) * | 2014-02-26 | 2014-05-14 | 北京京东尚科信息技术有限公司 | Method, device and system for cleaning up business data regularly |
| CN103838762A (en) * | 2012-11-26 | 2014-06-04 | 中国银联股份有限公司 | Uninterrupted data processing device and method based on data table rotation |
-
2014
- 2014-06-13 CN CN201410264994.8A patent/CN104036001B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6965888B1 (en) * | 1999-09-21 | 2005-11-15 | International Business Machines Corporation | Method, system, program, and data structure for cleaning a database table using a look-up table |
| CN102567428A (en) * | 2010-12-30 | 2012-07-11 | 中国移动通信集团浙江有限公司 | Method and device for managing life cycle of online data |
| CN102999528A (en) * | 2011-09-16 | 2013-03-27 | 阿里巴巴集团控股有限公司 | Method and device for ETL (Extract Transform and Load) task off-lining and data cleaning in data warehouse |
| CN103838762A (en) * | 2012-11-26 | 2014-06-04 | 中国银联股份有限公司 | Uninterrupted data processing device and method based on data table rotation |
| CN103390066A (en) * | 2013-08-08 | 2013-11-13 | 上海新炬网络技术有限公司 | Database overall automation optimizing early warning device and processing method thereof |
| CN103793530A (en) * | 2014-02-26 | 2014-05-14 | 北京京东尚科信息技术有限公司 | Method, device and system for cleaning up business data regularly |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104036001A (en) | 2014-09-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104036001B (en) | Dynamic hotlist priority scheduling based quick data cleaning method | |
| CN106599043A (en) | Middleware used for multilevel database and multilevel database system | |
| US20120150797A1 (en) | Method and system for safely transporting legacy data to an object semantic form data grid | |
| CN101937474A (en) | Mass data query method and device | |
| CN109344153A (en) | The processing method and terminal device of business datum | |
| CN105069134A (en) | Method for automatically collecting Oracle statistical information | |
| CN101141754B (en) | Value-added service analysis system and method thereof | |
| CN106998274A (en) | Application message method for pushing and device | |
| CN106021422B (en) | A kind of method and system forming Hive data warehouse based on relevant database | |
| CN108763234A (en) | A kind of real time data synchronization method and system | |
| CN107463610B (en) | Data warehousing method and device | |
| CN103607418B (en) | Large-scale data segmenting system based on cloud service data characteristics and dividing method | |
| CN103327124A (en) | File synchronization method, file synchronization method device, client, server side and equipment | |
| CN110083600A (en) | A kind of method, apparatus, calculating equipment and the storage medium of log collection processing | |
| CN115374102A (en) | Data processing method and system | |
| CN110008246A (en) | Metadata management method and device | |
| CN109474499A (en) | A kind of solution of distributed block chain monitoring system | |
| CN109828966A (en) | Gradual heavy hash method, device, computer equipment and storage medium | |
| CN110018993A (en) | A kind of data analysis system, method and monitoring analysis system | |
| CN101692641B (en) | High effective HLR/AUC data daily auditing method | |
| CN107609172A (en) | A kind of cross-system multi-dimensional data search processing method and device | |
| CN106844497A (en) | The check device and method of a kind of database code | |
| CN108287889B (en) | A kind of multi-source heterogeneous date storage method and system based on elastic table model | |
| CN104462548B (en) | A kind of database distributed system and its implementation | |
| CN111352982A (en) | Manpower extraction analysis system based on big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: SHANGHAI NEW CENTURY NETWORK INFORMATION TECHNOLOG Free format text: FORMER OWNER: SHANGHAI NEW CENTURY NETWORK CO., LTD. Effective date: 20150123 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20150123 Address after: 200063 Shanghai, Zhongshan North Road, No. 2000, building, building No. 3, B Applicant after: SHANGHAI XINJU NETWORK INFORMATION TECHNOLOGY CO., LTD. Address before: 200063 Shanghai, Zhongshan North Road, No. 2000, building, building No. 3, B Applicant before: Shanghai New Century Network Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: Qingpu 201707 waiqingsong road Shanghai City, No. 588 Lane 7548 Building 1 R zone 1 room 113 Patentee after: Shanghai new torch network information technology Limited by Share Ltd Address before: 200063 Shanghai, Zhongshan North Road, No. 2000, building, building No. 3, B Patentee before: SHANGHAI XINJU NETWORK INFORMATION TECHNOLOGY CO., LTD. |