CN103853713A - Efficient storage method of mass data - Google Patents
Efficient storage method of mass data Download PDFInfo
- Publication number
- CN103853713A CN103853713A CN201210495436.3A CN201210495436A CN103853713A CN 103853713 A CN103853713 A CN 103853713A CN 201210495436 A CN201210495436 A CN 201210495436A CN 103853713 A CN103853713 A CN 103853713A
- Authority
- CN
- China
- Prior art keywords
- warehouse
- task
- information
- storage
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013500 data storage Methods 0.000 claims abstract description 18
- 238000007726 management method Methods 0.000 claims abstract description 12
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 238000005538 encapsulation Methods 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 abstract 1
- 230000007246 mechanism Effects 0.000 description 3
- 230000003111 delayed effect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides an efficient storage method of mass data. The efficient storage method comprises step 1, configuring a configuration rule of the storage method; step 2, performing initialization; step 3, performing storage task packaging and scheduling management according to the configuration rule through a data storage controller; step 4, performing allocation scheduling on a storage task according to the configuration rule through a storage work terminal manager; step 5, handing over the task to a free storage terminal through the storage work terminal manager; step 6, executing the storage operation through the storage terminal; step 7, informing the data storage controller to scan a storage information storage file through the storage work terminal manager and performing rescheduling on the stored storage information; step 8, storing the information which is not stored in the closing process. Compared with the prior art, the scheduling management is added into the process of storage of the mass data and accordingly the efficient storage method can adapt to different monitored systems and size changes of the different monitored systems and can adapt to specific business characteristics of the different monitored systems and the accuracy and the integrity of the data storage are improved.
Description
Technical field
The present invention relates to IT O&M field, relate in particular to a kind of mass data storage method efficiently.
Background technology
In IT O&M field, data monitoring is as basic work, operation has important effect for guarantee system for its accuracy and treatment effeciency, along with the development of infosystem itself, data acquisition amount constantly increases, not only to monitor numerous equipment simultaneously, and different equipment relates to again many network element indexs, therefore within the unit interval, gather by supervisory system the network element data amount of coming huge, although the scheme about mass data storage in prior art is a lot, but the matching degree in IT O&M field is not high, lack the buffer memory to data task, unloading, the scheduling mechanisms such as sequence adjustment, lack for IT operation and maintenance tools, efficiently, accurately, the data loading scheme of magnanimity.
Summary of the invention
The object of the present invention is to provide the efficient storage method of a kind of mass data, provide a kind of highly effective gathering dispatching method of mass data warehouse-in for the applied characteristic of IT operation and maintenance tools, the efficiency and the accuracy that improve data loading.
Object of the present invention adopts following scheme to realize: the efficient storage method of mass data, comprises the following steps:
1), the configuration rule of configuration storage method;
2), log-on data memory controller carry out initialization;
3), data storage controller detects warehouse-in whether the information that needs warehouse-in in information cache, according to configuration rule, the information of needs warehouse-in is carried out to the encapsulation of warehouse-in task and management and running;
4), warehouse-in operational terminal manager carries out allocation schedule according to configuration rule to warehouse-in task;
5), warehouse-in operational terminal manager taking-up task from warehouse-in task buffer memory, and give the warehouse-in terminal of this free time and carry out;
6), warehouse-in terminal is carried out in-stockroom operation;
7), notification data memory controller is scanned into library information unloading file by warehouse-in operational terminal manager, and the library information that enters of unloading is re-started to scheduling;
8), when in controller closing process, the library information to be entered in unloading warehouse-in information cache, warehouse-in task buffer memory and by the library information to be entered of warehouse-in operational terminal refusal.
The present invention more specifically scheme is: the efficient storage method of mass data, comprises following key step:
1), the configuration rule of configuration storage method, the described configuration to configuration rule comprises setting: the every batch of optimum warehouse-in Information Number, wait for that optimal information counts that maximum duration, maximum can be carried number of tasks, warehouse-in mission failure number of retries, enter library information unloading strategy, entered library information unloading file, error message unloading file;
2), log-on data memory controller, in start-up course, data storage controller is initialization warehouse-in information cache first, then be scanned into library information unloading file and whether have the not information of warehouse-in, if had, these information are re-loaded in warehouse-in information cache, in order to scheduling;
3), after data storage controller starts, receive library information to be entered, controller detects in warehouse-in information cache whether have the information that needs warehouse-in, and whether the quantity that judges information has reached every batch of default optimum warehouse-in Information Number, if reached optimal number, be divided into a batch data, be packaged into a warehouse-in task according to optimal number, submit to warehouse-in operational terminal manager; If do not reach optimal number, wait for according to rule, count in maximum duration at wait optimal information, if data number reaches every batch of optimum warehouse-in Information Number in buffer memory, distribute a warehouse-in task by optimal data number, otherwise wait is encapsulated as total data a warehouse-in task, submits to warehouse-in operational terminal manager after finishing;
4), in the time that warehouse-in operational terminal manager receives warehouse-in task, according to configuration rule, judge whether task quantity in warehouse-in task buffer queue has reached default maximum and can carry number of tasks, if do not had, this task is put in warehouse-in task buffer memory; Can carry number of tasks if exceeded maximum, enter library information unloading strategy according to what set in configuration rule, directly this task is dumped in library information unloading file, and/or the part task in random unloading warehouse-in task buffer memory is to entering in library information unloading file in proportion;
5), warehouse-in operational terminal manager distributes warehouse-in task, first judge in warehouse-in task buffer memory and whether have unappropriated task, if had, judge whether the warehouse-in terminal under this warehouse-in operational terminal manager is all in operation, if available free warehouse-in terminal, from warehouse-in task buffer memory, take out a task, and give this free time warehouse-in terminal and carry out; If there is no idle warehouse-in terminal, wait for, until have warehouse-in terminal to be released, reallocation warehouse-in task;
6), when idle warehouse-in terminal receives after warehouse-in task, carry out immediately in-stockroom operation, if in the process of implementation, owing to putting abnormal that information errors causes in storage, warehouse-in terminal can be filtered this abnormal information, and continue to carry out unenforced enter library information the library information that enters of mistake is dumped in error message unloading file; If what cause because of reasons such as network, data base management system (DBMS), disk I/O is abnormal, put terminal in storage according to default configuration rule retry some or the in-stockroom operation of time, if still can not normal storage, the executed all operations of rollback, and the library information that enters in this warehouse-in task is dumped to into library information unloading file;
7), when the warehouse-in terminal of warehouse-in operational terminal manager administration is during in low load condition, notification data memory controller is scanned into library information unloading file by warehouse-in operational terminal manager, look into the library information that enters that sees if there is unloading, if had, these information are re-loaded in warehouse-in information cache, re-start scheduling;
8), when in warehouse-in operational terminal manager closing process, warehouse-in operational terminal manager stops submitting task to warehouse-in operational terminal manager, put the library information that enters in information cache in storage, due to concurrent reason, the task of being put in storage operational terminal manager refusal all dumps in library information unloading file; Warehouse-in operational terminal manager stops receiving new task, stops, to warehouse-in terminal distribution task, all unallocated task in task buffer memory being dumped in library information unloading file; Each warehouse-in terminal stops receiving an assignment, but continues to carry out uncompleted task, pending complete, the state of deactivating.
Adopt in this way, compared with prior art, mass data warehouse-in process has been increased to management and running, can significantly improve the extensibility of system, change to adapt to different monitored systems and scale thereof; Can realize the flexible configuration of putaway rule, to adapt to the concrete business characteristic of different monitored systems, easily adjust stock management according to the actual conditions of monitored system; And improved data loading accuracy and integrality, improved the reliability of data loading work, avoided the delayed impact of overload to system by unloading mechanism; Also improve the service efficiency of system, can be better and the concurrent work such as supervisory system, flow system, diagnostic system.
Adopt the efficient collection scheduling device of one of the efficient storage method of mass data of the present invention to be: a kind of collection scheduling device efficiently, includes data storage controller, warehouse-in information cache, warehouse-in operational terminal manager, warehouse-in task buffer memory, puts terminal in storage, enters library information unloading file, configuration rule, error message unloading file.
Data storage controller: be responsible for receiving, distributing library information to be entered, in start-up course, scan and be loaded into the library information that enters of unloading in library information unloading file; After data storage controller starts, according to the data number in warehouse-in information cache, according to configuration rule, the library information that enters in warehouse-in information cache is transformed and is packaged into warehouse-in task with optimum way, and submit to warehouse-in operational terminal manager; In system closing process, data storage controller by warehouse-in information cache, also can comprise the warehouse-in task that is rejected submission and dump in library information unloading file.
Warehouse-in operational terminal manager: after system starts, warehouse-in operational terminal manager receives the warehouse-in task that data storage controller is submitted to, the whether available free terminal of operational terminal under checking subsequently, if do not had, check according to configuration rule whether the task quantity in task buffer memory has reached maximum carrying quantity, in the time having reached the maximum carrying of task quantity, then according to the unloading strategy unloading warehouse-in task in configuration rule; When the affiliated warehouse-in operational terminal of this controller is during in low load condition, this manager should notification data memory controller, is scanned into the library information that enters that whether has the not warehouse-in of unloading in library information unloading file; In this system closing process, this warehouse-in operational terminal manager stops receiving new task and allocating task arrives warehouse-in operational terminal, and in preservation task buffer memory, unappropriated warehouse-in task is to entering library information unloading file.
Warehouse-in terminal: the task that warehouse-in terminal receiving management device distributes, carry out in-stockroom operation, in the process of implementation, if find the wrong library information that enters, by the warehouse-in information filtering of this mistake, and dump to error message unloading file, what then continuation was not finished enters library information, if it is abnormal that storage occurs, such as IO, database service is abnormal etc., all database manipulations of rollback, and the warehouse-in task that this operational terminal is being carried out dumps in library information unloading file.
Configuration rule: the configuration information of stored configuration rule, comprise every batch of optimum warehouse-in Information Number, wait for that optimal information counts that maximum duration, maximum can be carried number of tasks, warehouse-in mission failure number of retries, enter library information unloading strategy, entered library information unloading file, error message unloading file, for system operation provides criterion, make under specific circumstances corresponding reaction for all parts foundation is provided.
Adopt the collection scheduling device of this scheme, can be embodied as mass data stock management process has increased management and running, and extensibility, the customization of system are high, can adapt to different monitored systems and scale thereof and change; Putaway rule flexible configuration, can adapt to the concrete business characteristic of different monitored systems, easily adjusts stock management according to the actual conditions of monitored system; And the reliability of the accuracy of data loading and integrality, data loading work is high, has adopted unloading mechanism to have avoided the delayed impact of overload to system; Strong with the ability of the concurrent work such as supervisory system, flow system, diagnostic system.
Brief description of the drawings
Fig. 1 is principle schematic of the present invention.
Embodiment
Disclosed all features in this instructions, or step in disclosed all methods or process, except mutually exclusive feature and/or step, all can combine by any way.
As shown in Figure 1, with in IT O&M field, preserve the magnanimity performance index data that collected by different acquisition terminal.First configuration rule, define every batch of optimum warehouse-in information content for 1000, wait for optimal information count maximum duration for 3 seconds, maximum can carry 500 of number of tasks, in the time that the task amount in the warehouse-in task buffer memory that warehouse-in operational terminal manager is managed exceeds maximum number of tasks, the task 30% in random unloading task buffer memory to entering in library information unloading file, warehouse-in mission failure number of retries 3 times, to enter library information unloading file be message.dump, error message unloading file error_message.dump.
In system starting process, first data storage controller scans message.dump file, checks the library information that enters wherein whether having by unloading, if had, these information is loaded in the warehouse-in information cache of controller.
After system starts, whether data storage controller will exist into library information in inspection buffer memory, if, do not continue to wait for, if had, according in configuration rule, judge whether this cache information number has reached 1000, if reached, extract 1000 and enter library information, and be a task by these 1000 warehouse-in Information encapsulations, submit to warehouse-in operational terminal manager, if and for reaching 1000 data, wait for 3 seconds, in this waiting process of 3 seconds, if in buffer memory, data number has reached 1000 before 3 seconds, controller is not being waited for, immediately these 1000 data are extracted, be encapsulated as a warehouse-in task, submit to warehouse-in operational terminal manager, on the contrary, after 3 second stand-by period finished, appoint and so do not reach 1000 data demands, controller all extracts the library informations that enters all in buffer memory, be encapsulated as a job invocation warehouse-in operational terminal manager.
Warehouse-in terminal management device receives after task, first whether oepration at full load of the operational terminal under judgement, if do not had, directly puts into this task in task buffer memory, etc. to be allocated, if operational terminal all in working order, according to configuration rule, judge whether the data in task buffer memory have reached 500, if reached 500, at random 30% in 500 tasks are dumped in library information unloading file, then newly submitting to of task is put in task buffer memory, wait to be allocated.
Operational terminal under warehouse-in operational terminal manager is available free is, put operational terminal manager in storage, a task of submitting to the earliest in task buffer memory is distributed to this idle operational terminal and carry out, if there is no idle operational terminal, wait for that resource discharges.
When operational terminal receives warehouse-in task, carry out with that warehouse-in task, when carrying out in warehouse-in task process, find warehouse-in information errors, such as data type is not when abnormal to, the sql error etc. carried out, operational terminal filters out these error messages, and these information are dumped in file error_message.dump, and continue not carry out and do not carry out, preserve not other information of warehouse-in.
Operational terminal in the process of implementation, if run into that IO is abnormal, database service is not activated, Network Abnormal etc. is when abnormal, operational terminal will be according to configuration rule retry 3 subtasks, if appoint so occur identical abnormal, the all operations of rollback, and by this task all enter library information dump in message.dump file.
When the warehouse-in operational terminal under warehouse-in operational terminal manager is during in underrun, now put operational terminal manager notification controller in storage, in scanning message.dump file whether unloading a part enter library information, if had, be re-loaded in the buffer memory of controller, Reseal task, and re-execute these warehouse-in tasks.
When in the method system closing process, controller stops receiving the new library information that enters, and stop submission task to warehouse-in operational terminal manager, preserve warehouse-in in information cache all enter library information, and the warehouse-in task of being refused by warehouse-in operational terminal manager is in file message.dump, warehouse-in operational terminal manager stops receiving new task, stop distributing the task in buffer memory to arrive warehouse-in operational terminal, and all tasks in task buffer memory are dumped in message.dump file, warehouse-in operational terminal continues not being finished of task, occur when abnormal according to original logic unloading file.
Claims (9)
1. the efficient storage method of mass data, is characterized in that the method comprises the following steps:
1), the configuration rule of configuration storage method;
2), log-on data memory controller carry out initialization;
3), data storage controller detects warehouse-in whether the information that needs warehouse-in in information cache, according to configuration rule, the information of needs warehouse-in is carried out to the encapsulation of warehouse-in task and management and running;
4), warehouse-in operational terminal manager carries out allocation schedule according to configuration rule to warehouse-in task;
5), warehouse-in operational terminal manager taking-up task from warehouse-in task buffer memory, and give the warehouse-in terminal of this free time and carry out;
6), warehouse-in terminal is carried out in-stockroom operation;
7), notification data memory controller is scanned into library information unloading file by warehouse-in operational terminal manager, and the library information that enters of unloading is re-started to scheduling;
8), when in controller closing process, the library information to be entered in unloading warehouse-in information cache, warehouse-in task buffer memory and by the library information to be entered of warehouse-in operational terminal refusal.
2. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 1) is specially: the configuration rule of configuration storage method, the described configuration to configuration rule comprises setting: the every batch of optimum warehouse-in Information Number, wait for that optimal information counts that maximum duration, maximum can be carried number of tasks, warehouse-in mission failure number of retries, enter library information unloading strategy, entered library information unloading file, error message unloading file.
3. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 2) be specially: log-on data memory controller, in start-up course, data storage controller is initialization warehouse-in information cache first, then be scanned into library information unloading file and whether have the not information of warehouse-in, if had, these information are re-loaded in warehouse-in information cache, in order to scheduling.
4. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 3) is specially: after data storage controller starts, receive library information to be entered, controller detects in warehouse-in information cache whether have the information that needs warehouse-in, and whether the quantity that judges information has reached every batch of default optimum warehouse-in Information Number, if reached optimal number, be divided into a batch data, be packaged into a warehouse-in task according to optimal number, submit to warehouse-in operational terminal manager; If do not reach optimal number, wait for according to rule, count in maximum duration at wait optimal information, if data number reaches every batch of optimum warehouse-in Information Number in buffer memory, distribute a warehouse-in task by optimal data number, otherwise wait is encapsulated as total data a warehouse-in task, submits to warehouse-in operational terminal manager after finishing.
5. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 4) is specially: in the time that warehouse-in operational terminal manager receives warehouse-in task, according to configuration rule, judge whether task quantity in warehouse-in task buffer queue has reached default maximum and can carry number of tasks, if no, this task is put in warehouse-in task buffer memory; Can carry number of tasks if exceeded maximum, enter library information unloading strategy according to what set in configuration rule, directly this task is dumped in library information unloading file, and/or the part task in random unloading warehouse-in task buffer memory is to entering in library information unloading file in proportion.
6. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 5) is specially: warehouse-in operational terminal manager distributes warehouse-in task, first judge in warehouse-in task buffer memory and whether have unappropriated task, if had, judge whether the warehouse-in terminal under this warehouse-in operational terminal manager is all in operation, if available free warehouse-in terminal is taken out a task from warehouse-in task buffer memory, and give this free time warehouse-in terminal and carry out; If there is no idle warehouse-in terminal, wait for, until have warehouse-in terminal to be released, reallocation warehouse-in task.
7. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 6) is specially: when idle warehouse-in terminal receives after warehouse-in task, carry out immediately in-stockroom operation, if in the process of implementation, owing to putting abnormal that information errors causes in storage, warehouse-in terminal can be filtered this abnormal information, and continue to carry out unenforced enter library information the library information that enters of mistake is dumped in error message unloading file; If what cause because of storage failure reason is abnormal, put terminal in storage according to default configuration rule retry some or the in-stockroom operation of time, if still can not normal storage, the executed all operations of rollback, and the library information that enters in this warehouse-in task is dumped to into library information unloading file.
8. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 7) is specially: when the warehouse-in terminal of warehouse-in operational terminal manager administration is during in low load condition, notification data memory controller is scanned into library information unloading file by warehouse-in operational terminal manager, look into the library information that enters that sees if there is unloading, if had, these information are re-loaded in warehouse-in information cache, re-start scheduling.
9. the efficient storage method of mass data according to claim 1, it is characterized in that: described step 8) is specially: when in warehouse-in operational terminal manager closing process, warehouse-in operational terminal manager stops submitting task to warehouse-in operational terminal manager, put the library information that enters in information cache in storage, due to concurrent reason, the task of being put in storage operational terminal manager refusal all dumps in library information unloading file; Warehouse-in operational terminal manager stops receiving new task, stops, to warehouse-in terminal distribution task, all unallocated task in task buffer memory being dumped in library information unloading file; Each warehouse-in terminal stops receiving an assignment, but continues to carry out uncompleted task, pending complete, the state of deactivating.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210495436.3A CN103853713B (en) | 2012-11-28 | 2012-11-28 | The efficient storage method of mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210495436.3A CN103853713B (en) | 2012-11-28 | 2012-11-28 | The efficient storage method of mass data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103853713A true CN103853713A (en) | 2014-06-11 |
| CN103853713B CN103853713B (en) | 2018-04-24 |
Family
ID=50861381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210495436.3A Expired - Fee Related CN103853713B (en) | 2012-11-28 | 2012-11-28 | The efficient storage method of mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103853713B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354762A (en) * | 2015-11-11 | 2016-02-24 | 国网山东省电力公司电力科学研究院 | Work order identification and distribution system and method of electricity customer service business |
| CN105808778A (en) * | 2016-03-30 | 2016-07-27 | 中国银行股份有限公司 | Method and device for extracting, transforming and loading mass data |
| CN106649418A (en) * | 2015-11-04 | 2017-05-10 | 江苏引跑网络科技有限公司 | High-performance method for importing data into distributed database through direct connection of fragments in driver |
| CN107832901A (en) * | 2017-07-28 | 2018-03-23 | 平安科技（深圳）有限公司 | Method for allocating tasks, device, terminal device and storage medium |
| CN109656937A (en) * | 2018-12-05 | 2019-04-19 | 贵州电网有限责任公司 | A kind of hierarchical cache optimization system for electricity consumption acquisition system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1265141A2 (en) * | 2001-03-29 | 2002-12-11 | Kabushiki Kaisha Toshiba | System, method and computer program for data-management |
| CN101354713A (en) * | 2008-09-08 | 2009-01-28 | 大唐软件技术股份有限公司 | Method and system for storing data |
| CN101533417A (en) * | 2009-04-28 | 2009-09-16 | 阿里巴巴集团控股有限公司 | A method and system for realizing ETL scheduling |
| CN101576918A (en) * | 2009-06-19 | 2009-11-11 | 用友软件股份有限公司 | Data buffering system with load balancing function |
| CN101894163A (en) * | 2010-07-23 | 2010-11-24 | 中兴通讯股份有限公司 | Database operating and scheduling method and device for performance data acquisition system |
| CN101996244A (en) * | 2010-11-09 | 2011-03-30 | 中兴通讯股份有限公司 | Device, system and method for inputting batch data into database |
| CN102012840A (en) * | 2010-12-23 | 2011-04-13 | 中国农业银行股份有限公司 | Batch data scheduling method and system |
| CN102209118A (en) * | 2011-06-10 | 2011-10-05 | 成都勤智数码科技有限公司 | Distributed mass data gathering method |
| CN102752387A (en) * | 2012-06-29 | 2012-10-24 | 用友软件股份有限公司 | Data storage processing system and data storage processing method |
-
2012
- 2012-11-28 CN CN201210495436.3A patent/CN103853713B/en not_active Expired - Fee Related
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1265141A2 (en) * | 2001-03-29 | 2002-12-11 | Kabushiki Kaisha Toshiba | System, method and computer program for data-management |
| CN101354713A (en) * | 2008-09-08 | 2009-01-28 | 大唐软件技术股份有限公司 | Method and system for storing data |
| CN101533417A (en) * | 2009-04-28 | 2009-09-16 | 阿里巴巴集团控股有限公司 | A method and system for realizing ETL scheduling |
| CN101576918A (en) * | 2009-06-19 | 2009-11-11 | 用友软件股份有限公司 | Data buffering system with load balancing function |
| CN101894163A (en) * | 2010-07-23 | 2010-11-24 | 中兴通讯股份有限公司 | Database operating and scheduling method and device for performance data acquisition system |
| CN101996244A (en) * | 2010-11-09 | 2011-03-30 | 中兴通讯股份有限公司 | Device, system and method for inputting batch data into database |
| CN102012840A (en) * | 2010-12-23 | 2011-04-13 | 中国农业银行股份有限公司 | Batch data scheduling method and system |
| CN102209118A (en) * | 2011-06-10 | 2011-10-05 | 成都勤智数码科技有限公司 | Distributed mass data gathering method |
| CN102752387A (en) * | 2012-06-29 | 2012-10-24 | 用友软件股份有限公司 | Data storage processing system and data storage processing method |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649418A (en) * | 2015-11-04 | 2017-05-10 | 江苏引跑网络科技有限公司 | High-performance method for importing data into distributed database through direct connection of fragments in driver |
| CN105354762A (en) * | 2015-11-11 | 2016-02-24 | 国网山东省电力公司电力科学研究院 | Work order identification and distribution system and method of electricity customer service business |
| CN105808778A (en) * | 2016-03-30 | 2016-07-27 | 中国银行股份有限公司 | Method and device for extracting, transforming and loading mass data |
| CN105808778B (en) * | 2016-03-30 | 2019-04-26 | 中国银行股份有限公司 | A kind of mass data extracts, conversion, loading method and device |
| CN107832901A (en) * | 2017-07-28 | 2018-03-23 | 平安科技（深圳）有限公司 | Method for allocating tasks, device, terminal device and storage medium |
| CN109656937A (en) * | 2018-12-05 | 2019-04-19 | 贵州电网有限责任公司 | A kind of hierarchical cache optimization system for electricity consumption acquisition system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103853713B (en) | 2018-04-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102622316B (en) | Optionally enabled hosts transmission is interrupted | |
| CN106790565B (en) | A kind of network attached storage group system | |
| CN103853713A (en) | Efficient storage method of mass data | |
| CN111258737B (en) | Resource scheduling method and device and filter scheduler | |
| US20090276781A1 (en) | System and method for multi-level preemption scheduling in high performance processing | |
| CN111782360A (en) | Distributed task scheduling method and device | |
| CN104468407A (en) | Method and device for performing service platform resource elastic allocation | |
| US20220138012A1 (en) | Computing Resource Scheduling Method, Scheduler, Internet of Things System, and Computer Readable Medium | |
| CN111209110B (en) | Task scheduling management method, system and storage medium for realizing load balancing | |
| CN111258746B (en) | Resource allocation method and service equipment | |
| CN112286671B (en) | Containerization batch processing job scheduling method and device and computer equipment | |
| CN109710416B (en) | Resource scheduling method and device | |
| CN106528853A (en) | Data interaction management device and cross-database data interaction processing device and method | |
| CN107977257A (en) | A kind of method for scheduling task and device | |
| CN101158849A (en) | Method for preventing industrial automation system from snowslip | |
| CN101894163A (en) | Database operating and scheduling method and device for performance data acquisition system | |
| EP2908465A1 (en) | High availability method and system for improving the utility of physical servers in cloud computing resource pool | |
| CN112162839A (en) | Task scheduling method and device, computer equipment and storage medium | |
| KR101912393B1 (en) | Network device and interrupt management method of the network device | |
| CN114610567A (en) | Container monitoring method, network device and storage medium | |
| CN103853719A (en) | Extensible mass data collection system | |
| US20070174836A1 (en) | System for controlling computer and method therefor | |
| CN103617023B (en) | A kind of recovery method of system rubbish resource and device | |
| CN116127494A (en) | Control method and related device for concurrent access of users | |
| CN107832140B (en) | RPC request control method, storage medium, electronic device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Two Lu Tian Hua high tech Zone of Chengdu City, Sichuan province 610000 No. 219 Tianfu Software Park C District 10 building 20 layer Applicant after: CHINAWISERV TECHNOLOGIES Inc. Address before: Two Lu Tian Hua high tech Zone of Chengdu City, Sichuan province 610000 No. 81 Tianfu Software Park C District 10 building 20 layer Applicant before: CHENGDU QINZHI DIGITAL TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20191211 Granted publication date: 20180424 |
|
| PP01 | Preservation of patent right | ||
| PD01 | Discharge of preservation of patent |
Date of cancellation: 20221211 Granted publication date: 20180424 |
|
| PD01 | Discharge of preservation of patent | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180424 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |