CN110019117A - A kind of data archiving method and device - Google Patents
A kind of data archiving method and device Download PDFInfo
- Publication number
- CN110019117A CN110019117A CN201710898917.1A CN201710898917A CN110019117A CN 110019117 A CN110019117 A CN 110019117A CN 201710898917 A CN201710898917 A CN 201710898917A CN 110019117 A CN110019117 A CN 110019117A
- Authority
- CN
- China
- Prior art keywords
- data
- class
- audit
- timestamp
- filing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000012550 audit Methods 0.000 claims abstract description 52
- 238000013500 data storage Methods 0.000 claims abstract description 8
- 238000012216 screening Methods 0.000 claims description 5
- 238000007726 management method Methods 0.000 abstract description 3
- 239000013589 supplement Substances 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000003208 petroleum Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2291—User-Defined Types; Storage management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of data archiving method and devices, this method passes through acquisition data, task identification and timestamp are arranged to the data of acquisition, form filing data, data include graphics class data, date class data, video class data, audio related data and text class data；The filing data are audited according to task identification and timestamp, and using audit condition, the audit condition includes size of data and data format；After data are audited, it is marked to whether data pass through audit, and in the database by the data storage after label.The invention carries out instruction audit using the data that audit condition accesses needs, ensure that the quality of data, has made to be promoted and supplement for data archiving method, has facilitated the management of filing data.
Description
Technical field
The invention belongs to information technology field, in particular to a kind of data archiving method and device.
Background technique
Currently, all trades and professions such as petroleum, finance, communication all store the data of generation by opening relationships database.Together
When, in order to realize certain business demand, certain tasks are completed, need to pre-process data relevant to the mission requirements,
File in database again, when needing acquisition to meet the data of mission requirements, needs according to keywords time such as data name, types
Entire database is gone through, data relevant to task are filtered out from database, the access speed that this will lead to data is very slow.
It, can be by adding the identification informations logarithm such as timestamp or task identification to solve the problems, such as that data access speed is slow
According to being distinguish, when need to access in sometime meet the data of mission requirements when, so that it may according to timestamp and task mark
Know and access the data of storage in the database in this period, and does not have to traverse all databases, so that access speed is promoted,
Save access time.But this method not can guarantee the quality of filing data, and whether the data after not can guarantee filing, which meet, is worked as
First mission requirements only address only the slow problem of data access speed.Therefore, meet mission requirements to ensure to file data,
It needs to carry out data audit before data filing, to ensure the quality of data.
Summary of the invention
The purpose of the present invention is to provide a kind of data archiving method and devices, for solving filing number in the prior art
According to problem of low quality.
To achieve the above object, the technical scheme is that
A kind of data archiving method, includes the following steps:
1) data are acquired, task identification and timestamp are arranged to the data of acquisition, form filing data；
2) according to task identification and timestamp, and the filing data are audited using audit condition, the audit
Condition includes size of data and data format；
3) it after data audit, is marked to whether data pass through audit, and by the data storage after label in number
According in library.
Further, before to the data of acquisition, task identification and timestamp are set, the data need to be purged
Redundant data, invalid data, data classification, data screening.
Further, the data include image class data, date class data, video class data, audio related data and text
Word class data.
Further, the size of image class data, video class data, audio related data and text class data is set.
Further, the timestamp refers to initial time and the end time of task.
The present invention also provides a kind of data filing device, including processor, the processor is for executing following methods
Instruction:
1) data are acquired, task identification and timestamp are arranged to the data of acquisition, form filing data；
2) according to task identification and timestamp, and the filing data are audited using audit condition, the audit
Condition includes size of data and data format；
3) it after data audit, is marked to whether data pass through audit, and by the data storage after label in number
According in library.
Further, before to the data of acquisition, task identification and timestamp are set, the data need to be purged
Redundant data, invalid data, data classification, data screening.
Further, the data include graphics class data, date class data, video class data, audio related data and text
Word class data.
Further, the size of image class data, video class data, audio related data and text class data is set.
Further, the timestamp refers to initial time and the end time of task.
The beneficial effects of the present invention are:
Task identification and timestamp is arranged by acquisition data, to the data of acquisition in this method of the present invention, forms filing number
According to data include graphics class data, date class data, video class data, audio related data and text class data；According to task mark
Knowledge and timestamp, and the filing data are audited using audit condition, the audit condition includes size of data and number
According to format；After data are audited, it is marked to whether data pass through audit, and by the data storage after label in database
In.The invention carries out instruction audit using the data that audit condition accesses needs, ensure that the quality of data, meets task
Demand has been made to be promoted and supplemented, facilitated the management of filing data for data archiving method.
Detailed description of the invention
Fig. 1 is data checking method flow chart；
Fig. 2 is data archiving method flow chart.
Specific embodiment
A specific embodiment of the invention is further described with reference to the accompanying drawing:
A kind of quality of data checking method includes the following steps: as depicted in figs. 1 and 2
1, data are acquired according to mission requirements, data include graphics class data, date class data, video class data, audio
Class data and text class data；Taxonomic revision is carried out to the data of acquisition, data screening, removes the edit operations such as redundancy, is obtained
Data after edit operation.
2, task identification, timestamp are arranged to above-mentioned data, to distinguish different task demand in different time periods
File data, and by the data storage comprising task identification and timestamp to database, obtains pending data record, as
Task identification is arranged in data, to distinguish different task settings；Timestamp is set, at the beginning of timestamp label is data
With the end time of data.By taking primary activity as an example, the corresponding task of activity each time, task identification is for distinguishing difference
Task, timestamp is the start and end time for logger task.If first task presentation is T1, initial time is
On June 1st, 2016, termination time are on June 31st, 2016；Second task presentation is T2, and initial time is July 1 in 2016
Day, the termination time is on July 31st, 2016, and the format of the timestamp in the present embodiment is expressed as xx xx month xx day, as it
His embodiment, also referred to as xxxx.xx.xx..
Building includes the task list on task identification column, field and condition data column, being arranged for current task for task
The timestamp being arranged for current task (period for executing current task) is write on task identification column in mark write-in task list
Enter condition data column, field is used to storing data, as shown in table 1.
The pending data record of table 1
3, the data record of formation is audited according to the timestamp of data and task identification, audits condition by data
Data audit is carried out to data, data audit condition includes data format, size of data, as shown in table 2.
4, the data of acquisition are audited using audit condition shown in table 2, whether full audits inquired data
The original mission requirements of foot, data audit condition specifically: and image class size of data is no more than 3M, format JPEG, BMP,
The general images format such as PNG, GIF；Video class data size is no more than 1G, the generic videos such as format AVI, RM, RMVB, MP4
Format；Audio related data size is no more than 1M, the universal audios format such as format mp3, wma, wav；Text class size of data is not
More than 5M, the general formats such as format docx, doc, txt；Date class data format is xxxx xx month xx day, and size is not wanted
It asks.If original mission requirements are image class data, if the data format of data is JPEG, BMP, PNG, that is, image class number
According to judging whether image class data meet the requirements in conjunction with the requirement of image class size of data, the audit to other class data
It also is such.
2 data of table audit condition
| Set of circumstances | Data format | Size of data | Corresponding data type |
| Condition 1 | JPEG、BMP、PNG、GIF | ≤ 3M | Image class |
| Condition 2 | AVI、RM、RMVB、MP4 | ≤ 1G | Video class |
| Condition 3 | Xxxx xx month xx day | Date class | |
| Condition 4 | mp3、wma、wav | ≤ 1M | Audio class |
| Condition 5 | Doc、docx、TXT | ≤ 5M | Text class |
5, after auditing, obtain auditing result, auditing result includes two kinds: audit, which passes through and audits, not to be passed through, and right
Data after audit are marked, and pass through if data audit passes through labeled as audit, if the audit fails is labeled as data
The audit fails, as shown in table 3.
Data and data audit label after the audit of table 3
6, by the data record after audit obtained above, by relevant database and database management language by data
Backup to database.
7, need to obtain in certain time period meet mission requirements and data relevant to task when, task can be passed through
Mark and timestamp filter out data corresponding with the task from database, avoid traversing entire database, access speed pole
Fastly；And the data inquired are the data after audit, improve the quality of data, meet mission requirements.
The present invention also provides a kind of data filing device, which includes processor, and processor is for executing with lower section
The instruction of method:
1) data are acquired, task identification and timestamp are arranged to the data of acquisition, form filing data；
2) according to task identification and timestamp, and filing data are audited using audit condition, audit condition includes
Size of data and data format；
3) it after data audit, is marked to whether data pass through audit, and by the data storage after label in number
According in library.
The present invention pass through for filing data be arranged task identification, timestamp and audit mark, realize meet mission requirements,
The filing data of timing node and data quality requirement are separated with overall data in database, when needing to inquire or obtain filing
When data, data can be obtained as combination condition from relational database by task identification, timestamp and audit mark, i.e.,
Inquiry velocity is improved, in turn ensures the quality of data.This method is the promotion and supplement to traditional data archiving method.
Specific embodiment is presented above, but the present invention is not limited to embodiment described above.The present invention
Basic ideas be above-mentioned basic scheme, for those of ordinary skill in the art, introduction according to the present invention is designed each
The model of kind deformation, formula, parameter do not need to spend creative work.The case where not departing from the principle and spirit of the invention
Under to embodiment carry out change, modification, replacement and modification still fall in protection scope of the present invention.
Claims (10)
1. a kind of data archiving method, which comprises the steps of:
1) data are acquired, task identification and timestamp are arranged to the data of acquisition, form filing data；
2) according to task identification and timestamp, and the filing data are audited using audit condition, the audit condition
Including size of data and data format；
3) it after data audit, is marked to whether data pass through audit, and by the data storage after label in database
In.
2. data archiving method according to claim 1, which is characterized in that the data of acquisition setting task identification and
Before timestamp, redundant data, invalid data, data classification, data screening need to be purged to the data.
3. data archiving method according to claim 2, which is characterized in that the data include image class data, date
Class data, video class data, audio related data and text class data.
4. data archiving method according to claim 3, which is characterized in that setting image class data, video class data, sound
The size of frequency class data and text class data.
5. data archiving method according to claim 1, which is characterized in that the timestamp refer to task initial time and
End time.
6. a kind of data filing device, which is characterized in that including processor, the processor is used to execute the finger of following methods
It enables:
1) data are acquired, task identification and timestamp are arranged to the data of acquisition, form filing data；
2) according to task identification and timestamp, and the filing data are audited using audit condition, the audit condition
Including size of data and data format；
3) it after data audit, is marked to whether data pass through audit, and by the data storage after label in database
In.
7. data archiving method according to claim 6, which is characterized in that the data of acquisition setting task identification and
Before timestamp, redundant data, invalid data, data classification, data screening need to be purged to the data.
8. data filing device according to claim 7, which is characterized in that the data include graphics class data, date
Class data, video class data, audio related data and text class data.
9. data filing device according to claim 8, which is characterized in that setting image class data, video class data, sound
The size of frequency class data and text class data.
10. data filing device according to claim 6, which is characterized in that the timestamp refers to the initial time of task
And the end time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710898917.1A CN110019117A (en) | 2017-09-28 | 2017-09-28 | A kind of data archiving method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710898917.1A CN110019117A (en) | 2017-09-28 | 2017-09-28 | A kind of data archiving method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110019117A true CN110019117A (en) | 2019-07-16 |
Family
ID=67186465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710898917.1A Pending CN110019117A (en) | 2017-09-28 | 2017-09-28 | A kind of data archiving method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019117A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110837592A (en) * | 2019-11-08 | 2020-02-25 | 京东数字科技控股有限公司 | Method, apparatus and computer readable storage medium for data archiving |
| CN111263173A (en) * | 2020-01-09 | 2020-06-09 | 安徽文香信息技术有限公司 | Data resource adding method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040088313A1 (en) * | 2001-11-02 | 2004-05-06 | Medical Research Consultants | Knowledge management system |
| CN1783063A (en) * | 2004-11-29 | 2006-06-07 | 中兴通讯股份有限公司 | Device and method for archiving and inquiry historical data |
| CN106326467A (en) * | 2016-08-31 | 2017-01-11 | 湖南德立信软件开发有限公司 | Method for data archiving and paper assembling based on nonlinear data |
| CN106599240A (en) * | 2016-12-12 | 2017-04-26 | 长春理工大学 | Multi-source data processing method and system |
-
2017
- 2017-09-28 CN CN201710898917.1A patent/CN110019117A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040088313A1 (en) * | 2001-11-02 | 2004-05-06 | Medical Research Consultants | Knowledge management system |
| CN1783063A (en) * | 2004-11-29 | 2006-06-07 | 中兴通讯股份有限公司 | Device and method for archiving and inquiry historical data |
| CN106326467A (en) * | 2016-08-31 | 2017-01-11 | 湖南德立信软件开发有限公司 | Method for data archiving and paper assembling based on nonlinear data |
| CN106599240A (en) * | 2016-12-12 | 2017-04-26 | 长春理工大学 | Multi-source data processing method and system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110837592A (en) * | 2019-11-08 | 2020-02-25 | 京东数字科技控股有限公司 | Method, apparatus and computer readable storage medium for data archiving |
| CN111263173A (en) * | 2020-01-09 | 2020-06-09 | 安徽文香信息技术有限公司 | Data resource adding method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109074383B (en) | Document search with visualization within the context of a document | |
| CN102270225B (en) | Data change daily record method for supervising and data change daily record supervising device | |
| US20130006996A1 (en) | Clustering E-Mails Using Collaborative Information | |
| CN107729330B (en) | Method and apparatus for acquiring data set | |
| WO2019056789A1 (en) | Method, device, computer device, storage medium for identifying related party transaction | |
| CN105550270B (en) | Data base query method and device | |
| US20130332422A1 (en) | Defining Content Retention Rules Using a Domain-Specific Language | |
| CN107622068A (en) | A kind of blog management method and device based on JSON forms | |
| CN103761341A (en) | Information matching method and device | |
| US10146881B2 (en) | Scalable processing of heterogeneous user-generated content | |
| CN110019117A (en) | A kind of data archiving method and device | |
| CN1588352A (en) | Recording method for extendable mark language file repairing trace | |
| US10210165B2 (en) | File management for eDiscovery | |
| WO2018226255A1 (en) | Functional equivalence of tuples and edges in graph databases | |
| CN103678263A (en) | Graphical interface display method and system for incidence relations among document chapters | |
| WO2018019310A1 (en) | Big data system data backup and recovery methods and devices, and computer storage medium | |
| CN109359202B (en) | Electronic operation and maintenance work order automatic generation method and system based on audio and video recording | |
| CN103077187A (en) | Log management method and log management system | |
| US9286348B2 (en) | Dynamic search system | |
| US20210141996A1 (en) | System and method for note taking and management | |
| US11803356B1 (en) | Software requirements creating and tracking system and method | |
| CN110941952A (en) | Method and device for perfecting audit analysis model | |
| CN110909112B (en) | Data extraction method, device, terminal equipment and medium | |
| CN112632266B (en) | Data writing method and device, computer equipment and readable storage medium | |
| CN107609038A (en) | Data clearing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190716 |
|
| RJ01 | Rejection of invention patent application after publication |