CN109189609A - A kind of unstructured data quick backup system and method - Google Patents
A kind of unstructured data quick backup system and method Download PDFInfo
- Publication number
- CN109189609A CN109189609A CN201810932671.XA CN201810932671A CN109189609A CN 109189609 A CN109189609 A CN 109189609A CN 201810932671 A CN201810932671 A CN 201810932671A CN 109189609 A CN109189609 A CN 109189609A
- Authority
- CN
- China
- Prior art keywords
- unstructured data
- backup
- quick
- master
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
Abstract
The invention discloses a kind of unstructured data quick backup system and method, system includes: data source server, and storage has unstructured data；Client modules are used to receive quick backup message, generate position bibliography information for reading and sending unstructured data and for recording according to the position of unstructured data；Master backup module is used to generate and send quick backup message, for sending and receiving position bibliography information and for sending the unstructured data under the entire catalogue received according to position bibliography information；Database is used to receive and store the position bibliography information that master backup module sends over；Equipment is stored, is used to receive and store the unstructured data that master backup module sends over.Beneficial effects of the present invention: substantially increasing the backup efficiency of magnanimity unstructured data, save BACKUP TIME window, simplifies disaster tolerance management of the user to unstructured data.
Description
Technical field
The present invention relates to sound technical field of data backup, it particularly relates to a kind of unstructured data quick backup system
System and method.
Background technique
With further popularizing for computer, internet and Digital Media etc., with office documents, text, picture, XML,
The data structures such as HTML, all kinds of reports, image and audio/visual information are irregular, without the non-structural of predefined data model
Change the information based on data to sharply increase, in face of such huge information ocean, especially unstructured data, how fastly
Speed backup stores these massive information resources and just seems particularly critical.
Data backup is the basis of disaster tolerance.Modern data protection trend for non-structured data backup in particulate
The salvo that degree, frequent quick backup, data classification and filing etc. also lack.
For the problems in the relevant technologies, currently no effective solution has been proposed.
Summary of the invention
For above-mentioned technical problem in the related technology, the present invention propose a kind of unstructured data quick backup system and
Method can promote the efficiency of its backup and filing for magnanimity unstructured data.
To realize the above-mentioned technical purpose, the technical scheme of the present invention is realized as follows:
A kind of unstructured data quick backup system, comprising:
Data source server, storage have unstructured data；
Client modules are used to receive quick backup message, for reading and sending the unstructured data and be used for
It is recorded according to the position of the unstructured data and generates position bibliography information；
Master backup module is used to generate and send the quick backup message, for sending and receiving the position catalogue rope
Fuse breath and for according to the position bibliography information by the unstructured data under the entire catalogue received
It sends；
Database is used to receive and store the position bibliography information that the master backup module sends over；
Equipment is stored, is used to receive and store the unstructured data that the master backup module sends over.
Further, the master backup module includes quick backup module, and the quick backup module is described for generating
Quick backup message.
Further, the database is Postgres database.
Further, the client modules are mounted in the data source server.
Further, between the master backup module and the client modules, the master backup module and the data
Pass through network connection between library and between the master backup module and the storage equipment, the network is LAN or LAN-
Free。
Further, the unstructured data includes content-data and metadata；
The invention also discloses a kind of unstructured data flash backup methods, comprising the following steps:
S1 master backup module generates quick backup information and sends；
S2 client modules receive the quick backup information, read the unstructured data stored in data source server,
Recorded according to the position of the unstructured data generate position bibliography information and by the unstructured data and
The position bibliography information is sent；
Master backup module described in S3 receives the unstructured data and the position bibliography information, according to the position
The unstructured data under entire catalogue is sent to storage equipment by bibliography information, and by the position directory index
Information is sent to database.
Further, the unstructured data includes content-data and metadata.
Further, the database is Postgres database.
Further, between the master backup module and the client modules, the master backup module and the data
Carried out data transmission by network between library and between the master backup module and the storage equipment, the network is
LAN or LAN-Free.
Beneficial effects of the present invention: the backup efficiency of magnanimity unstructured data is substantially increased, BACKUP TIME is saved
Window simplifies disaster tolerance management of the user to unstructured data.
Detailed description of the invention
It in order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, below will be to institute in embodiment
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without creative efforts, can also obtain according to these attached drawings
Obtain other attached drawings.
Fig. 1 is the functional block diagram of the unstructured data quick backup system described according to embodiments of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art's every other embodiment obtained belong to what the present invention protected
Range.
As shown in Figure 1, a kind of unstructured data quick backup system described according to embodiments of the present invention, comprising:
Data source server, storage have unstructured data；
Client modules are used to receive quick backup message, for reading and sending the unstructured data and be used for
It is recorded according to the position of the unstructured data and generates position bibliography information；
Master backup module is used to generate and send the quick backup message, for sending and receiving the position catalogue rope
Fuse breath and for according to the position bibliography information by the unstructured data under the entire catalogue received
It sends；
Database is used to receive and store the position bibliography information that the master backup module sends over；
Equipment is stored, is used to receive and store the unstructured data that the master backup module sends over.
In one particular embodiment of the present invention, the master backup module includes quick backup module, described quickly standby
Part module is for generating the quick backup message.
In one particular embodiment of the present invention, the database is Postgres database.
In one particular embodiment of the present invention, the client modules are mounted in the data source server.
In one particular embodiment of the present invention, between the master backup module and the client modules, the master
Pass through network connection, institute between backup module and the database and between the master backup module and the storage equipment
Stating network is LAN or LAN-Free.
In one particular embodiment of the present invention, the unstructured data includes content-data and metadata；
The invention also discloses a kind of unstructured data flash backup methods, comprising the following steps:
S1 master backup module generates quick backup information and sends；
S2 client modules receive the quick backup information, read the unstructured data stored in data source server,
Recorded according to the position of the unstructured data generate position bibliography information and by the unstructured data and
The position bibliography information is sent；
Master backup module described in S3 receives the unstructured data and the position bibliography information, according to the position
The unstructured data under entire catalogue is sent to storage equipment by bibliography information, and by the position directory index
Information is sent to database.
In one particular embodiment of the present invention, the unstructured data includes content-data and metadata.
In one particular embodiment of the present invention, the database is Postgres database.
In one particular embodiment of the present invention, between the master backup module and the client modules, the master
It is counted by network between backup module and the database and between the master backup module and the storage equipment
According to transmission, the network is LAN or LAN-Free.
In order to facilitate above-mentioned technical proposal of the invention is understood, below by way of specifically used mode to above-mentioned skill of the invention
Art scheme is described in detail.
The present invention relates to two main functional modules, first is that master backup module, is the core of entire quick backup system,
It is responsible for message communicating, the read-write quick backup information of entire quick backup system, the operation of control backup and recovery.Second is that client
End module (backup agent), is mounted in data source server, and the unstructured data for being responsible for back up is sent to master backup
Module and the data for receiving recovery.Wherein quick backup module is embedded in master backup module, and Postgres database is for remembering
Record the position bibliography information of unstructured data.Backup mode mainly uses LAN, LAN-Free mixing backup mode.
Storage has unstructured data in data source server, and client modules are pre-installed in data source server.
Content-data and metadata form unstructured data, are placed in data source server storage equipment.
Client modules are responsible for reading the content-data and metadata of unstructured data.
Client modules can will in data source server want backup unstructured data etc. contents by network hair
Message queue is sent to give master backup module.
Quick backup function is realized by opening quick backup module in master backup module.It is standby to improve unstructured data
Part efficiency.
Master backup module gets the message queue of client modules transmission, the unstructured data that client is got
Lan network or fiber optic network (LAN-Free) mode is called to transmit data.
Master backup module can guarantee that message queue transmission is normal, obtain client modules with client modules real time communication
Unstructured data in the data source server at place.
The unstructured data that master backup module will acquire is sent to storage equipment.
When specifically used, the quick backup module embedded in master backup module is opened to realize quick backup function.At this time
After client modules receive the quick backup message of master backup module sending, client modules are in place to unstructured data institute
It sets and carries out the position bibliography information that record generates unstructured data, client modules are no longer to non-structured metadata
It is recorded with file index, path, size and the timestamp of content-data etc. to improve whole efficiency.
Position bibliography information is sent to master backup module by client, and Postgres is recorded in master backup module
It is saved in database.Master backup module is according to position bibliography information simultaneously, the entire catalogue that client modules are got
Under all unstructured datas by network be sent to storage equipment in, to complete to back up.
In conclusion substantially increasing the backup of magnanimity unstructured data by means of above-mentioned technical proposal of the invention
Efficiency saves BACKUP TIME window, simplifies disaster tolerance management of the user to unstructured data.
The above is merely preferred embodiments of the present invention, be not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (9)
1. a kind of unstructured data quick backup system characterized by comprising
Data source server, storage have unstructured data；
Client modules are used to receive quick backup message, for reading and sending the unstructured data and be used for
It is recorded according to the position of the unstructured data and generates position bibliography information；
Master backup module is used to generate and send the quick backup message, for sending and receiving the position catalogue rope
Fuse breath and for according to the position bibliography information by the unstructured data under the entire catalogue received
It sends；
Database is used to receive and store the position bibliography information that the master backup module sends over；
Equipment is stored, is used to receive and store the unstructured data that the master backup module sends over.
2. unstructured data quick backup system according to claim 1, which is characterized in that the master backup module packet
Quick backup module is included, the quick backup module is for generating the quick backup message.
3. unstructured data quick backup system according to claim 1, which is characterized in that the database is
Postgres database.
4. unstructured data quick backup system according to claim 1, which is characterized in that the client modules peace
In the data source server.
5. unstructured data quick backup system according to claim 4, which is characterized in that the master backup module with
Between the client modules, between the master backup module and the database and the master backup module and the storage
By network connection between equipment, the network is LAN or LAN-Free.
6. unstructured data quick backup system according to claim 1, which is characterized in that the unstructured data
Including content-data and metadata；
A kind of unstructured data flash backup method, which comprises the following steps:
S1 master backup module generates quick backup information and sends；
S2 client modules receive the quick backup information, read the unstructured data stored in data source server,
Recorded according to the position of the unstructured data generate position bibliography information and by the unstructured data and
The position bibliography information is sent；
Master backup module described in S3 receives the unstructured data and the position bibliography information, according to the position
The unstructured data under entire catalogue is sent to storage equipment by bibliography information, and by the position directory index
Information is sent to database.
7. unstructured data flash backup method according to claim 6, which is characterized in that the unstructured data
Including content-data and metadata.
8. unstructured data flash backup method according to claim 6, which is characterized in that the database is
Postgres database.
9. unstructured data flash backup method according to claim 6, which is characterized in that the master backup module with
Between the client modules, between the master backup module and the database and the master backup module and the storage
Carried out data transmission by network between equipment, the network is LAN or LAN-Free.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810932671.XA CN109189609A (en) | 2018-08-16 | 2018-08-16 | A kind of unstructured data quick backup system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810932671.XA CN109189609A (en) | 2018-08-16 | 2018-08-16 | A kind of unstructured data quick backup system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109189609A true CN109189609A (en) | 2019-01-11 |
Family
ID=64918082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810932671.XA Pending CN109189609A (en) | 2018-08-16 | 2018-08-16 | A kind of unstructured data quick backup system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109189609A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112764975A (en) * | 2020-12-29 | 2021-05-07 | 中国人寿保险股份有限公司上海数据中心 | Disaster recovery system for unstructured data storage protection |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070027937A1 (en) * | 2004-05-21 | 2007-02-01 | Mcgrattan Emma K | Method and apparatus for storage backup |
| US20070043705A1 (en) * | 2005-08-18 | 2007-02-22 | Emc Corporation | Searchable backups |
| CN102722584A (en) * | 2012-06-08 | 2012-10-10 | 北京天地云箱科技有限公司 | Data storage system and method |
| US8433863B1 (en) * | 2008-03-27 | 2013-04-30 | Symantec Operating Corporation | Hybrid method for incremental backup of structured and unstructured files |
| CN104298574A (en) * | 2014-09-16 | 2015-01-21 | 南京斯坦德云科技股份有限公司 | Data high-speed storage processing system |
| CN104331343A (en) * | 2014-11-06 | 2015-02-04 | 深圳市中兴移动通信有限公司 | File backup method and system |
| CN105005515A (en) * | 2015-07-10 | 2015-10-28 | 上海爱数软件有限公司 | LAN-free backup system |
| CN105956491A (en) * | 2016-05-19 | 2016-09-21 | 北京春鸿企业管理咨询有限公司 | Method and system for performing multi-account backup on data of public cloud storage system |
-
2018
- 2018-08-16 CN CN201810932671.XA patent/CN109189609A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070027937A1 (en) * | 2004-05-21 | 2007-02-01 | Mcgrattan Emma K | Method and apparatus for storage backup |
| US20070043705A1 (en) * | 2005-08-18 | 2007-02-22 | Emc Corporation | Searchable backups |
| US8433863B1 (en) * | 2008-03-27 | 2013-04-30 | Symantec Operating Corporation | Hybrid method for incremental backup of structured and unstructured files |
| CN102722584A (en) * | 2012-06-08 | 2012-10-10 | 北京天地云箱科技有限公司 | Data storage system and method |
| CN104298574A (en) * | 2014-09-16 | 2015-01-21 | 南京斯坦德云科技股份有限公司 | Data high-speed storage processing system |
| CN104331343A (en) * | 2014-11-06 | 2015-02-04 | 深圳市中兴移动通信有限公司 | File backup method and system |
| CN105005515A (en) * | 2015-07-10 | 2015-10-28 | 上海爱数软件有限公司 | LAN-free backup system |
| CN105956491A (en) * | 2016-05-19 | 2016-09-21 | 北京春鸿企业管理咨询有限公司 | Method and system for performing multi-account backup on data of public cloud storage system |
Non-Patent Citations (2)
| Title |
|---|
| 张睿: "面向云存储的非结构化数据存储分析", 《电脑知识与技术》 * |
| 徐英辉等: "MongoDB索引的用电信息非结构化数据存储方法", 《电力系统及其自动化学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112764975A (en) * | 2020-12-29 | 2021-05-07 | 中国人寿保险股份有限公司上海数据中心 | Disaster recovery system for unstructured data storage protection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8688992B2 (en) | System and method for generating agreements | |
| US20120173633A1 (en) | Email conversation management support | |
| US9639545B2 (en) | Online binders | |
| CN109313634A (en) | File synchronization for single file is suspended | |
| US9843625B2 (en) | System and method for aggregating and providing data from enterprise systems to mobile devices | |
| TW201106173A (en) | Multimedia system providing database of shared text comment data indexed to video source data and related methods | |
| CN109240997A (en) | A kind of upload store method, system and the client of file | |
| CN106960164A (en) | A kind of method and system of the fixed electronic evidence in scene | |
| KR20180042145A (en) | A method for determining forgeries of media files | |
| Acker et al. | Tweets may be archived: civic engagement, digital preservation and Obama White House social media data | |
| CN105828017B (en) | A kind of cloud storage access system and method towards video conference | |
| US20050165888A1 (en) | Peer-to-peer data binding | |
| EP3497875B1 (en) | A method of generating a secure record of a conversation | |
| CN110175316B (en) | Media number interaction method, system and storage medium based on blockchain | |
| CN109189609A (en) | A kind of unstructured data quick backup system and method | |
| TWI263915B (en) | System and method for logging event of telecommunications devices | |
| CN106330661A (en) | Data centralized processing method | |
| US20140214966A1 (en) | Social Networking With Video Annotation | |
| CN103176866A (en) | File permission back-up method applied in new technology file system (NTFS) | |
| CN106230880B (en) | A kind of storage method and application server of data | |
| CN103092932A (en) | Distributed document transcoding system | |
| CN103778231A (en) | Method and system for managing operation record information, of databank | |
| CN105704236A (en) | Information issuing method and system | |
| CN104834724B (en) | A kind of database synchronization method and device | |
| KR101426482B1 (en) | System of preserving twitter records |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190111 |