CN108038168A - Data outer storage method - Google Patents
Data outer storage method Download PDFInfo
- Publication number
- CN108038168A CN108038168A CN201711281753.4A CN201711281753A CN108038168A CN 108038168 A CN108038168 A CN 108038168A CN 201711281753 A CN201711281753 A CN 201711281753A CN 108038168 A CN108038168 A CN 108038168A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- storage system
- external storage
- management module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
Abstract
The invention discloses data outer storage method, it comprises the following steps：External storage system is managed, monitor, configures authority；For data to be stored, external storage system is selected by storing operation module；The operating right of external storage system selected by verification；Stored after data to be stored is converted to agreement form, and record fileinfo；Create the index file of the data to be stored for agreement form；Based on index file, search the storage location of corresponding data and be downloaded or delete operation.The present invention can realize the diversification of data external storage, and enterprise can flexibly select according to their needs, reduce the use cost and maintenance cost of business data external storage.
Description
Technical field
The present invention relates to technical field of data storage, more particularly to data outer storage method and system.
Background technology
The rapid proliferation of the application such as big data epoch, mobile interchange, social networks, data analysis, cloud service, in data
The heart proposes revolutionary demand, and storage architecture has become one of IT cores.Government, army's military project, scientific research institutions, aviation
The emerging applications of every field such as space flight, large scale business are chain, medical, financial, new media, broadcasting and TV emerge in an endless stream.The value of data
Increasingly prominent, data have become indispensable assets.As data medium and driving force, storage system becomes big data
The most key core in architecture.
Traditional data center far can not meet either in performance, efficiency, or in investment return, safety
The demand of emerging application, data center traffic are badly in need of new big data processing center to support.Except traditional highly reliable, high superfluous
Outside remaining, green energy conservation, new large data center also needs to possess virtualization, modularization, resilient expansion, automation etc. a series of
Feature, could meet possess the application demand of big data feature.These unprecedented demands, allow the framework and work(of storage system
Can all there occurs unprecedented change.
Based on big data application demand, " application definition storage " concept is suggested.Storage system is as data center's most core
The data basis of the heart, is no longer only traditional scattered, single underlying device.Except high-performance to be possessed, high safety, highly reliable
Outside feature, also there are virtualization, parallel distributed, AUTOMATIC ZONING, resilient expansion, heterogeneous resource integration, global buffer to accelerate
Etc. many features, it could meet the service application demand for possessing big data feature.Big data to the capacity of storage device, read
Write performance, reliability, autgmentability etc. made higher requirement, it is necessary to take into full account functional integration, Information Security, number
According to stability, the system expandability, performance and cost each side factor.
Storage architecture currently on the market mainly includes following several：
(1) storage system based on embedded architecture
Embedded system is a major concept, (such as embedded for there is the embedded system of operating system and file system
Linux system) for, its data storage method (file) and the computer system difference of standard are little, it is except supporting one
A little special file system (such as yaffs, jffs) outside, also support the file system that many standard computer systems realize (such as
Ext2, ext3, fat etc.) and Network File System.In addition many embedded systems also support the equipment such as USB flash disk, SD card, completely may be used
To carry out exchange files with standard file system.The reliability of the benefit including higher of embedded storage switching technology, preferably
Performance and the ability that hard disk is added under conditions of performance is not reduced.
(2) storage system based on X86-based
Storage system based on X86-based, such as HDFS distributed file management systems, its bottom is by substantial amounts of X86 systems
Equipment of uniting composition, while failure tolerant ability and data High Availabitity is ensured, can substantially reduce business data carrying cost.
(3) storage system based on cloud
Cloud storage be in the conceptive extension of cloud computing (cloud computing) and a new concept developed,
It is a kind of emerging Network storage technology, refers to by functions such as cluster application, network technology or distributed file systems, by net
A large amount of various types of storage devices gather collaborative work by application software in network, and the common data that externally provide are deposited
Storage and the system of Operational Visit function.
At present, when enterprise or public institution need data carrying out external storage, single external storage frame can only be selected
Structure (such as cloud storage system), and cannot go flexibly to select using its own external storage resources possessed, undoubtedly also increase enterprise
The data external carrying cost of public institution.
The content of the invention
It is an object of the invention to provide data outer storage method, it can realize the diversification of data external storage, enterprise
Industry can be selected flexibly according to their needs, reduce the use cost and maintenance cost of business data external storage.
To achieve the above object, the present invention uses following technical scheme：
Data outer storage method, this method are based on the outer storage system of data and realize, the outer storage system of the data is including being
System management module, authority management module and storage operation module, the system management module are used to manage external storage system
Basic information and the operating status for monitoring external storage system, the authority management module are used to define each external storage system
The operating right of system, the storage operation module are used to carry out corresponding data manipulation for external storage system；
This method comprises the following steps：
The basic information of each external storage system is collected, managed to S1, system management module, and to each external storage system
The operating status of system is monitored, and authority management module sets the behaviour to each external storage system according to different user roles
Make authority；
S2, for data to be stored, select external storage system by storing operation module, apply for memory space；
Whether S3, authority management module verification active user role possess the corresponding external storage of the application memory space
The operating right of system, if so then execute step S4, if otherwise return to step S2；
S4, storage operation module are stored to the memory space applied after data to be stored is converted to agreement form
In, and fileinfo is recorded, the agreement form is reversible AES encryption form, and the fileinfo is believed including storage location
Breath and data byte number；
S5, collect the fileinfo, creates the index file of the data to be stored for agreement form；
S6, based on index file, search the storage location of corresponding data by storing operation module, and be downloaded or delete
Division operation.
Preferably, the external storage system include Embedded Memory System, X86 storage systems, cloud storage system,
Ftp server, cloud platform.
Preferably, the positional information in step S4 includes data storage start position information, final position information.
Preferably, the step S4 is further included, and is established for storage to the data in memory space and is created data directory；Institute
State step S6 to further include, based on the index file and data directory, the storage of data, the data of the lookup comprising corresponding information
Position and the storage location of the information, and be read.
After adopting the above technical scheme, the present invention has the following advantages that compared with background technology：
The present invention can realize the diversification of data external storage, and enterprise can flexibly select, reduce according to their needs
The use cost and maintenance cost of business data external storage.
Brief description of the drawings
Fig. 1 is the flow diagram of the present invention.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, it is right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
Embodiment
Referring to Fig. 1, the invention discloses data outer storage method, it is based on the outer storage system of data and realizes, outside data
Storage system includes system management module, authority management module and storage operation module, and system management module is used to manage outside
The basic information of storage system and the operating status for monitoring external storage system, authority management module are used to define each outside
The operating right of storage system, storage operation module are used to carry out corresponding data manipulation for external storage system.In this reality
Apply in example, external storage system includes Embedded Memory System, X86 storage systems, cloud storage system, ftp server, Yun Ping
Platform, certainly, external storage system can also include other kinds of storage architecture, and the present invention is not specifically limited.
The data outer storage method of the present invention comprises the following steps：
The basic information of each external storage system is collected, managed to S1, system management module, and to each external storage system
The operating status of system is monitored, and the operating status of external storage system includes network connection information and residual memory space is believed
Breath；Authority management module sets the operating right to each external storage system according to different user roles.
S2, for data to be stored, select external storage system by storing operation module, apply for memory space.
Whether S3, authority management module verification active user role possess the corresponding external storage system of application memory space
Operating right, if so then execute step S4, if otherwise return to step S2.
S4, storage operation module are stored to the memory space applied after data to be stored is converted to agreement form
In, and record fileinfo.Agreement form is reversible AES encryption form, and the encryption to data to be stored so can be achieved, prevent
Only third party directly knows data content.Fileinfo includes storage location information and data byte number, and positional information includes number
According to storage start position information, final position information.
S5, collect fileinfo, creates the index file of the data to be stored for agreement form, index file is with behavior
Unit.
S6, based on index file, search the storage location of corresponding data by storing operation module, and be downloaded or delete
Division operation.It is, of course, also possible to other data manipulations are carried out, such as data transfer, at this time using data to be transferred as number to be stored
According to repeating step S4, S5.
To realize directly reading after data external storage, step S4 is further included, for storage to the number in memory space
Data directory is created according to establishing；Step S6 is further included, and based on index file and data directory, searches the number for including corresponding information
According to, the storage location of the data and the storage location of the information, and it is read.
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto,
Any one skilled in the art the invention discloses technical scope in, the change or replacement that can readily occur in,
It should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with scope of the claims
Subject to.
Claims (4)
1. data outer storage method, it is characterised in that this method is based on the outer storage system of data and realizes, the data store outside is
System includes system management module, authority management module and storage operation module, the system management module and is deposited for managing outside
The basic information of storage system and the operating status for monitoring external storage system, the authority management module are used to define each outer
The operating right of portion's storage system, the storage operation module are used to carry out corresponding data manipulation for external storage system；
This method comprises the following steps：
The basic information of each external storage system is collected, managed to S1, system management module, and to each external storage system
Operating status is monitored, and authority management module sets the operating rights to each external storage system according to different user roles
Limit；
S2, for data to be stored, select external storage system by storing operation module, apply for memory space；
Whether S3, authority management module verification active user role possess the corresponding external storage system of the application memory space
Operating right, if so then execute step S4, if otherwise return to step S2；
S4, storage operation module are stored into the memory space applied after data to be stored is converted to agreement form, and
Fileinfo is recorded, the agreement form is reversible AES encryption form, and the fileinfo includes storage location information sum number
According to byte number；
S5, collect the fileinfo, creates the index file of the data to be stored for agreement form；
S6, based on index file, search the storage location of corresponding data by storing operation module, and be downloaded or delete behaviour
Make.
2. data outer storage method as claimed in claim 1, it is characterised in that：The external storage system includes embedded
Storage system, X86 storage systems, cloud storage system, ftp server, cloud platform.
3. data outer storage method as claimed in claim 1 or 2, it is characterised in that：The positional information bag in step S4
Include data storage start position information, final position information.
4. data outer storage method as claimed in claim 1, it is characterised in that：The step S4 is further included, and is arrived for storage
Data in memory space, which are established, creates data directory；The step S6 is further included, based on the index file and data directory,
The storage location of the data comprising corresponding information, the storage location of the data and the information is searched, and is read.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711281753.4A CN108038168A (en) | 2017-12-07 | 2017-12-07 | Data outer storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711281753.4A CN108038168A (en) | 2017-12-07 | 2017-12-07 | Data outer storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108038168A true CN108038168A (en) | 2018-05-15 |
Family
ID=62095974
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711281753.4A Pending CN108038168A (en) | 2017-12-07 | 2017-12-07 | Data outer storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108038168A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109522272A (en) * | 2018-11-14 | 2019-03-26 | 泰康保险集团股份有限公司 | Storage method, device, system, medium and the electronic equipment of recording file |
| CN110474959A (en) * | 2019-07-12 | 2019-11-19 | 平安普惠企业管理有限公司 | Data interactive method, device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101021768A (en) * | 2006-02-13 | 2007-08-22 | 株式会社日立制作所 | Virtual storage system and control method thereof |
| US7693884B2 (en) * | 2006-01-02 | 2010-04-06 | International Business Machines Corporation | Managing storage systems based on policy-specific proability |
| CN104008207A (en) * | 2014-06-18 | 2014-08-27 | 广东绿源巢信息科技有限公司 | Optical disc based external data storage system for database and data storage method |
| CN104021451A (en) * | 2014-06-20 | 2014-09-03 | 江苏易合大成网络科技有限公司 | Mixed enterprise content management method and system based on cloud storage and local storage |
| CN104486110A (en) * | 2014-12-09 | 2015-04-01 | 上海爱数软件有限公司 | Unified storage resource management method based on RPC (remote procedure call) protocols |
-
2017
- 2017-12-07 CN CN201711281753.4A patent/CN108038168A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7693884B2 (en) * | 2006-01-02 | 2010-04-06 | International Business Machines Corporation | Managing storage systems based on policy-specific proability |
| CN101021768A (en) * | 2006-02-13 | 2007-08-22 | 株式会社日立制作所 | Virtual storage system and control method thereof |
| CN104008207A (en) * | 2014-06-18 | 2014-08-27 | 广东绿源巢信息科技有限公司 | Optical disc based external data storage system for database and data storage method |
| CN104021451A (en) * | 2014-06-20 | 2014-09-03 | 江苏易合大成网络科技有限公司 | Mixed enterprise content management method and system based on cloud storage and local storage |
| CN104486110A (en) * | 2014-12-09 | 2015-04-01 | 上海爱数软件有限公司 | Unified storage resource management method based on RPC (remote procedure call) protocols |
Non-Patent Citations (2)
| Title |
|---|
| 李欣荣: "《现代图书馆 数字内容管理与创新技术》", 30 April 2010 * |
| 陈懿: "《网络管理员考试辅导》", 28 February 2005 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109522272A (en) * | 2018-11-14 | 2019-03-26 | 泰康保险集团股份有限公司 | Storage method, device, system, medium and the electronic equipment of recording file |
| CN110474959A (en) * | 2019-07-12 | 2019-11-19 | 平安普惠企业管理有限公司 | Data interactive method, device, computer equipment and storage medium |
| CN110474959B (en) * | 2019-07-12 | 2023-02-03 | 平安普惠企业管理有限公司 | Data interaction method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11954002B1 (en) | Automatically provisioning mediation services for a storage system | |
| US11061714B2 (en) | System for assignment of proxies for virtual-machine secondary copy operations | |
| US20210397359A1 (en) | Storing Data For Machine Learning And Artificial Intelligence Applications In A Decentralized Storage Network | |
| US11451391B1 (en) | Encryption key management in a storage system | |
| US10795905B2 (en) | Data stream ingestion and persistence techniques | |
| US10691716B2 (en) | Dynamic partitioning techniques for data streams | |
| US10148780B2 (en) | Assignment of data agent proxies for executing virtual-machine secondary copy operations including streaming backup jobs | |
| US20220035714A1 (en) | Managing Disaster Recovery To Cloud Computing Environment | |
| EP3350728B1 (en) | Data storage management operations in a secondary storage subsystem using image recognition and image-based criteria | |
| US9237195B2 (en) | Virtual storage appliance gateway | |
| US20210019063A1 (en) | Utilizing data views to optimize secure data access in a storage system | |
| US10635644B2 (en) | Partition-based data stream processing framework | |
| DE112019005770T5 (en) | Storage management for a cloud-based storage system | |
| US20220215111A1 (en) | Data Protection For Container Storage | |
| US20220083370A1 (en) | Migrating Workloads To A Preferred Environment | |
| CN102521072B (en) | Virtual tape library equipment and data recovery method | |
| US11296944B2 (en) | Updating path selection as paths between a computing device and a storage system change | |
| US20200174671A1 (en) | Bucket views | |
| US20210133167A1 (en) | Efficient implementation of multiple deduplication databases in a heterogeneous data storage system | |
| CN103959264A (en) | Managing redundant immutable files using deduplication in storage clouds | |
| US10521151B1 (en) | Determining effective space utilization in a storage system | |
| CN101449559A (en) | Distributed storage | |
| CN105339903A (en) | Restoring a file system object | |
| US20210055885A1 (en) | Enhanced data access using composite data views | |
| US11816356B2 (en) | Container orchestrator-aware storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180515 |
|
| RJ01 | Rejection of invention patent application after publication |