CN110175162A - A kind of isomery storage repository virtualization storage management system based on file operation - Google Patents
A kind of isomery storage repository virtualization storage management system based on file operation Download PDFInfo
- Publication number
- CN110175162A CN110175162A CN201910424054.3A CN201910424054A CN110175162A CN 110175162 A CN110175162 A CN 110175162A CN 201910424054 A CN201910424054 A CN 201910424054A CN 110175162 A CN110175162 A CN 110175162A
- Authority
- CN
- China
- Prior art keywords
- file
- storage
- management
- user
- virtualization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The isomery storage repository virtualization storage management system based on file operation that the invention discloses a kind of, the system uses for reference linux system for software, the management of hardware uniformly becomes the management method of file, device virtualization and file are virtualized, the data management of user and storage space management are virtually turned into file management together, it realizes physically discrete, the data management of fragmentation and storage management are continuous in logic, the transformation of integrated management, a variety of isomery storage repositories are integrated into a virtual storage system, improve the storage efficiency and safety of whole system, and the resource management of whole system becomes more consistent and simple.
Description
Technical field
The present invention relates to the virtual memory management fields of storage repository, more particularly to a kind of isomery based on file operation
Storage repository virtualization storage management system.
Background technique
Mass data be reflect social informatization degree vital signs, and these mass datas be necessarily stored in it is a variety of different
In structure storage repository.Seek the more scientific efficiently management to storage repository itself and the management of its data is that the field is ground
The important directions and important content studied carefully.
The isomery storage repository bottom is physical storage.It can be the cracking solid-state of at high price but access rate
Hard disk；It is also possible to the mechanical hard disk cheap, sequential storage speed is fast but arbitrary access is slow；Can even is that can only
Sequential access but cheaper tape, etc..The second layer is memory block, is made of the identical memory of those physiques
Storage array (for example the same class hard disk produced by same class producer forms).Third layer is storage sites, is by same
Numerous memory blocks under a working environment and management system (distributed, centralization is ok) are constituted.One storage sites can be with
There are many different memory blocks, there is respective security level, access rate and capacity.The set of multiple storage sites is just constituted
One storage repository.
In order to improve the storage utilization efficiency and safety of isomery storage repository entirety, it is empty in data, storage to improve user
Between managerial usage experience, existing way is that will have the management of the memory space of storage repository of heterogeneous file system empty
Quasi-ization.Virtualization is the technology for solving a kind of shared classics of physical resource.In virtualization process, the prior art is to pass through
The interface (such as read and write) that one layer of standard is covered in the equipment of isomery, shields underlying device resource operation upwards
Realize details.If system is considerably complicated, such as the storage system to be faced in practice, in the dilatation Jing Guo different historical
After adjustment, there are also difference in management system is numerous for physical storage device capacity, rate, safety in whole system, it is desirable to
Virtualization is realized on the basis of such diversified physical equipment, they are abstracted as to the logical device of one group of behavioral unity, is used
Hierarchical system successively masks unnecessary difference and realizes that details just becomes one and naturally selects.But in design level
When subsystem, needs characteristic that is apparent and grasping all underlying device resources and be finally provided to the service of user
Characteristic re-defines the function of every layer of hierarchical system and the details that should be masked, could be by the equipment of real work and user
Needs got up with suitable logical connection.It realizes that process is very complicated, and the inheritance of management program is poor, is unfavorable for storing
The management optimization of the memory space in warehouse upgrades.
Summary of the invention
Goal of the invention: the object of the present invention is to provide it is a kind of be able to solve defect existing in the prior art based on file
The isomery storage repository virtualization storage management system of operation.
A kind of technical solution: isomery storage repository virtualization storage management system based on file operation of the present invention
System uses for reference the management method that the management of software, hardware is uniformly become file by linux system, and device virtualization and file is empty
The data management of user and storage space management are virtually turned to file management together, realize physically discrete, fragment by quasi-ization
The data management of change and storage management integrate a variety of isomery storage repositories to continuous, integrated management transformation in logic
At a virtual storage system.The virtual storage system includes:
Storage sites are the set of a variety of physical storages；
Virtualization storage administrative center, for the storage resource of all storage sites to be logically unified into an entirety
It is managed；
Virtualizing file administrative center, for patrolling the file fragmentation of virtualization storage administrative center physical store with single
It collects storage form and carries out storage management, shield the logical access details to data；Data are obtained to user terminal；
User's request management center is virtualized, for judging the legitimacy of all requests that user submits in client, if
It is legal, user is converted to the form of file manipulation command in all requests that client is submitted, consigns to the file pipe of lower layer
Reason center；
Multiple user terminals: for winning the confidence breath in Installed System Memory.
Further, the virtualization storage administrative center includes multiple super memory blocks, and the super memory block is pair
The logic integration of of a sort memory block, attribute indicate by vector, and vector element includes: that access rate, data store
Security level, block capacity；The memory block is the identical storage of security level of access rate in storage sites, data storage
The set of device.
Further, the virtualizing file administrative center will be various for reference linux system file management mode
The storage space management of type is abstracted as file operation；The operation to memory space, the operation to data in memory space, system
One is the operation to file.
Preferably, the virtualizing file administrative center increases one layer of virtual file on the basis of Linux file management
With the mapping mechanism of file fragmentation, data are obtained to user terminal according to mapping mechanism, and controls storage sites and writes data into and deposit
Store up space；The mapping mechanism is the file segmental information that is provided using mapped file as guide, and user is complete in logic
Virtual file accessing operation, be mapped to the mechanism of the accessing operation to each file fragmentation；Wherein, the mapped file is used
In to complete original, logically storage mode is divided into the division information after file fragmentation and is managed；The division letter
Breath includes: old file name, the logical address of file fragmentation, size, the position in original and data volume and IP address.
Further, virtualization user's request management center, for judging that user owns what user terminal was submitted
The legitimacy of request is then converted to the form of file manipulation command if legal, consigns to the file management center of lower layer；And it shows
The controlled state of its data, and the file operation requests for obtaining user are linked up, convert it to totally the receiving of file management center
File operation, thus to the presence of virtualizing file administrative center shielding user, the access for shielding file to user is realized thin
Section.
Further, the user terminal is directly connected to storage sites, completes the biography of the specific data flow of user's operation
It is defeated.
The utility model has the advantages that the present invention virtualizes device virtualization and file, by the data management of user and memory space pipe
Reason virtually turn to file management together, realize physically discrete, fragmentation data management and storage management in logic
Continuous, integrated management transformation, a variety of isomery storage repositories are integrated into a virtual storage system, improve entire system
The storage efficiency and safety of system, and the resource management of whole system becomes more consistent and simple.
Detailed description of the invention
Fig. 1 is structural block diagram of the invention.
Specific embodiment
With reference to the accompanying drawings and detailed description, technical solution of the present invention is further introduced.
A kind of isomery storage repository virtualization storage management system based on file operation, comprising:
K storage sites: the set of K a variety of physical storages；
Virtualization storage administrative center, for the storage resource of all storage sites to be logically unified into an entirety
It is managed；
Virtualizing file administrative center: for patrolling the file fragmentation of virtualization storage administrative center physical store with single
It collects storage form and carries out storage management, shield the logical access details to data；Data are obtained to user terminal；
Virtualization user's request management center: for judging the legitimacy of all requests that user submits in client, if
It is legal, user is converted to the form of file manipulation command in all requests that client is submitted, consigns to the virtualization of lower layer
File management center；
N user terminal: it wins the confidence n terminal of breath in Installed System Memory.
The virtualization storage administrative center includes multiple super memory blocks, and the super memory block is deposited to of a sort
The logic integration of storage area block, attribute are indicated that vector element includes: access rate, the security level of data storage, area by vector
Block capacity；The memory block be storage sites in access rate, data storage the identical memory of security level set.Together
When, provide the file access operations method in these memory blocks.
The virtualizing file administrative center, for reference linux system file management mode, by various types of storages
Space management is abstracted as file operation；The operation to memory space, the operation to data in memory space, it is unified for file
Operation.File management center is exactly a super file system to upper layer performance.Realize the text of physically fragmentation storage
Part shields the logical access details to data with the storage management of unity logic storage form.Its realization is in Linux file pipe
On the basis of reason, the mapping mechanism of one layer of virtual file and file fragmentation is increased, solving linux system, can not to handle outside more
File system integration problem under heterogeneous file system.
The virtualizing file administrative center increases one layer of virtual file and file on the basis of Linux file management
The mapping mechanism of fragment obtains data to user terminal according to mapping mechanism, and controls storage sites and write data into memory space；
The mapping mechanism is the file segmental information that is provided using mapped file as guide, by user to complete virtual text in logic
The accessing operation of part is mapped to the mechanism of the accessing operation to each file fragmentation；Wherein, the mapped file is used for complete
Original logically storage mode is divided into the division information after file fragmentation and is managed；The division information includes:
Old file name, the logical address of file fragmentation, size, the position in original and data volume and IP address.
Virtualization user's request management center, all requests submitted in user terminal for judging user it is legal
Property, it is then converted to the form of file manipulation command if legal, consigns to the file management center of lower layer；And show the pipe of its data
Reason state, and the file operation requests for obtaining user are linked up, convert it to totally the file operation of file management center receiving,
Thus to the presence of virtualizing file administrative center shielding user, details is realized in the access for shielding file to user.
The user terminal is directly connected to storage sites, completes the transmission of the specific data flow of user's operation.
A kind of virtualization storage management method based on virtualization storage management system of the present invention uses and concentrates control
System, the mode of Distributed Transmission, comprising the following steps:
S1: file of the user in user terminal interface manipulation personal document's system, user terminal are used to virtualization user
Family request management center issues file operation requests.
S2: virtualization user's request management center judges whether user's operation is legal, if legal, by the personal document of user
Operation resolves to the file operation under system virtualization file system, consigns to the virtualizing file administrative center of lower layer.
S3: after virtualizing file administrative center receives the operational order of virtual file, it is parsed into super memory block
File operation sequence in middle file system consigns to the virtualization storage administrative center of lower layer.
S4: it after virtualization storage administrative center receives the file operation sequence in super memory block, is parsed into corresponding
File system operation sequence under storage sites, consigns to the storage sites of lower layer.
S5: each storage sites carry out actual file operation under the instruction that upper layer is sent, such as requirement and user in instruction
Terminal establishes connection, then storage sites can realize data-direct connection with user terminal, and data transmission is realized under the control of upper-level system
Operation.
When some big file being written such as user's request, step S3's is specifically included:
(3.1) virtualizing file administrative center receives the file position that user requests the filename of write operation, required write-in
It sets, the IP address of data volume and user；
(3.2) virtualizing file administrative center reads the corresponding mapped file of this document according to filename；
(3.3) according to request write-in position and mapped file in location information of each file fragmentation in original,
Determine the file fragmentation being written；
(3.4) if finding corresponding file fragmentation, enter in next step；If can not find corresponding file fragmentation,
Apply for a piece of new logical memory space to virtualization storage administrative center, and increases a file fragmentation letter newly in mapped file
The information such as the logical address of file fragmentation, size, position in original are written, then by the newly-increased file fragmentation in breath
As the file fragmentation found, into next step；
(3.5) further determine that data write-in is corresponding logically according to the position of the logical address of file fragmentation and operation
Location；
(3.6) logical address of data manipulation, the data volume of required write-in, the IP address of user are consigned to virtualization and deposited
Administrative center is stored up, is responsible for by virtualization storage administrative center from user's acquisition data, and control storage sites and write data into and deposit
Store up space.
Claims (6)
1. a kind of isomery storage repository virtualization storage management system based on file operation, it is characterised in that: the system includes:
Storage sites, virtualization storage administrative center, virtualizing file administrative center, virtualization user's request management center and user；Its
In,
Storage sites are the set of a variety of physical storages；
Virtualization storage administrative center carries out for the storage resource of all storage sites to be logically unified into an entirety
Management；
Virtualizing file administrative center, for depositing the file fragmentation of virtualization storage administrative center physical store with unity logic
Storage form carries out storage management, shields the logical access details to data；Data are obtained to user terminal；
Virtualization user's request management center: for judging the legitimacy of all requests that user submits in client, if legal
User is then converted to the form of file manipulation command in all requests that client is submitted, consigns to the virtualizing file of lower layer
Administrative center；
Multiple user terminals: for winning the confidence breath in Installed System Memory.
2. the isomery storage repository virtualization storage management system according to claim 1 based on file operation, feature
Be: the virtualization storage administrative center includes multiple super memory blocks, and the super memory block is to of a sort storage
The logic integration of block, attribute are indicated that vector element includes: access rate, the security level of data storage, block by vector
Capacity；The memory block be storage sites in access rate, data storage the identical memory of security level set.
3. the isomery storage repository virtualization storage management system according to claim 1 based on file operation, feature
It is: the virtualizing file administrative center, it is for reference linux system file management mode, various types of storages are empty
Between management be abstracted as file operation；The operation to memory space, the operation to data in memory space, it is unified for file
Operation.
4. the isomery storage repository virtualization storage management system according to claim 1 based on file operation, feature
It is: virtualization user's request management center, for judging the legitimacy of all requests that user submits in user terminal,
It is then converted to the form of file manipulation command if legal, consigns to the file management center of lower layer；And show the management of its data
State, and the file operation requests for obtaining user are linked up, convert it to totally the file operation of file management center receiving, by
This shields the presence of user to virtualizing file administrative center, and details is realized in the access for shielding file to user.
5. the isomery storage repository virtualization storage management system according to claim 1 based on file operation, feature
Be: the user terminal is directly connected to storage sites, completes the transmission of the specific data flow of user's operation.
6. the isomery storage repository virtualization storage management system according to claim 3 based on file operation, feature
Be: the virtualizing file administrative center increases one layer of virtual file and file fragmentation on the basis of Linux file management
Mapping mechanism, data are obtained to user terminal according to mapping mechanism, and storage sites is controlled and writes data into memory space；It is described
Mapping mechanism is the file segmental information that is provided using mapped file as guide, by user to complete virtual file in logic
Accessing operation is mapped to the mechanism of the accessing operation to each file fragmentation；Wherein, the mapped file is used for complete original
Logically storage mode is divided into the division information after file fragmentation and is managed file；The division information includes: original text
Part name, the logical address of file fragmentation, size, the position in original and data volume and IP address.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910424054.3A CN110175162A (en) | 2019-05-21 | 2019-05-21 | A kind of isomery storage repository virtualization storage management system based on file operation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910424054.3A CN110175162A (en) | 2019-05-21 | 2019-05-21 | A kind of isomery storage repository virtualization storage management system based on file operation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110175162A true CN110175162A (en) | 2019-08-27 |
Family
ID=67691763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910424054.3A Pending CN110175162A (en) | 2019-05-21 | 2019-05-21 | A kind of isomery storage repository virtualization storage management system based on file operation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110175162A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112068780A (en) * | 2020-09-10 | 2020-12-11 | 国家卫星气象中心(国家空间天气监测预警中心) | Data unified management system based on heterogeneous storage |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104283960A (en) * | 2014-10-15 | 2015-01-14 | 福建亿榕信息技术有限公司 | System for achieving heterogeneous network storage virtualization integration and hierarchical management |
-
2019
- 2019-05-21 CN CN201910424054.3A patent/CN110175162A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104283960A (en) * | 2014-10-15 | 2015-01-14 | 福建亿榕信息技术有限公司 | System for achieving heterogeneous network storage virtualization integration and hierarchical management |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112068780A (en) * | 2020-09-10 | 2020-12-11 | 国家卫星气象中心(国家空间天气监测预警中心) | Data unified management system based on heterogeneous storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11593037B2 (en) | File system block-level tiering and co-allocation | |
| US9285992B2 (en) | System and method for optimally creating storage objects in a storage system | |
| US9489396B2 (en) | Intermediation of hypervisor file system and storage device models | |
| US20210181963A1 (en) | Native key-value storage enabled distributed storage system | |
| US11734040B2 (en) | Efficient metadata management | |
| US9459897B2 (en) | System and method for providing data analysis service in cloud environment | |
| US11030169B1 (en) | Data re-sharding | |
| CN108351885A (en) | Reduce resource consumption associated with the storage of container and operation | |
| CN103023982A (en) | Low-latency metadata access method of cloud storage client | |
| US20150242311A1 (en) | Hybrid dram-ssd memory system for a distributed database node | |
| CN106570113B (en) | Mass vector slice data cloud storage method and system | |
| US20110238715A1 (en) | Complex object management through file and directory interface | |
| CN107408132B (en) | Method and system for moving hierarchical data objects across multiple types of storage | |
| CN114391141A (en) | Automatic derivation of sharded key values and transparent multi-sharded transaction and query support | |
| US11500822B2 (en) | Virtualized append-only interface | |
| CN103501341A (en) | Method and device for establishing Web service | |
| CN110175162A (en) | A kind of isomery storage repository virtualization storage management system based on file operation | |
| Lu et al. | Design and implementation of the tianhe-2 data storage and management system | |
| WO2021094885A1 (en) | Intelligent data pool | |
| US10509578B2 (en) | Logical address space for storage resource pools | |
| US11287973B2 (en) | Polymorphic storage devices | |
| US11507402B2 (en) | Virtualized append-only storage device | |
| Wei et al. | A high-bandwidth and low-cost data processing approach with heterogeneous storage architectures | |
| Cheng et al. | Tiered data management system: Accelerating data processing on HPC systems | |
| Zhang et al. | HSAStore: A Hierarchical Storage Architecture for Computing Systems Containing Large-Scale Intermediate Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190827 |