CN111212138A - Cross-site storage system and data information access method - Google Patents
Cross-site storage system and data information access method Download PDFInfo
- Publication number
- CN111212138A CN111212138A CN201911425951.2A CN201911425951A CN111212138A CN 111212138 A CN111212138 A CN 111212138A CN 201911425951 A CN201911425951 A CN 201911425951A CN 111212138 A CN111212138 A CN 111212138A
- Authority
- CN
- China
- Prior art keywords
- site
- storage
- data
- controller
- accessed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
Abstract
The application provides a cross-site storage system and a data information access method. The system comprises a plurality of sites, wherein each site comprises a storage unit, a storage set controller, a storage site controller and an access node; the stations are in communication connection through a storage station controller; the storage unit, the storage set controller and the storage site controller of each site are in communication connection; the access node of each site is respectively in communication connection with the storage unit, the storage set controller and the storage site controller; the storage unit is used for storing data information; the storage set controller is used for controlling the storage of the data information by the storage unit; the storage site controller is used for receiving an access request of a client and acquiring corresponding data information from one or more sites; the access node is used for providing an interface for the client to access the system. The cross-site storage system provided by the application is used for performing communication connection on all sites through the storage site controller, so that the storage system capable of storing and managing billions of levels of data information is formed.
Description
Technical Field
The application relates to the technical field of distributed storage, in particular to a cross-site storage system and a data information access method.
Background
With the explosive growth of data information, the development of storage systems is driven to higher density and higher storage capacity. Currently, PB-class storage systems have fallen to a large number of levels and will step toward EB-class scales.
In the prior art, a medium-scale and small-scale storage system is mainly supported, and for a large-scale storage system, the limitation of sites cannot be broken through well, so that the name spaces of all the sites can be presented in a unified view, and the real sharing of data information in the whole system is realized.
Disclosure of Invention
An object of an embodiment of the present application is to provide a cross-site storage system and a data information access method, so as to implement storage and management of billions of data information.
In a first aspect, an embodiment of the present application provides a cross-site storage system, including: a plurality of sites, wherein each site comprises: the system comprises a storage unit, a storage set controller, a storage site controller and an access node; the plurality of stations are in communication connection through a storage station controller; the storage unit, the storage set controller and the storage site controller of each site are in communication connection; the access node of each site is respectively in communication connection with the storage unit, the storage set controller and the storage site controller; the storage unit is used for storing data information; the storage set controller is used for controlling the storage of the data information by the storage unit; the storage site controller is used for receiving an access request of a client and acquiring corresponding data information from storage units of one or more sites according to the access request; the access node is used for providing an interface for a client to access the system.
The cross-site storage system provided by the embodiment of the application is used for performing communication connection on all sites through the storage site controller, so that the storage system capable of storing and managing billions of data information is formed.
Furthermore, the storage site controller comprises a data information service module, a metadata information service module, a management scheduling module, a cross-site service initiating terminal and a cross-site service destination terminal; wherein: the data information service module is used for acquiring corresponding data information according to the received access request and returning the data information to the client; the metadata information service module is used for acquiring corresponding metadata information according to the received access request; the management scheduling module is used for managing the received multiple access requests; the cross-site service initiating terminal is used for determining an initiating site of data information corresponding to the access request; and the cross-site server destination end is used for determining a destination site of the data information corresponding to the access request.
According to the embodiment of the application, the communication connection among all the sites is realized through the storage site controller, and the management of billions of levels of data information is realized through the data information service module, the metadata information service module, the management scheduling module, the cross-site service initiating terminal and the cross-site service destination terminal.
Further, a storage site controller of each site stores global metadata information and metadata information of corresponding sites, wherein the global metadata information comprises the metadata information respectively stored by each site; wherein the metadata information includes file path, site ID, authority, copy information, and access authority information of the directory.
According to the embodiment of the application, the storage site controller is used for storing the global metadata information, so that the specific position of the data information requested by the client can be inquired, and the management and scheduling of the global metadata information are further realized.
Further, the storage site controller is specifically configured to: and the storage site controller acquires corresponding metadata information from the global metadata information according to the access request, and acquires corresponding data to be accessed from a storage unit according to the metadata information.
According to the embodiment of the application, the storage site controller is used for storing the global metadata information, so that the specific position of the data information requested by the client can be inquired, and the management and scheduling of the global metadata information are further realized.
Further, the storage site controller further comprises a local file system and a local client.
In a second aspect, an embodiment of the present application provides a cross-site data information access method, which is applied to a cross-site storage system, where the system includes multiple sites, and each site includes: the system comprises a storage unit, a storage set controller, a storage site controller and an access node; the plurality of stations are in communication connection through a storage station controller; the storage unit, the storage set controller and the storage site controller are in communication connection; the access node is respectively in communication connection with the storage unit, the storage set controller and the storage site controller; the method comprises the following steps: a target site receives an access request sent by a client through an access node; the target site is any one of a plurality of sites; the target site determines a site for storing corresponding data to be accessed according to the access request; the target site acquires corresponding data to be accessed from sites storing the corresponding data to be accessed; and the target site sends the data to be accessed to the client through the access node.
In the embodiment of the application, when a target site of the cross-site storage system receives an access request, a site where data to be accessed is located is determined, and then the corresponding data to be accessed is obtained in a cross-site mode, so that cross-site data information access is realized.
Further, global metadata information is stored in a storage site controller of the target site, the global metadata information includes metadata information respectively stored by each site, and the metadata information includes a site ID; the target site determines a site storing corresponding data to be accessed according to the access request, and the method comprises the following steps: and the target site acquires the site ID for storing the data to be accessed through the global metadata information.
According to the embodiment of the application, the storage site controller can obtain the sites for storing the data to be accessed according to the global metadata information by storing the global metadata information in the storage site controller, and can communicate with the storage site controller of the corresponding site to obtain the corresponding data to be accessed, so that each site is opened, and the data information access of cross sites is realized.
Further, the obtaining, by the target site, the corresponding data to be accessed from the site storing the corresponding data to be accessed includes: if the data to be accessed is stored in the target site, acquiring the corresponding data to be accessed from a storage unit of the target site; and if the data to be accessed is stored in other sites, the target site acquires the data to be accessed from other sites through a storage site controller.
In the embodiment of the application, the target station acquires the data to be accessed from the corresponding station according to the position where the data to be accessed is stored, so that the cross-station data information access is realized.
Further, the storage site controller comprises a data information service module and a metadata information service module; the acquiring the corresponding data to be accessed from the storage unit of the target site includes: the metadata information service module acquires metadata information of the data to be accessed; and the data information service module acquires the corresponding data to be accessed from the storage unit according to the metadata information of the data to be accessed.
Further, the storage site controller includes a management scheduling module, and the method further includes: and if a plurality of access requests are received, determining the processing sequence of the plurality of access requests by using the management scheduling module.
According to the embodiment of the application, the management scheduling module in the storage site controller is used for scheduling the plurality of access requests, so that effective management of data information is realized.
Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
FIG. 1 is a diagram of a cross-site storage system architecture provided by an embodiment of the present application;
fig. 2 is a schematic structural diagram of a storage site controller according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating storage metadata in a storage site controller according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a cross-site data information access method provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
Fig. 1 is a block diagram of a cross-site storage system according to an embodiment of the present invention, and as shown in fig. 1, the system includes a plurality of sites 10, and each site 10 includes a storage unit 101, a storage set controller 102, a storage site controller 103, and an access node 104. The stations 10 are communicatively connected to each other via a storage station controller 103. For each site, the storage unit 101, the storage set controller 102, and the storage site controller 103 are communicatively coupled to form an interconnect. The access node 104 is communicatively connected to the storage unit 101, the storage set controller 102, and the storage site controller 103, respectively, to form an external interconnection. The access node 104 is also communicatively connected to the application node 105, and if the access node 104 and the application node 105 are co-deployed on the same server, the server may be referred to as the application and access node 106.
It should be noted that the communication connection in the above may be a wired communication connection or a wireless communication connection.
The storage unit 101 is used for storing various data information. It is understood that the storage unit may include a plurality of storage areas, and the storage unit 101 may store copies of data information of other sites in addition to data information of the site. In addition, the storage unit 101 may be a magnetic disk, a solid state disk, or the like.
The storage set controller 102 is used for controlling the storage of the data information by the storage unit 101, for example: when there is data information to be stored in the storage unit 101, security detection is performed on the data information, and the data information is stored in the storage unit 101 if the detection is passed.
The storage site controller 103 is configured to implement communication between the sites, connect the sites to form an integrated system, and the storage site controller 103 is configured to receive an access request from a client and obtain corresponding data information from a storage unit of one or more sites according to the access request.
The access node 104 is used to provide an interface for clients to access the system, and the application node 105 may also be referred to as a client, i.e. a node accessing the system.
It should be noted that fig. 1 only shows a schematic diagram of communication between two stations, in a specific implementation process, more stations may be communicatively connected, the number of specific stations may be determined according to the size of data information, and the number of stations is not specifically limited in this embodiment of the application. In addition, the distance between the stations can be set according to actual conditions, and the distance refers to the logical distance between the stations in the system.
According to the cross-site storage system, the sites are in communication connection through the storage site controller, and the storage system capable of storing and managing billions of data information is formed. And through the external interconnection network, cross-site storage can be realized, and the total capacity of the system can be further expanded.
The storage site controller runs a plurality of service modules, runs peer-to-peer services among each other, and provides data requests and service support for each other. Fig. 2 is a schematic structural diagram of a storage site controller according to an embodiment of the present application, and as shown in fig. 2, the storage site controller 103 includes a data information service module 201, a metadata information service module 202, a management scheduling module 203, a cross-site service initiator 204, and a cross-site service destination 205; wherein:
the data information service module 201 is configured to, after the station receives an access request sent by the client, obtain corresponding data information through the data information service module 201, and return the data information to the client.
The metadata information service module 202 is configured to obtain corresponding metadata information according to the received access request; the metadata (Meta Data) is Data about the Data warehouse, and refers to relevant key Data generated during the construction process of the Data warehouse, such as Data source definition, target definition, conversion rule, and the like. While metadata also contains business information about the meaning of the data, all of this information should be well preserved and well managed. And convenience is provided for the development and the use of the data warehouse.
The management scheduling module 203 is configured to manage the received multiple access requests, so as to determine a sequence in which the multiple access requests are processed, or to process the multiple access requests concurrently.
The cross-site service initiator 204 is configured to determine an initiator site of data information corresponding to the access request.
The cross-site server destination 205 is configured to determine a destination site of the data information corresponding to the access request.
For example, the data to be accessed may be divided into three parts, the first part is stored in the a site, the second part is stored in the B site, and the third part is stored in the C site. Then, the cross-site service initiator 204 is a site a, and the cross-site server destination 205 corresponds to a site B and a site C.
The storage site controller 103 of each site stores global metadata information and metadata information of a corresponding site, wherein the global metadata information includes metadata information respectively stored by each site; wherein the metadata information includes file path, site ID, authority, copy information, and access authority information of the directory.
Fig. 3 is a schematic diagram of metadata stored in a storage site controller according to an embodiment of the present application, where fig. 3 shows three sites, namely a site X, a site Y, and a site Z, local metadata information and global metadata information of the site X are stored in the storage site controller of the site X, local metadata information and global metadata information of the site Y are stored in the storage site controller of the site Y, and local metadata information and global metadata information of the site Z are stored in the storage site controller of the site Z. It should be noted that the global metadata information in site X, site Y and site Z is the same. And only the metadata information corresponding to the data in the site X is stored in the site X, only the metadata information corresponding to the data in the site Y is stored in the site Y, and only the metadata information corresponding to the data in the site Z is stored in the site Z.
In addition, Lino in fig. 3 is a local file identifier; gino is the global file identification. If Meta represents the file, the file is represented to have copies at a plurality of sites; if Meta represents a directory, it means that the directory exists shards at multiple sites. It can be seen that the global metadata information for the same file or directory remains consistent at each site.
According to the embodiment of the application, the communication connection among all the sites is realized through the storage site controller, and the management of billions of levels of data information is realized through the data information service module, the metadata information service module, the management scheduling module, the cross-site service initiating terminal and the cross-site service destination terminal.
In order to solve the performance problem of billions of I/O access, the embodiment of the application adopts a method of separately storing metadata and data, and avoids the influence on metadata access caused by the pressure of data access on a disk and a network. Meanwhile, in order to realize that the EB-level cloud storage system manages the global namespace of the cross-region, the embodiment of the application adopts hierarchical management on the metadata information stored in the storage node controller, and the metadata information is divided into two layers: a first level metadata location management layer and a second level metadata management layer.
Metadata location management layer: the system comprises a directory tree structure, a file access layer and a file management module, wherein the directory tree structure is used for managing server position information where file metadata are located and storing directory tree structures forming user views, and the application access layer is provided with specific metadata storage position information of each file and a traditional directory tree structure. The copies of the metadata information are at least 2 copies at the local site to increase the availability of metadata, and copies may be made at the remote site if desired. Meanwhile, in order to implement the global namespace, the metadata location information of the file is synchronized with each other at each site, so that the metadata information of the site can be accessed at other sites.
Metadata management layer: the layer is used for managing basic attributes (such as size and modification time) of the file, storage location information of data information, file extension attributes (such as authority configuration) and the like. In the storage system, because massive files need to be managed, the metadata amount of each file is large, and the storage requirement of the metadata information of the level is larger than that of the metadata information of the level.
On the basis of the foregoing embodiment, the storage site controller is specifically configured to:
and the storage site controller acquires corresponding metadata information from the global metadata information according to the access request, and acquires corresponding data to be accessed from a storage unit according to the metadata information.
In a specific implementation process, since the storage site controller stores global metadata information, when a site receives an access request sent by a certain client, a metadata information service module in the storage site controller can be called to obtain metadata information corresponding to data to be accessed from the global metadata information, and after the metadata information is obtained, a data information service module is called to obtain corresponding data information from a storage unit and return the data information to the client.
In addition, the storage site controller 103 may further include a local client 206 and a local file system 207; wherein:
the local client 206 is a client running at a corresponding site.
The local file system 207 is a method and data structure used by the operating system to clarify files on partitions of the storage device (often disks, also NANDFlash-based solid state drives); i.e. a method of organizing files on a storage device. The local file system 207 includes a file system interface 208.
Fig. 4 is a schematic flowchart of a cross-site data information access method provided in an embodiment of the present application, and as shown in fig. 4, the method is applied to the cross-site storage system provided in each of the embodiments, and includes:
step 401: a target site receives an access request sent by a client through an access node; wherein the target site is any one of a plurality of sites.
In a specific implementation process, the client is in communication connection with the target station through the access node, and therefore, the client can send an access request to the target station through the access node. It is understood that one station can establish communication with a plurality of clients, and for convenience of description, the embodiments of the present application are described by processing an access request received by one station (target station), and the processing flow of other stations is consistent with that of the target station.
Step 402: and the target site determines a site for storing corresponding data to be accessed according to the access request.
In a specific implementation process, after receiving an access request sent by a client, a target site determines a site storing data to be accessed according to the access request. It can be understood that if the client needs one data set and the data in the data set is not stored in one site, it can be determined that there are multiple sites to access the data.
Step 403: and the target site acquires the corresponding data to be accessed from the site storing the corresponding data to be accessed.
In a specific implementation process, since the stations are all in communication connection through the storage station controller, after the station for storing the data to be accessed is determined, the data to be accessed can be acquired from the corresponding station through the storage station controller.
Step 404: and the target site sends the data to be accessed to the client through the access node.
In a specific implementation process, after the target station acquires the data to be accessed, the data to be accessed is returned to the data to be accessed through the access node.
In the embodiment of the application, when a target site of the cross-site storage system receives an access request, a site where data to be accessed is located is determined, and then the corresponding data to be accessed is obtained in a cross-site mode, so that cross-site data information access is realized.
On the basis of the above embodiment, the storage site controller of the target site stores global metadata information, where the global metadata information includes metadata information stored by each site, and the metadata information includes a site ID; the target site determines a site storing corresponding data to be accessed according to the access request, and the method comprises the following steps:
and the target site acquires the site ID for storing the data to be accessed through the global metadata information.
In a specific implementation process, the global metadata information refers to a sum of metadata information stored in each site in the system. A copy of global metadata information is stored in the storage node controller of each site. One piece of metadata information includes a site ID, that is, a site ID storing the metadata information, each site has a unique site ID, and thus, the site ID can be used to know where the metadata information is stored. In addition, the metadata information also includes file path, authority, copy information, and access authority information of the entire directory, and the like. The file path refers to a file path for storing data information corresponding to the metadata information; the authority refers to the operated authority of the data information corresponding to the metadata information, for example: read only or read and write, etc. The copy information refers to the copy condition of the data corresponding to the metadata, such as how many copies are in total, and the storage location of each copy.
After receiving the access request, the target site may obtain a site ID corresponding to the data to be accessed from the global metadata information to know which sites the data to be accessed is stored on, and then obtain the data on the corresponding sites.
According to the embodiment of the application, the storage site controller can obtain the sites for storing the data to be accessed according to the global metadata information by storing the global metadata information in the storage site controller, and can communicate with the storage site controller of the corresponding site to obtain the corresponding data to be accessed, so that each site is opened, and the data information access of cross sites is realized.
On the basis of the above embodiment, the acquiring, by the target station, the corresponding data to be accessed from the station storing the corresponding data to be accessed includes:
if the data to be accessed is stored in the target site, acquiring the corresponding data to be accessed from a storage unit of the target site;
and if the data to be accessed is stored in other sites, the target site acquires the data to be accessed from other sites through a storage site controller.
In a specific implementation process, if the target site determines that the data to be accessed is stored in the local site, the metadata information service module in the local site may be called to obtain metadata information corresponding to the data to be accessed, and then the data information service module in the local site is called to obtain data corresponding to the metadata information, where the data is the data to be accessed.
If the target site determines that the data to be accessed is stored in other sites, it can be understood that one or more other sites may be used, the data request may be sent to the site storing the data to be accessed through the storage site controller of the local site, the site storing the data to be accessed acquires metadata information corresponding to the data to be accessed from the local metadata information according to the data request, then acquires the corresponding data to be accessed according to the metadata information, and returns the data to be accessed to the target site.
It is understood that, among the data to be accessed, there may be a portion of the data stored in the destination site and another portion of the data stored in the other site. The target site can acquire the data stored in the target site through the method, acquire another part of data from other sites through the method, integrate the data to obtain the data to be accessed, and send the data to be accessed to the corresponding client.
In the embodiment of the application, the target station acquires the data to be accessed from the corresponding station according to the position where the data to be accessed is stored, so that the cross-station data information access is realized.
On the basis of the above embodiment, the storage site controller includes a management scheduling module, and the method further includes:
and if a plurality of access requests are received, determining the processing sequence of the plurality of access requests by using the management scheduling module.
In a specific implementation process, since one station can be in communication connection with a plurality of clients, the station may receive access requests sent by the plurality of clients in a short time. The management scheduling module may perform management scheduling on the multiple access requests, that is, determine a processing order of the multiple access requests. For example: the determination can be made according to the time of the access request received by the target station, or according to the priority of the client. Thereby achieving orderly management of access requests.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A cross-site storage system, comprising: a plurality of sites, wherein each site comprises: the system comprises a storage unit, a storage set controller, a storage site controller and an access node; the plurality of stations are in communication connection through a storage station controller; the storage unit, the storage set controller and the storage site controller of each site are in communication connection; the access node of each site is respectively in communication connection with the storage unit, the storage set controller and the storage site controller;
the storage unit is used for storing data information;
the storage set controller is used for controlling the storage of the data information by the storage unit;
the storage site controller is used for receiving an access request of a client and acquiring corresponding data information from storage units of one or more sites according to the access request;
the access node is used for providing an interface for a client to access the system.
2. The system of claim 1, wherein the storage site controller comprises a data information service module, a metadata information service module, a management scheduling module, a cross-site service initiator, and a cross-site service destination; wherein:
the data information service module is used for acquiring corresponding data information according to the received access request and returning the data information to the client;
the metadata information service module is used for acquiring corresponding metadata information according to the received access request;
the management scheduling module is used for managing the received multiple access requests;
the cross-site service initiating terminal is used for determining an initiating site of data information corresponding to the access request;
and the cross-site server destination end is used for determining a destination site of the data information corresponding to the access request.
3. The system according to claim 2, wherein the storage site controller of each site stores global metadata information and metadata information of the corresponding site, and the global metadata information includes metadata information stored by each site respectively; wherein the metadata information includes file path, site ID, authority, copy information, and access authority information of the directory.
4. The system of claim 3, wherein the storage site controller is specifically configured to:
and the storage site controller acquires corresponding metadata information from the global metadata information according to the access request, and acquires corresponding data to be accessed from a storage unit according to the metadata information.
5. The system of claim 2, wherein the storage site controller further comprises a local file system and a local client.
6. A cross-site data information access method is applied to a cross-site storage system, the system comprises a plurality of sites, and each site comprises: the system comprises a storage unit, a storage set controller, a storage site controller and an access node; the plurality of stations are in communication connection through a storage station controller; the storage unit, the storage set controller and the storage site controller are in communication connection; the access node is respectively in communication connection with the storage unit, the storage set controller and the storage site controller; the method comprises the following steps:
a target site receives an access request sent by a client through an access node; the target site is any one of a plurality of sites;
the target site determines a site for storing corresponding data to be accessed according to the access request;
the target site acquires corresponding data to be accessed from sites storing the corresponding data to be accessed;
and the target site sends the data to be accessed to the client through the access node.
7. The method according to claim 6, wherein the storage site controller of the target site stores therein global metadata information, the global metadata information including metadata information stored respectively by the respective sites, the metadata information including site IDs; the target site determines a site storing corresponding data to be accessed according to the access request, and the method comprises the following steps:
and the target site acquires the site ID for storing the data to be accessed through the global metadata information.
8. The method of claim 6, wherein the target site obtains the corresponding data to be accessed from a site storing the corresponding data to be accessed, and comprises:
if the data to be accessed is stored in the target site, acquiring the corresponding data to be accessed from a storage unit of the target site;
and if the data to be accessed is stored in other sites, the target site acquires the data to be accessed from other sites through a storage site controller.
9. The method of claim 8, wherein the storage site controller comprises a data information service module and a metadata information service module; the acquiring the corresponding data to be accessed from the storage unit of the target site includes:
the metadata information service module acquires metadata information of the data to be accessed;
and the data information service module acquires the corresponding data to be accessed from the storage unit according to the metadata information of the data to be accessed.
10. The method of any of claims 6-9, wherein the storage site controller comprises a management scheduling module, the method further comprising:
and if a plurality of access requests are received, determining the processing sequence of the plurality of access requests by using the management scheduling module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911425951.2A CN111212138B (en) | 2019-12-31 | 2019-12-31 | Cross-site storage system and data information access method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911425951.2A CN111212138B (en) | 2019-12-31 | 2019-12-31 | Cross-site storage system and data information access method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111212138A true CN111212138A (en) | 2020-05-29 |
| CN111212138B CN111212138B (en) | 2022-11-22 |
Family
ID=70788593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911425951.2A Active CN111212138B (en) | 2019-12-31 | 2019-12-31 | Cross-site storage system and data information access method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111212138B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190625A (en) * | 2021-05-25 | 2021-07-30 | 中国工商银行股份有限公司 | Request processing method, device, electronic equipment, medium and program product |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101271458A (en) * | 2007-03-22 | 2008-09-24 | 阿里巴巴公司 | Distributed computation file management method, system and device thereof |
| CN101520805A (en) * | 2009-03-25 | 2009-09-02 | 中兴通讯股份有限公司 | Distributed file system and file processing method thereof |
| US20090282240A1 (en) * | 2008-05-12 | 2009-11-12 | Huazhong University Of Science & Technology | Secure Decentralized Storage System |
| CN102158546A (en) * | 2011-02-28 | 2011-08-17 | 中国科学院计算技术研究所 | Cluster file system and file service method thereof |
| CN107920131A (en) * | 2017-12-08 | 2018-04-17 | 郑州云海信息技术有限公司 | A kind of metadata management method and device of HDFS storage systems |
-
2019
- 2019-12-31 CN CN201911425951.2A patent/CN111212138B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101271458A (en) * | 2007-03-22 | 2008-09-24 | 阿里巴巴公司 | Distributed computation file management method, system and device thereof |
| US20090282240A1 (en) * | 2008-05-12 | 2009-11-12 | Huazhong University Of Science & Technology | Secure Decentralized Storage System |
| CN101520805A (en) * | 2009-03-25 | 2009-09-02 | 中兴通讯股份有限公司 | Distributed file system and file processing method thereof |
| CN102158546A (en) * | 2011-02-28 | 2011-08-17 | 中国科学院计算技术研究所 | Cluster file system and file service method thereof |
| CN107920131A (en) * | 2017-12-08 | 2018-04-17 | 郑州云海信息技术有限公司 | A kind of metadata management method and device of HDFS storage systems |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190625A (en) * | 2021-05-25 | 2021-07-30 | 中国工商银行股份有限公司 | Request processing method, device, electronic equipment, medium and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111212138B (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220284046A1 (en) | Application Programming Interface-based Writing of Data to a Directory of a File Structure Layer of a Data Storage System | |
| CN103067461B (en) | A kind of metadata management system of file and metadata management method | |
| CN101576915B (en) | Distributed B+ tree index system and building method | |
| CN102708165B (en) | Document handling method in distributed file system and device | |
| CN105025053A (en) | Distributed file upload method based on cloud storage technology and system | |
| CN105138571B (en) | Distributed file system and method for storing massive small files | |
| CN105933376A (en) | Data manipulation method, server and storage system | |
| CN109542861B (en) | File management method, device and system | |
| CN104539681A (en) | Distributed GIS accelerating system and GIS service processing method | |
| KR100977159B1 (en) | Method And System Of File Metadata Management Transparent About Address Changing Of Data Servers And Their Disk Movements | |
| CN105426469A (en) | Database cluster metadata management method and system | |
| KR20120044550A (en) | Cloud storage server and system by use of virtual nas and method thereof | |
| CN109597903A (en) | Image file processing apparatus and method, document storage system and storage medium | |
| US11055259B2 (en) | Method and system for deleting obsolete files from a file system | |
| CN111212138B (en) | Cross-site storage system and data information access method | |
| CN111435286A (en) | Data storage method, device and system | |
| CN111274004B (en) | Process instance management method and device and computer storage medium | |
| CN107479827A (en) | A kind of mixing storage system implementation method based on IO and separated from meta-data | |
| EP3572951A1 (en) | A method for handling requests in a storage system and a storage node for a storage system | |
| CN113168405A (en) | Database management service providing system | |
| KR20120073799A (en) | Data synchronizing and servicing apparatus and method based on cloud storage | |
| CN101741861B (en) | Data storage method and device | |
| WO2023040504A1 (en) | Data processing system, data processing method and related apparatus | |
| CN114546580A (en) | Cache deployment system, cache deployment method, electronic device and storage medium | |
| CN115202814A (en) | Business resource management and control method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |