CN106293490A - Data storage, the method read, Apparatus and system - Google Patents
Data storage, the method read, Apparatus and system Download PDFInfo
- Publication number
- CN106293490A CN106293490A CN201510240369.4A CN201510240369A CN106293490A CN 106293490 A CN106293490 A CN 106293490A CN 201510240369 A CN201510240369 A CN 201510240369A CN 106293490 A CN106293490 A CN 106293490A
- Authority
- CN
- China
- Prior art keywords
- data
- interface
- storage resource
- storage
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides the storage of a kind of data, the method for reading, Apparatus and system, wherein, the method by data distribution reason system according to the feature of data to be stored and/or metadata, and the storage strategy of this data distribution reason system, in the default storage resource that this storage strategy is corresponding, obtaining the storage resource corresponding with this feature and/or metadata, wherein, this is preset storage resource and is made up of the storage resource of at least one or more memory node；These data to be stored are stored to the memory node corresponding with this storage resource, solves storage architecture incompatible, interface disunity, it is impossible to the problem sharing storage resource, it is achieved that the unification of storage resource is shared, and has saved resource and maintenance costs.
Description
Technical field
The present invention relates to the communications field, in particular to the storage of a kind of data, the method for reading, Apparatus and system.
Background technology
Along with the arrival in the Internet secondary (Web2.0) epoch, in the face of Internet data center (Internet Data Center,
Referred to as IDC) the big demand data of explosion type, and diversified intelligent terminal, popularizing of Internet of Things.Conventional store
The drawback of service is more and more obvious, it is impossible to meet the demand of growing the Internet.Set up service-oriented " cloud storage "
Environment becomes a kind of urgent current demand from unreachable extravagant hope.
In the related, it is the storage architecture of conventional dedicated that storage architecture can be divided into two kinds: one, such as memory area
Network (Network Attached Storage, referred to as SAN) and network access server (Storage Area Network
And SAN Protocols, referred to as NAS) etc., it being made up of common server, its unit cost is expensive, expansible
Property is poor, is not suitable for any wide-area deployment, but its Performance And Reliability has certain advantage.
Another is conventional cloud storage architecture, such as distributed file system, and this distributed file system low cost, can
Linear expansion, so its scale and system-wide handling capacity can be made the biggest.But have a disadvantage in that the reliable of a single point
Property and performance not as SAN and NAS.
Above two storage architecture is incompatible, and what user can not dynamically select according to practical situation oneself to want flexibly is
System, the storage system of both frameworks, also uses the interface of each self-retaining simultaneously, it is impossible to tackle diversified Ethernet
(Internet) client and their respective storage agreement.
For in correlation technique, storage architecture is incompatible, interface disunity, it is impossible to the problem sharing storage resource, mesh
Before effective solution is not yet proposed.
Summary of the invention
The invention provides the storage of a kind of data, the method for reading, Apparatus and system, at least to solve correlation technique is deposited
Storage framework is incompatible, interface disunity, it is impossible to the problem sharing storage resource.
According to an aspect of the invention, it is provided a kind of date storage method, including:
Data distribution reason system is according to the feature of data to be stored and/or metadata, and described data distribution reason system
The storage strategy of system, in the default storage resource that described storage strategy is corresponding, obtains and described feature and/or metadata
Corresponding storage resource, wherein, described default storage resource is made up of the storage resource of at least one or more memory node；
Described data to be stored are stored to the memory node corresponding with described storage resource.
Further, described data distribution reason system also includes:
Key assignments KV application programming interface API client and/or key assignments KV interface.
Further, feature and/or the metadata of described data to be stored includes:
The key assignments KV of described data to be stored.
Further, described data to be stored are stored to the memory node corresponding with described storage resource, including:
Obtaining the storage resource burst corresponding with described storage resource, wherein, described default storage resource is by described storage money
Source burst composition；
According to described storage resource burst in the space routing table of described default storage resource, search and described data to be stored
Corresponding memory node, described memory node is used for receiving the described data to be stored of write.
Further, in the case of described storage strategy is concordance Hash hash algorithm, described storage resource burst
For the burst on concordance Hash hash algorithm ring, wherein, the corresponding described default storage resource of described ring.
Further, described data to be stored are stored to the memory node corresponding with described storage resource and include:
By one or more copies of described data to be stored to being written in other memory node.
Further, described method includes:
Visited by key assignments KV interface, object accesses client OAC interface, block access customer end B AC interface or file
Ask that client FAC interface obtains described data to be stored.
Further, when obtaining described data to be stored by described OAC interface, described method also includes:
Described data distribution reason system receives application program and carries out REST association by declarative state transmission REST gateway
View resolves, and object information is submitted to OAC, described OAC described right by object data server OMDS acquisition
The data described to be stored that image information is corresponding.
Further, when obtaining described data to be stored by described BAC interface, described method also includes:
Described data distribution reason system is received application program and is received by small computer system interface ISCSI object
After ISCSI instruction, being converted to the internal interface of BAC, described BAC is converted to the described to be stored of correspondence blocks of data
Data.
Further, when obtaining described data to be stored by described FAC interface, described method also includes:
Described data distribution reason system receives application program
System interface POSIX interface, or commodity ethernet file system CIFS interface be converted into inside file access client
FAC interface, described FAC carries out the metadata information of file the group of metadata by file data server FMDS
Knit management, obtain the data to be stored that described metadata information is corresponding.
According to another aspect of the present invention, additionally provide a kind of method for reading data, including:
Data distribution reason system is according to the feature of data to be read and/or metadata, and described data distribution reason system
The storage strategy of system, in the default storage resource that described storage strategy is corresponding, obtains and described feature and/or metadata
Corresponding storage resource, wherein, described default storage resource is made up of the storage resource of at least one or more memory node；
Read the memory node that the described storage resource of described data to be read is corresponding.
Further, described data distribution reason system also includes:
Key assignments KV application programming interface API client and/or key assignments KV interface.
Further, feature and/or the metadata of described data to be read includes:
The key assignments KV of described data to be read.
Further, the memory node that the described storage resource of the described data to be read of described reading is corresponding includes:
Obtaining the storage resource burst corresponding with described storage resource, wherein, described default storage resource is by described storage money
Source burst composition；
According to described storage resource burst in the space routing table of described default storage resource, search and described data to be read
Corresponding memory node, reads the memory node of described correspondence.
Further, in the case of described storage strategy is concordance Hash hash algorithm, described storage resource burst
For the burst on concordance Hash hash algorithm ring, wherein, the corresponding described default memory space of described ring.
Further, the memory node reading the described storage resource of described data to be read corresponding includes:
Search the memory node of copy set corresponding to described data to be read, from the memory node of described copy set, read institute
State data to be read.
Further, described method includes:
Visited by key assignments KV interface, object accesses client OAC interface, block access customer end B AC interface or file
Ask that client FAC interface reads described data to be read.
Further, when reading described data to be read by described OAC interface, described method also includes:
Described data distribution reason system reads described data to be read and returns to object data server OMDS, described
Described data to be read are returned to described OAC interface by OMDS, and are entered by declarative state transmission REST gateway
Row REST protocol analysis, returns to application program.
Further, when reading described data to be read by described BAC interface, described method also includes:
Described data distribution reason system reads described data to be read by the interface of described BAC and returns to small-sized calculating
Machine system interface ISCSI object, described data to be read are returned to application program by described ISCSI object.
Further, when reading described data to be read by described FAC interface, described method also includes:
Described data distribution reason system reads described data to be read and returns to file data server FMDS, and passes through
Described FAC interface reads described data to be read, then by NFS NFS interface, portable operating system
Interface POSIX interface, or commodity ethernet file system CIFS interface returns to application program.
According to another aspect of the present invention, additionally provide a kind of data storage device, including:
First acquisition module, for data distribution reason system according to the feature of data to be stored and/or metadata, and
The storage strategy of described data distribution reason system, in the default storage resource that described storage strategy is corresponding, obtains and institute
Stating feature and/or storage resource corresponding to metadata, wherein, described default storage resource is stored by least one or more
The storage resource composition of node；
Memory module, for storing described data to be stored to the memory node corresponding with described storage resource.
According to another aspect of the present invention, additionally provide a kind of digital independent device, including:
Second acquisition module, for data distribution reason system according to the feature of data to be read and/or metadata, and
The storage strategy of described data distribution reason system, in the default storage resource that described storage strategy is corresponding, obtains and institute
Stating feature and/or storage resource corresponding to metadata, wherein, described default storage resource is stored by least one or more
The storage resource composition of node；
Read module, for reading the memory node that the described storage resource of described data to be read is corresponding.
According to another aspect of the present invention, additionally provide a kind of data handling system, including: key assignments KV stores data
Storehouse, key assignments KV application programming interface api interface；
Described key assignments KV stores data base, forms unified default storage for integrating the storage resource on each memory node
Resource；
Described key assignments KV application programming interface api interface, is used for the storage according to data or read requests, with
And the key value Key application of described data presets the different memory node of memory space, wherein, described default storage money
Source is made up of the storage resource of multiple memory nodes.
Further, the function of described key assignments KV application programming interface api interface includes at least one of:
Write data-interface, read data-interface, and delete the data-interface described data of process；
Described default memory space is carried out burst by concordance Hash hash algorithm, generates storage resource burst；
Manage described storage resource burst；
Manage the space routing table that described storage resource burst is corresponding.
Further, described system also includes: operating maintenance module OMM, monitoring module,
Described OMM, is used for arranging at least one of function: the management of described storage resource, described storage resource
Divide, the configuration of described storage resource, and the alarm of system shows；
Described monitoring module, for monitoring described memory node and the state of described disk.
Further, described system also includes: object accesses client OAC interface, and block access customer end B AC connects
Mouthful, and file access client FAC interface,
Object accesses client OAC interface, block access customer end B AC interface, or file access client FAC
Interface is for storing the data of described key assignments KV storage data base or read.
Further, also include: configuration module
Described configuration module is for configuring the storage strategy of storage resource, and described storage strategy includes: copy pattern, snapshot
Support pattern, compact model, cache mode.
By the present invention, by data distribution reason system according to the feature of data to be stored and/or metadata, and should
Data distribution reason system storage strategy, in the default storage resource that this storage strategy is corresponding, obtain with this feature and
/ or storage resource corresponding to metadata, wherein, this presets storage resource by the storage of at least one or more memory node
Resource forms；These data to be stored are stored to the memory node corresponding with this storage resource, solves storage architecture mutual
Incompatible, interface disunity, it is impossible to the problem sharing storage resource, it is achieved that the unification of storage resource is shared, and saves
Resource and maintenance costs.
Accompanying drawing explanation
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, the present invention
Schematic description and description be used for explaining the present invention, be not intended that inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the flow chart of a kind of date storage method according to embodiments of the present invention；
Fig. 2 is the flow chart of a kind of method for reading data according to embodiments of the present invention；
Fig. 3 is the structured flowchart of a kind of data storage device according to embodiments of the present invention；
Fig. 4 is the structured flowchart of a kind of digital independent device according to embodiments of the present invention；
Fig. 5 is the schematic diagram storing system overall architecture according to the preferred embodiment of the invention；
Fig. 6 is the schematic diagram of KV database store process according to the preferred embodiment of the invention；
Fig. 7 is the schematic diagram of the internal composition of KV API according to the preferred embodiment of the invention；
Fig. 8 is the schematic diagram of distributed objects storage according to the preferred embodiment of the invention；
Fig. 9 is the schematic diagram of distributed block storage according to the preferred embodiment of the invention；
Figure 10 is the schematic diagram of distributed document storage according to the preferred embodiment of the invention.
Detailed description of the invention
Below with reference to accompanying drawing and describe the present invention in detail in conjunction with the embodiments.It should be noted that in the feelings do not conflicted
Under condition, the embodiment in the application and the feature in embodiment can be mutually combined.
It should be noted that term " first " in description and claims of this specification and above-mentioned accompanying drawing, " second "
Etc. being for distinguishing similar object, without being used for describing specific order or precedence.
Providing a kind of date storage method in the present embodiment, Fig. 1 is a kind of data storage according to embodiments of the present invention
The flow chart of method, as it is shown in figure 1, this flow process comprises the steps:
Step S102, data distribution reason system is according to the feature of data to be stored and/or metadata, and these data are divided
Send out the storage strategy of processing system, in the default storage resource that this storage strategy is corresponding, obtain and this feature and/or unit
The storage resource that data are corresponding, wherein, this presets storage resource by the storage resource group of at least one or more memory node
Become；
These data to be stored are stored to the memory node corresponding with this storage resource by step S104.
By above-mentioned steps, data distribution reason system is according to the feature of data to be stored and/or metadata, and this number
According to the algorithm of the storage strategy of distributing and processing system, obtain the storage resource corresponding with this feature and/or metadata, storage
These data to be stored store to the memory node corresponding with this storage resource, solve storage architecture incompatible, interface
Disunity, it is impossible to the problem sharing storage resource, it is achieved that the unification of storage resource is shared, and has saved resource and maintenance is opened
Pin.
This data distribution reason system may include that key assignments (Key Value, referred to as KV) applies journey in the present embodiment
Sequence DLL client (Application Programming Interface, referred to as API) client and/or key assignments
(KV) interface.Feature and/or the metadata of these data to be stored include: the key assignments (KV) of these data to be stored.
In the present embodiment, these data to be stored are stored to the memory node corresponding with this storage resource, including:
Obtaining the storage resource burst corresponding with this storage resource, wherein, this presets storage resource by this storage resource burst
Composition；
According to this storage resource burst in this space routing table presetting storage resource, search corresponding with these data to be stored
Memory node, this memory node is used for receiving these data to be stored of write.
In the present embodiment, in the case of this storage strategy is concordance Hash hash algorithm, this storage resource burst
For the burst on concordance Hash hash algorithm ring, wherein, this ring is to presetting storage resource.
In the present embodiment, these data to be stored are stored to the memory node corresponding with this storage resource and include:
By one or more copies of these data to be stored to being written in other memory node.
In the present embodiment, can pass through key assignments KV interface, object accesses client (Object Access Client,
Referred to as OAC) interface, block access client (Block Access Client, referred to as BAC) interface or file
Access client (File Access Client, referred to as FAC) file interface obtains this data to be stored.
Wherein, when obtaining these data to be stored by this OAC interface, this data distribution reason system receives application journey
Sequence is transmitted (Representational State Transfer is called for short REST) gateway by declarative state and is carried out REST
Protocol analysis, submits to OAC by object information, and this OAC passes through object data server (Object Meta Data
Server, referred to as OMDS) obtain these data to be stored that this object information is corresponding.
Wherein, when obtaining these data to be stored by this BAC interface, this data distribution reason system receives application journey
Sequence passes through small computer system interface (Internet Small Computer System Interface, referred to as ISCSI)
After object receives ISCSI instruction, being converted to the internal interface of BAC, this BAC is converted to being somebody's turn to do of correspondence blocks of data
Data to be stored.
Wherein, when by these these data to be stored of FAC file acquisition, this data distribution reason system receives application journey
Sequence passes through NFS (Network File System, referred to as NFS) interface, portable operating system interface
(Portable Operating System Interface, referred to as POSIX) interface, or commodity ethernet file system
File within (Common Internet File System, referred to as CIFS) interface conversion one-tenth accesses client FAC
Interface, the metadata information of file, by file data server, (File Meta Data Server is called for short this FAC
For FMDS) carry out the organization and administration of metadata, obtain the data to be stored that this metadata information is corresponding.
Providing a kind of method for reading data in the present embodiment, Fig. 2 is a kind of digital independent according to embodiments of the present invention
The flow chart of method, as in figure 2 it is shown, this flow process comprises the steps:
Step S202, data distribution reason system is according to the feature of data to be read and/or metadata, and these data are divided
Send out the storage strategy of processing system, in the default storage resource that this storage strategy is corresponding, obtain and this feature and/or unit
The storage resource that data are corresponding, wherein, this presets storage resource by the storage resource group of at least one or more memory node
Become；
Step S204, reads the memory node that this storage resource of these data to be read is corresponding.
By above-mentioned steps, data distribution reason system is according to the feature of data to be read and/or metadata, and this number
According to the algorithm of the storage strategy of distributing and processing system, obtain the storage resource corresponding with this feature and/or metadata, read
The memory node that this storage resource of these data to be read is corresponding, solves storage architecture incompatible, interface disunity,
The problem that can not share storage resource, it is achieved that the unification of storage resource is shared, and has saved resource and maintenance costs.
In the present embodiment, this data distribution reason system also includes:
Key assignments KV application programming interface API client and/or key assignments KV interface.
In the present embodiment, feature and/or the metadata of these data to be read includes:
The key assignments KV of these data to be read.
In the present embodiment, the memory node that this storage resource of these data to be read of this reading is corresponding includes:
Obtaining the storage resource burst corresponding with this storage resource, wherein, this presets storage resource by this storage resource burst
Composition；
According to this storage resource burst in this space routing table presetting storage resource, search corresponding with these data to be read
Memory node, reads the memory node of this correspondence.
In the present embodiment, in the case of this storage strategy is concordance Hash hash algorithm, this storage resource burst
For the burst on concordance Hash hash algorithm ring, wherein, this ring is to presetting memory space.
In the present embodiment, the memory node reading this storage resource of these data to be read corresponding includes:
Search the memory node of copy set corresponding to these data to be read, from the memory node of this copy set, read this continue
Fetch data.
In the present embodiment, key assignments KV interface, object accesses client OAC interface, block access client can be passed through
End BAC interface or file access client FAC interface read this data to be read.
Wherein, when reading these data to be read by this OAC interface, this data distribution reason system reads this and continues
Fetching data and return to object data server OMDS, these data to be read are returned to this OAC interface by this OMDS,
And carry out REST protocol analysis by declarative state transmission REST gateway, return to application program.
Wherein, when reading these data to be read by this BAC interface, this data distribution reason system is by this BAC
Interface read these data to be read and return to small computer system interface ISCSI object, this is treated by this ISCSI object
Read data and return to application program.
Wherein, when reading these data to be read by this FAC file, this data distribution reason system reads this and continues
Fetch data and return to file data server FMDS, and read this data to be read by this FAC interface, then pass through
NFS NFS interface, portable operating system interface POSIX interface, or commodity ethernet file system CIFS
Interface returns to application program.
Additionally providing a kind of data storage device in the present embodiment, this device is used for realizing above-described embodiment and being preferable to carry out
Mode, had carried out repeating no more of explanation.As used below, term " module " can realize predetermined function
Software and/or the combination of hardware.Although the device described by following example preferably realizes with software, but firmly
Part, or the realization of the combination of software and hardware also may and be contemplated.
Fig. 3 is the structured flowchart of a kind of data storage device according to embodiments of the present invention, as it is shown on figure 3, this device bag
Include
First acquisition module 32, for data distribution reason system according to the feature of data to be stored and/or metadata, with
And the storage strategy of this data distribution reason system, in the default storage resource that this storage strategy is corresponding, obtain and this spy
Levying and/or storage resource that metadata is corresponding, wherein, this presets storage resource by least one or more memory node
Storage resource composition；
Memory module 34, for storing these data to be stored to the memory node corresponding with this storage resource.
By said apparatus, data distribution reason system is according to the feature of data to be stored and/or metadata, and this number
According to the algorithm of the storage strategy of distributing and processing system, obtain the storage resource corresponding with this feature and/or metadata, storage
These data to be stored store to the memory node corresponding with this storage resource, solve storage architecture incompatible, interface
Disunity, it is impossible to the problem sharing storage resource, it is achieved that the unification of storage resource is shared, and has saved resource and maintenance is opened
Pin.
Fig. 4 is the structured flowchart of a kind of digital independent device according to embodiments of the present invention, as shown in Figure 4, this device bag
Include
Second acquisition module 42, for data distribution reason system according to the feature of data to be read and/or metadata, with
And the storage strategy of this data distribution reason system, in the default storage resource that this storage strategy is corresponding, obtain and this spy
Levying and/or storage resource that metadata is corresponding, wherein, this presets storage resource by least one or more memory node
Storage resource composition；
Read module 44, for reading the memory node that this storage resource of these data to be read is corresponding.
By above-mentioned steps, data distribution reason system is according to the feature of data to be read and/or metadata, and this number
According to the algorithm of the storage strategy of distributing and processing system, obtain the storage resource corresponding with this feature and/or metadata, read
The memory node that this storage resource of these data to be read is corresponding, solves storage architecture incompatible, interface disunity,
The problem that can not share storage resource, it is achieved that the unification of storage resource is shared, and has saved resource and maintenance costs.
In an embodiment of the present invention, additionally provide a kind of data handling system, including: key assignments KV stores data base,
Key assignments KV application programming interface api interface,
This key assignments KV stores data base, forms unified default storage money for integrating the storage resource on each memory node
Source；
This key assignments KV application programming interface api interface, for the storage according to data or read requests, and
The different memory node of memory space is preset in the key value Key application of these data, and wherein, this presets storage resource by extremely
The storage resource composition of few one or more memory nodes.
In the present embodiment, the function of this key assignments KV application programming interface api interface includes at least one of:
Write data-interface, read data-interface, and deletion data-interface processes this data；
This default memory space is carried out burst by concordance Hash hash algorithm, generates storage resource burst；
Manage this storage resource burst；
Manage the space routing table that this storage resource burst is corresponding.
In the present embodiment, this system also includes: operating maintenance module OMM, monitoring module,
This OMM, is used for arranging at least one of function: the management of this storage resource, the division of this storage resource,
The configuration of this storage resource, and the alarm of system shows；
This monitoring module, for monitoring the state of this memory node and this disk.
In the present embodiment, this system also includes: object accesses client OAC interface, block access customer end B AC
Interface, and file access client FAC interface,
Object accesses client OAC interface, block access customer end B AC interface, or file access client FAC
Interface is for storing the data of this key assignments KV storage data base or read.
In the present embodiment, also include: configuration module
This configuration module is for configuring the storage strategy of storage resource, and this storage strategy includes: copy pattern, snapshot support
Pattern, compact model, cache mode.
Below in conjunction with preferred embodiment and embodiment, the present invention is described in detail.
This preferred embodiment provides a kind of ultra-large, elastic, unified storage system system and method, it is possible to fill
Divide utilization and shared client original storage resource and storage architecture, be converted into advanced person, safety, unified mass memory
Framework, meets compatibility distributed type high speed buffering, block storage, traditional data library storage, distributed structured data simultaneously and deposits
The ability of storage.
The storage system of this preferred embodiment can intelligence tackle various different internet devices storage demand, comprise biography
System equipment and novel smart machine.The present invention uses a kind of novel algorithm, it is provided that unified client Access Layer, logical
Cross unified Key Value API (KV API) interface, support the access of plurality of devices.Compatibility KV key assignments of the present invention
Data base interface, object memory interface, block interface, and file interface, can apply to publicly-owned cloud, privately owned cloud, mix
Close the plurality of application scenes such as cloud, it is possible to meet various new internet device, such as tradition PC, Novel tablet computer,
Intelligent mobile terminal etc., Intelligent internet of things equipment, Andriod equipment, the requirements for access to storage such as Fructus Mali pumilae iOS device.
Existing multiple conventional store model and storage architecture are supported in the storage system perfection of this preferred embodiment, well integrate
With utilize existing resource.Traditional storage mode is varied, but it is single generally to there is interface, the simple fraud of function
Sick, it is impossible to meet growing the Internet memory requirement.The storage device that presently, there are, some employing structuring algorithms,
Some employing destructuring algorithm organizations；Some offers mode based on file accesses, the object-based mode of some offers
Accessing, some offers access based on block storage mode, it is impossible to accomplish to store the unification of resource, share and elastic telescopic.This
Invent the distributed KV Store by bottom is unified and store data, it is provided that be expansible, highly reliable, the system of elastic telescopic
One storage resource, provides the user KV interface, object interface, block interface, file interface, it is possible to well meet each
Plant the application demand to storage.
It is big that this preferred embodiment storage system has capacity, low cost, and performance is high, the simple feature of dilatation.By to biography
The integration of system storage device and unified management, it is provided that the memory capacity of PB levels up to a hundred, hundred billion literary compositions of efficient management
Part, the quantity of documents of millions can be effectively supported in monocular record.Client application supports the storage of TERA-SCALE record, high efficient and flexible
Distributed recording data storage architecture, the data storage and query of information age explosive growth can be met.Simultaneity factor
Utilizing common hardware and general legacy system to build, support on-line rapid estimation function, holistic cost is deposited far below single tradition
Storage framework, and there is good expandability and motility.
This preferred embodiment storage system supports large-scale concurrent read-write, meets large-scale cloud storage needs.Present invention intelligence
Storage data are cut into slices, with in the formal distribution of KV to each memory node, makes the read-write operation can be in each storage
Node concurrently performs, and is greatly promoted performance.Support the tertiary storage system of internal memory/SSD/ hard disk, accelerate application to file
Access, it is provided that the most concurrent literacy.KV Store provides the high reliability of data by many copies or correcting and eleting codes,
The dynamic retractility of storage system is realized by the redistribution of data fragmentation and Data Migration.
The unified storage mode of this preferred embodiment is particularly suited for the virtualization storage part under cloud computing environment.Cloud computing
When externally providing cloud service, block memory interface can be as the local disk carry of cloud main frame, and file interface is suitable for
File-sharing service under cloud computing environment, or for storage virtual machine mirror image, and object interface and KV interface are suitable for
Third-party application under being applied to as cloud environment provides general storage platform service.Advantage of the invention is that bottom
Storage resource is unified sharing, it is not necessary to sets up for each memory interface and individually stores resource, so can be significantly
Economize on resources and subtract reduced-maintenance expense.Certainly, the present invention also can have other various embodiments, without departing substantially from the present invention
In the case of spirit and essence thereof, those of ordinary skill in the art can make various corresponding change and change according to the present invention
Shape, but these change accordingly and deform the protection domain that all should belong to appended claims of the invention.
Each example of this preferred embodiment meet can run, can manage, can safeguard, high reliability, low cost and possess
Carrier class mass memory solution open, standardized, supports KV storage, file storage, block storage, object
Storage, externally provides abundant external api interface and the file interface of standard and block interface accessing, meets various application
Different demands.
Fig. 5 is the schematic diagram storing system overall architecture according to the preferred embodiment of the invention, as shown in Figure 5, it is provided that
Storage system integrated stand composition, is divided into two layers.
One layer is the storage core of system below.By the method for software, the SSD being distributed on each host node
Dish and SAS/SATA hard disk integrate, and externally provide unified storage resource.On the basis of storage resource, application
Different memory spaces, each memory space is made up of the part or all of disk of different server node.At memory space
On the basis of, can be with the strategy of configuration storage space, including copy pattern, snapshot support, compression support, cache mode
Deng.Operating maintenance module (Operation Maintenance Module, referred to as OMM) is for system memory resources
Management, the division of memory space, the configuration of memory space, the alarm of system shows.Monitoring module (Monitor),
For monitoring the state of memory node and disk, when memory node or disk exception, produce alarm, and enter disk or
Service node troubleshooting process.
One layer is that data distribution manages system access layer above, and application can be directly by the storage of KV API Access bottom
Resource, or accessed by OAC object interface, or by BAC block interface accessing, or visited by FAC file interface
Ask memory space.
Fig. 6 is the schematic diagram of KV database store process according to the preferred embodiment of the invention, as shown in Figure 6, in detail
The thin client that describes initiates the implementation method that during data write request, the present invention proposes.Comprise the steps:
Step 601, KV API client (Client) during write data, obtains in KV data base (Store)
The hash value of key value Key, is mapped to a position on ring；
Step 602, according to the position of the hash value of Key, obtains the data place burst of correspondence, divides by searching
The routing table of sheet, finds out master (Master) memory node of all copy sets in corresponding storage territory, writes data into Master
On node；
Step 603, by this Master node, according to replication policy, writes multiple copy to its of KV Stroe data
In its memory node；
Step 604, after all nodes store successfully, returns read-write requests success message to client, completes this and reads
Write flow process.
The implementation method of corresponding KV API Client read request comprises the steps:
Step 605, when KV API Client reads data from KV Store, obtains the hash value reading Key, reflects
It is mapped to a position on ring；
Step 606, according to the position of the hash value of Key, obtains the data place burst of correspondence, divides by searching
The routing table of sheet, finds out the Master memory node of all copy sets in corresponding storage territory
Step 607, if normal by this Master node state, then reads the data of storage from the Master of copy set,
Otherwise read data from other replica node；
Step 608, after reading data, has data distribution reason system to return the data of response to application.
Fig. 7 is the schematic diagram of the internal composition of KV API according to the preferred embodiment of the invention, as it is shown in fig. 7, retouch in detail
Stating the internal composition of KV API, including the routing table of spatial lamination, it is right that key uses consistent hashing algorithm to calculate
The place burst answered, by searching routing table, finds the storage server node that burst is corresponding.Data distribution reason system
Contain the link management of bottom；The offer of api interface, mainly includes that Set writes data-interface, and Get reads data-interface,
Delete deletes data-interface；Concordance hash algorithm and the management of burst；The management etc. of routing table.KV API right and wrong
The simple interface of normal open, is such as applied to store user data etc..
Fig. 8 is the schematic diagram of distributed objects storage according to the preferred embodiment of the invention, as shown in Figure 8, describes in detail
Data storage procedure under object memory interface scene.External application passes through Rest interface accessing unified storage system.
REST Gateway carries out the protocol analysis of REST, and the data message and object information needing storage is submitted to OAC
(Object Access Client) object accesses client, OAC passes through OMDS (Object the metadata information of object
Meta Data Server) store in KV Store, generate the key information of corresponding objects data, object simultaneously
Data message is directly stored in KV Store by KV API.
Fig. 9 is the schematic diagram of distributed block storage according to the preferred embodiment of the invention, as it is shown in figure 9, describe block in detail
Data storage procedure under memory interface scene.External application passes through iSCSI interface accessing block storage system.ISCSI
After Target receives iSCSI instruction, be converted to the internal interface of BAC (Block Access Client), BAC handle
Blocks of data is converted to the KV data of correspondence, is stored in KV Store by KV API Client.Corresponding reading number
According to time, have ISCSI Target module that read request is converted to the interface of BAC of inside, BAC by KV API from
In KV Store read data return to ISCSI Target, further ISCSI Target data return to client should
With.
Figure 10 is the schematic diagram of distributed document storage according to the preferred embodiment of the invention, as shown in Figure 10, retouches in detail
State the data storage procedure under file memory interface scene.External application passes through NFS interface, or Posix, or CIFS
Interface accessing document storage system.Result NFS protocol, Posix interface, or CIFS interface are converted into the FAC of inside
(File Access Client) interface, FAC passes through FMDS (File Meta Data Server) the metadata information of file
Carrying out the organization and administration of metadata, FMDS stores file metadata information in KV Store by KV API, with
Time FMDS return the location of content key information of file, FAC again corresponding file data information by KV API
Client interface stores in KV Store.
Through the above description of the embodiments, those skilled in the art is it can be understood that arrive according to above-described embodiment
Method can add the mode of required general hardware platform by software and realize, naturally it is also possible to by hardware, but a lot
In the case of the former is more preferably embodiment.Based on such understanding, technical scheme is the most in other words to existing
The part having technology to contribute can embody with the form of software product, and this computer software product is stored in one
In storage medium (such as ROM/RAM, magnetic disc, CD), including some instructions with so that a station terminal equipment (can
To be mobile phone, computer, server, or the network equipment etc.) perform the method described in each embodiment of the present invention.
Embodiments of the invention additionally provide a kind of storage medium.Alternatively, in the present embodiment, above-mentioned storage medium can
To be arranged to storage for the program code performing above-described embodiment method step:
Alternatively, in the present embodiment, above-mentioned storage medium can include but not limited to: USB flash disk, read only memory (ROM,
Read-Only Memory), random access memory (RAM, Random Access Memory), portable hard drive,
The various medium that can store program code such as magnetic disc or CD.
Alternatively, in the present embodiment, processor performs above-described embodiment according to the program code stored in storage medium
Method.
Obviously, those skilled in the art should be understood that each module of the above-mentioned present invention or each step can be with general
Calculating device to realize, they can concentrate on single calculating device, or be distributed in multiple calculating device and formed
Network on, alternatively, they can realize, it is thus possible to by them with calculating the executable program code of device
Storage is performed by calculating device in the storage device, and in some cases, can hold with the order being different from herein
Step shown or described by row, or they are fabricated to respectively each integrated circuit modules, or by many in them
Individual module or step are fabricated to single integrated circuit module and realize.So, the present invention is not restricted to any specific hardware
Combine with software.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for the technology of this area
For personnel, the present invention can have various modifications and variations.All within the spirit and principles in the present invention, that is made is any
Amendment, equivalent, improvement etc., should be included within the scope of the present invention.
Claims (27)
1. a date storage method, it is characterised in that including:
Data distribution reason system is according to the feature of data to be stored and/or metadata, and described data distribution
The storage strategy of reason system, in the default storage resource that described storage strategy is corresponding, obtain with described feature and/
Or the storage resource that metadata is corresponding, wherein, described default storage resource is by least one or more memory node
Storage resource composition；
Described data to be stored are stored to the memory node corresponding with described storage resource.
Method the most according to claim 1, it is characterised in that described data distribution reason system also includes:
Key assignments KV application programming interface API client and/or key assignments KV interface.
Method the most according to claim 1, it is characterised in that the feature of described data to be stored and/or metadata bag
Include:
The key assignments KV of described data to be stored.
Method the most according to claim 1, it is characterised in that described data to be stored are stored and provides to described storage
In the memory node that source is corresponding, including:
Obtaining the storage resource burst corresponding with described storage resource, wherein, described default storage resource is deposited by described
Storage resource burst composition；
According to described storage resource burst in the space routing table of described default storage resource, search to be stored with described
The memory node that data are corresponding, described memory node is used for receiving the described data to be stored of write.
Method the most according to claim 4, it is characterised in that described storage strategy be concordance Hash hash calculation
In the case of method, described storage resource burst is the burst on concordance Hash hash algorithm ring, wherein, described
The corresponding described default storage resource of ring.
Method the most according to claim 1, it is characterised in that described data to be stored are stored and provides to described storage
Memory node corresponding to source includes:
By one or more copies of described data to be stored to being written in other memory node.
Method the most according to claim 1, it is characterised in that described method includes:
By key assignments KV interface, object accesses client OAC interface, block access customer end B AC interface or file
Access client FAC interface obtains described data to be stored.
Method the most according to claim 7, it is characterised in that obtaining described to be stored by described OAC interface
During data, described method also includes:
Described data distribution reason system receives application program and carries out REST by declarative state transmission REST gateway
Protocol analysis, submits to object information OAC, described OAC and obtains institute by object data server OMDS
State the data described to be stored that object information is corresponding.
Method the most according to claim 7, it is characterised in that obtaining described number to be stored by described BAC interface
According to time, described method also includes:
Described data distribution reason system is received application program and is received by small computer system interface ISCSI object
After instructing to ISCSI, being converted to the internal interface of BAC, described BAC is converted to blocks of data to treat described in correspondence
Storage data.
Method the most according to claim 7, it is characterised in that obtaining described number to be stored by described FAC interface
According to time, described method also includes:
Described data distribution reason system is received application program and is operated by NFS NFS interface, transplantation
System interface POSIX interface, or commodity ethernet file system CIFS interface be converted into inside file access client
End FAC interface, described FAC carries out metadata the metadata information of file by file data server FMDS
Organization and administration, obtain the data to be stored that described metadata information is corresponding.
11. 1 kinds of method for reading data, it is characterised in that including:
Data distribution reason system is according to the feature of data to be read and/or metadata, and described data distribution
The storage strategy of reason system, in the default storage resource that described storage strategy is corresponding, obtain with described feature and/
Or the storage resource that metadata is corresponding, wherein, described default storage resource is by by least one or more memory node
Storage resource composition；
Read the memory node that the described storage resource of described data to be read is corresponding.
12. methods according to claim 11, it is characterised in that described data distribution reason system also includes:
Key assignments KV application programming interface API client and/or key assignments KV interface.
13. methods according to claim 11, it is characterised in that the feature of described data to be read and/or metadata bag
Include:
The key assignments KV of described data to be read.
14. methods according to claim 11, it is characterised in that the described storage of the described data to be read of described reading
Memory node corresponding to resource includes:
Obtaining the storage resource burst corresponding with described storage resource, wherein, described default storage resource is deposited by described
Storage resource burst composition；
According to described storage resource burst in the space routing table of described default storage resource, search to be read with described
The memory node that data are corresponding, reads the memory node of described correspondence.
15. methods according to claim 11, it is characterised in that described storage strategy be concordance Hash hash calculation
In the case of method, described storage resource burst is the burst on concordance Hash hash algorithm ring, wherein, described
The corresponding described default memory space of ring.
16. methods according to claim 11, it is characterised in that read the described storage resource of described data to be read
Corresponding memory node includes:
Search the memory node of copy set corresponding to described data to be read, read from the memory node of described copy set
Take described data to be read.
17. methods according to claim 11, it is characterised in that described method includes:
By key assignments KV interface, object accesses client OAC interface, block access customer end B AC interface or file
Access client FAC interface reads described data to be read.
18. methods according to claim 17, it is characterised in that reading described to be read by described OAC interface
During data, described method also includes:
Described data distribution reason system reads described data to be read and returns to object data server OMDS, institute
State OMDS and described data to be read are returned to described OAC interface, and transmit REST net by declarative state
Pass carries out REST protocol analysis, returns to application program.
19. methods according to claim 17, it is characterised in that reading described to be read by described BAC interface
During data, described method also includes:
Described data distribution reason system reads described data to be read by the interface of described BAC and returns to small-sized
Computer system interface ISCSI object, described data to be read are returned to application program by described ISCSI object.
20. methods according to claim 17, it is characterised in that reading described to be read by described FAC interface
During data, described method also includes:
Described data distribution reason system reads described data to be read and returns to file data server FMDS, and
Read described data to be read by described FAC interface, then by NFS NFS interface, transplantation is grasped
Make system interface POSIX interface, or commodity ethernet file system CIFS interface returns to application program.
21. 1 kinds of data storage devices, it is characterised in that including:
First acquisition module, for data distribution reason system according to the feature of data to be stored and/or metadata,
And the storage strategy of described data distribution reason system, in the default storage resource that described storage strategy is corresponding,
Obtaining the storage resource corresponding with described feature and/or metadata, wherein, described default storage resource is by least one
Individual or the storage resource composition of multiple memory node；
Memory module, for storing described data to be stored to the memory node corresponding with described storage resource.
22. 1 kinds of digital independent devices, it is characterised in that including:
Second acquisition module, for data distribution reason system according to the feature of data to be read and/or metadata,
And the storage strategy of described data distribution reason system, in the default storage resource that described storage strategy is corresponding,
Obtaining the storage resource corresponding with described feature and/or metadata, wherein, described default storage resource is by least one
Individual or the storage resource composition of multiple memory node；
Read module, for reading the memory node that the described storage resource of described data to be read is corresponding.
23. 1 kinds of data handling systems, it is characterised in that including: key assignments KV stores data base, key assignments KV application program is compiled
Journey interface api interface,
Described key assignments KV stores data base, forms unified presetting for integrating the storage resource on each memory node
Storage resource；
Described key assignments KV application programming interface api interface, for the storage according to data or read requests,
And the key value Key application of described data presets the different memory node of memory space, wherein, described preset
Storage resource is made up of the storage resource of multiple memory nodes.
24. systems according to claim 23, it is characterised in that described key assignments KV application programming interface API connects
Mouthful function include at least one of:
Write data-interface, read data-interface, and delete the data-interface described data of process；
Described default memory space is carried out burst by concordance Hash hash algorithm, generates storage resource burst；
Manage described storage resource burst；
Manage the space routing table that described storage resource burst is corresponding.
25. systems according to claim 23, it is characterised in that described system also includes: operating maintenance module OMM,
Monitoring module,
Described OMM, is used for arranging at least one of function: the management of described storage resource, described storage provides
The division in source, the configuration of described storage resource, and the alarm of system show；
Described monitoring module, for monitoring described memory node and the state of described disk.
26. systems according to claim 23, it is characterised in that described system also includes: object accesses client
OAC interface, block access customer end B AC interface, and file access client FAC interface,
Object accesses client OAC interface, block access customer end B AC interface, or file access client FAC
Interface is for storing the data of described key assignments KV storage data base or read.
27. systems according to claim 23, it is characterised in that also include: configuration module
Described configuration module is for configuring the storage strategy of storage resource, and described storage strategy includes: copy pattern,
Snapshot supports pattern, compact model, cache mode.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510240369.4A CN106293490A (en) | 2015-05-12 | 2015-05-12 | Data storage, the method read, Apparatus and system |
| PCT/CN2016/072035 WO2016180055A1 (en) | 2015-05-12 | 2016-01-25 | Method, device and system for storing and reading data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510240369.4A CN106293490A (en) | 2015-05-12 | 2015-05-12 | Data storage, the method read, Apparatus and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106293490A true CN106293490A (en) | 2017-01-04 |
Family
ID=57248717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510240369.4A Pending CN106293490A (en) | 2015-05-12 | 2015-05-12 | Data storage, the method read, Apparatus and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106293490A (en) |
| WO (1) | WO2016180055A1 (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108415854A (en) * | 2018-02-11 | 2018-08-17 | 中国神华能源股份有限公司 | Data collecting system based on shared buffer memory and method |
| CN108920725A (en) * | 2018-08-02 | 2018-11-30 | 网宿科技股份有限公司 | A kind of method and object storage gateway of object storage |
| CN109302499A (en) * | 2018-11-30 | 2019-02-01 | 成都华栖云科技有限公司 | A kind of storage gateway of compatible access nas storage and oss storage |
| CN111796767A (en) * | 2020-06-24 | 2020-10-20 | 北京青云科技股份有限公司 | Distributed file system and data management method |
| CN111857592A (en) * | 2020-07-21 | 2020-10-30 | 星辰天合（北京）数据科技有限公司 | Data storage method and device based on object storage system and electronic equipment |
| CN111930316A (en) * | 2020-09-09 | 2020-11-13 | 上海七牛信息技术有限公司 | Cache read-write system and method for content distribution network |
| CN112839076A (en) * | 2020-12-29 | 2021-05-25 | 杭州海康威视系统技术有限公司 | Data storage method, data reading method, gateway, electronic equipment and storage medium |
| CN113010526A (en) * | 2021-04-19 | 2021-06-22 | 星辰天合（北京）数据科技有限公司 | Storage method and device based on object storage service |
| CN113296714A (en) * | 2021-06-17 | 2021-08-24 | 杭州海康威视系统技术有限公司 | Data storage system based on NAS protocol |
| CN113806125A (en) * | 2021-09-07 | 2021-12-17 | 济南浪潮数据技术有限公司 | Processing method, device and equipment for unloading volume exception and readable medium |
| CN114465770A (en) * | 2021-12-29 | 2022-05-10 | 天翼云科技有限公司 | Data processing method and related device |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109358813B (en) * | 2018-10-10 | 2022-03-04 | 郑州云海信息技术有限公司 | Capacity expansion method and device for distributed storage system |
| CN111290700B (en) * | 2018-12-10 | 2024-08-16 | 北京京东尚科信息技术有限公司 | Distributed data read-write method and system |
| CN112434015B (en) * | 2020-12-08 | 2022-08-19 | 新华三大数据技术有限公司 | Data storage method and device, electronic equipment and medium |
| WO2023125507A1 (en) * | 2021-12-29 | 2023-07-06 | 华为技术有限公司 | Method and apparatus for generating block group, and device |
| CN118377431A (en) * | 2024-05-07 | 2024-07-23 | 广州市卓航信息科技有限公司 | Computer system data storage and reading method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080059602A1 (en) * | 2006-08-31 | 2008-03-06 | Mari Matsuda | Load balancing method for data I/O paths between groups in which multi-path management is employed |
| CN101800762A (en) * | 2009-12-30 | 2010-08-11 | 中兴通讯股份有限公司 | Service cloud system for fusing multiple services and service implementation method |
| CN102132528A (en) * | 2008-09-03 | 2011-07-20 | 帝斯贝思数字信号处理和控制工程有限公司 | Communication system and interface device for a communication system |
| CN102625307A (en) * | 2011-01-31 | 2012-08-01 | 电信科学技术研究院 | Wireless network access system |
| CN102799620A (en) * | 2012-06-19 | 2012-11-28 | 中国电力科学研究院 | IEC 61850 universal database information model and design method for interface of IEC 61850 universal database information model |
| CN103218175A (en) * | 2013-04-01 | 2013-07-24 | 无锡成电科大科技发展有限公司 | Multi-tenant cloud storage platform access control system |
| CN103647712A (en) * | 2013-12-17 | 2014-03-19 | 南京联创科技集团股份有限公司 | Distributed route processing business method and distributed route processing business system |
| CN103984607A (en) * | 2013-02-08 | 2014-08-13 | 华为技术有限公司 | Distributed storage method, device and system |
| CN104281414A (en) * | 2013-07-10 | 2015-01-14 | 中兴通讯股份有限公司 | Distributed file system and small file access method thereof |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102821398B (en) * | 2011-06-09 | 2015-03-18 | 中国科学院声学研究所 | Method and system for storing sensor network data facing user multiple demands |
| CN102567495B (en) * | 2011-12-22 | 2013-08-21 | 国家电网公司 | Mass information storage system and implementation method |
| CN103019960B (en) * | 2012-12-03 | 2016-03-30 | 华为技术有限公司 | Distributed caching method and system |
-
2015
- 2015-05-12 CN CN201510240369.4A patent/CN106293490A/en active Pending
-
2016
- 2016-01-25 WO PCT/CN2016/072035 patent/WO2016180055A1/en active Application Filing
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080059602A1 (en) * | 2006-08-31 | 2008-03-06 | Mari Matsuda | Load balancing method for data I/O paths between groups in which multi-path management is employed |
| CN102132528A (en) * | 2008-09-03 | 2011-07-20 | 帝斯贝思数字信号处理和控制工程有限公司 | Communication system and interface device for a communication system |
| CN101800762A (en) * | 2009-12-30 | 2010-08-11 | 中兴通讯股份有限公司 | Service cloud system for fusing multiple services and service implementation method |
| CN102625307A (en) * | 2011-01-31 | 2012-08-01 | 电信科学技术研究院 | Wireless network access system |
| CN102799620A (en) * | 2012-06-19 | 2012-11-28 | 中国电力科学研究院 | IEC 61850 universal database information model and design method for interface of IEC 61850 universal database information model |
| CN103984607A (en) * | 2013-02-08 | 2014-08-13 | 华为技术有限公司 | Distributed storage method, device and system |
| CN103218175A (en) * | 2013-04-01 | 2013-07-24 | 无锡成电科大科技发展有限公司 | Multi-tenant cloud storage platform access control system |
| CN104281414A (en) * | 2013-07-10 | 2015-01-14 | 中兴通讯股份有限公司 | Distributed file system and small file access method thereof |
| CN103647712A (en) * | 2013-12-17 | 2014-03-19 | 南京联创科技集团股份有限公司 | Distributed route processing business method and distributed route processing business system |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108415854A (en) * | 2018-02-11 | 2018-08-17 | 中国神华能源股份有限公司 | Data collecting system based on shared buffer memory and method |
| CN108920725A (en) * | 2018-08-02 | 2018-11-30 | 网宿科技股份有限公司 | A kind of method and object storage gateway of object storage |
| CN109302499A (en) * | 2018-11-30 | 2019-02-01 | 成都华栖云科技有限公司 | A kind of storage gateway of compatible access nas storage and oss storage |
| CN111796767B (en) * | 2020-06-24 | 2023-12-08 | 北京青云科技股份有限公司 | Distributed file system and data management method |
| CN111796767A (en) * | 2020-06-24 | 2020-10-20 | 北京青云科技股份有限公司 | Distributed file system and data management method |
| CN111857592A (en) * | 2020-07-21 | 2020-10-30 | 星辰天合（北京）数据科技有限公司 | Data storage method and device based on object storage system and electronic equipment |
| CN111930316A (en) * | 2020-09-09 | 2020-11-13 | 上海七牛信息技术有限公司 | Cache read-write system and method for content distribution network |
| CN112839076A (en) * | 2020-12-29 | 2021-05-25 | 杭州海康威视系统技术有限公司 | Data storage method, data reading method, gateway, electronic equipment and storage medium |
| CN112839076B (en) * | 2020-12-29 | 2022-07-05 | 杭州海康威视系统技术有限公司 | Data storage method, data reading method, gateway, electronic equipment and storage medium |
| CN113010526A (en) * | 2021-04-19 | 2021-06-22 | 星辰天合（北京）数据科技有限公司 | Storage method and device based on object storage service |
| CN113296714A (en) * | 2021-06-17 | 2021-08-24 | 杭州海康威视系统技术有限公司 | Data storage system based on NAS protocol |
| CN113296714B (en) * | 2021-06-17 | 2022-03-04 | 杭州海康威视系统技术有限公司 | Data storage system based on NAS protocol |
| CN113806125A (en) * | 2021-09-07 | 2021-12-17 | 济南浪潮数据技术有限公司 | Processing method, device and equipment for unloading volume exception and readable medium |
| CN113806125B (en) * | 2021-09-07 | 2023-12-22 | 济南浪潮数据技术有限公司 | Method, device, equipment and readable medium for processing unloaded volume exception |
| CN114465770A (en) * | 2021-12-29 | 2022-05-10 | 天翼云科技有限公司 | Data processing method and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2016180055A1 (en) | 2016-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106293490A (en) | Data storage, the method read, Apparatus and system | |
| US10387673B2 (en) | Fully managed account level blob data encryption in a distributed storage environment | |
| US9667720B1 (en) | Shard reorganization based on dimensional description in sharded storage systems | |
| US10659225B2 (en) | Encrypting existing live unencrypted data using age-based garbage collection | |
| CN107566533B (en) | Internal and external network file sharing system based on NAS (network attached storage) | |
| CA2923068C (en) | Method and system for metadata synchronization | |
| US11287994B2 (en) | Native key-value storage enabled distributed storage system | |
| US20210019063A1 (en) | Utilizing data views to optimize secure data access in a storage system | |
| CN102651775B (en) | Based on method, the equipment and system of many tenants shared object management of cloud computing | |
| US20200174671A1 (en) | Bucket views | |
| CN105447071B (en) | Composition of content for distributed file system | |
| CN104020961B (en) | Distributed data storage method, apparatus and system | |
| CN103237046A (en) | Distributed file system supporting mixed cloud storage application and realization method thereof | |
| CN106663022A (en) | Live migration of virtual machines that use externalized memory pages | |
| US20210055885A1 (en) | Enhanced data access using composite data views | |
| US11442927B1 (en) | Storage performance-based distribution of deduplicated data to nodes within a clustered storage environment | |
| US10503693B1 (en) | Method and system for parallel file operation in distributed data storage system with mixed types of storage media | |
| Vijayakumari et al. | Comparative analysis of google file system and hadoop distributed file system | |
| CN103595799A (en) | Method for achieving distributed shared data bank | |
| Zeng et al. | Optimal metadata replications and request balancing strategy on cloud data centers | |
| CN105354250A (en) | Data storage method and device for cloud storage | |
| CN108959660A (en) | A kind of storage method and application method based on HDFS distributed file system | |
| CN117008818A (en) | Data processing method, apparatus, computer device, and computer readable storage medium | |
| CN110297869A (en) | A kind of AI Data Warehouse Platform and operating method | |
| US10387384B1 (en) | Method and system for semantic metadata compression in a two-tier storage system using copy-on-write |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170104 |