CN107291889A - A kind of date storage method and system - Google Patents
A kind of date storage method and system Download PDFInfo
- Publication number
- CN107291889A CN107291889A CN201710473540.5A CN201710473540A CN107291889A CN 107291889 A CN107291889 A CN 107291889A CN 201710473540 A CN201710473540 A CN 201710473540A CN 107291889 A CN107291889 A CN 107291889A
- Authority
- CN
- China
- Prior art keywords
- data
- metadata
- storage device
- storage
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/185—Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
Abstract
The present invention provides a kind of date storage method and system, and the above method comprises the following steps：The data of receiving front-end access, create the multiple index of the data and create the metadata of the data, and the metadata is stored on multiple Metadata Service nodes；The data are write on object storage device node, for accessing, two-stage index structure design considerably reduces the metadata quantity that state monitoring and management device needs to manage, and eliminates performance bottleneck；The relation that flexible storage server grouping-hierarchy mode and group memory storage node is backed up mutually, allows it preferably to meet the extension of internet mass data system and fault-tolerant demand.
Description
Technical field
The invention belongs to computer realm, more particularly to a kind of date storage method and system.
Background technology
With the propulsion of Internet Engineering, network data scale is increasing, and network data content is constantly towards " at a high speed
The direction of change, magnanimity " is constantly advanced, and with the lifting of data volume, big data is carried out to these valuable data resources
Analysis little by little turns into inevitable, but traditional is given disk array carry by iSCSI protocol based on IP-SAN storage schemes
Front-end server does standard disk, and the centrally stored of unstructured data is carried out by the way of file system.In this pattern
Under, data flow needs that by unloading server disk array could be write, there are problems that Single Point of Faliure and.For
Internet data usually requires that 7 × 24 hours continuous services, in face of concurrently persistently writing for multiparty data, based on common file
The storage system of system constructing will produce substantial amounts of file fragmentation, can cause over time, under storage efficiency is continuous
Drop.Therefore, traditional mode can not meet internet high-performance magnanimity scale data to the high reliability of storage system, high-performance,
Demand in terms of scalability and manageability.Now with the development of memory technology, distributed cloud storage system becomes more next
More important, increasing Internet enterprises employ distributed cloud storage scheme to tackle the performance constantly increased, stability
Demand, internet data and traditional data such as mail, file, picture, tables of data etc. are compared, and have following characteristics：(1) data volume
Greatly.With being continuously increased for Internet user, the data volume that each user uploads beyond the clouds is increasing, almost can just produce within one day
Raw PB DBMS amounts, therefore the storage system of internet must is fulfilled for massive store demand, and there is the energy of linear expansion
Power.(2) write intensive.The time that the read-write of traditional data meets Pareto Law, i.e., 20% writes data, 80% time read evidence.
Internet data does not apply to this rule then, and the storage system that read-write operation has almost accounted for 100%, i.e. internet data will set
Need to pay the utmost attention to how to read and write bandwidth while improving in meter and realization.(3) 7 × 24 hours continuous services.Traditional data one
As follow access cycle, can be safeguarded system-down is offline and is upgraded normal without influenceing in the case where visit capacity is seldom
Business activity.Such cycle is not present in internet data, and video data stream continuously and uninterruptedly pours into storage system, any
Upgrade maintenance operation can not influence the storage of input data, it is necessary to dynamically carry out.Internet data has high concurrent, big
The characteristics of capacity.Universal document system such as NTFS and Ext4 etc. are designed not directed to internet data, conventional store scheme with
File carrys out tissue and stores video stream data, there are following two problems:(1) internet data is deposited with file mode it cannot be guaranteed that
Storage of the data on disk is continuous, while the multiple sources of storage, a large amount of unstructured datas are come to traditional file systemses
Say that performance has bottleneck, and as system long-play, file are continually created and deleted, substantial amounts of disk certainly will be produced
Fragment so that magnetic head needs continually to move when writing continuous video flowing, causes disk access hydraulic performance decline.(2) file point
The length of piece determines the precision of video frequency searching, and burst is too small to produce the small documents of magnanimity, cause inode number not enough.
Therefore, urgently need it is a kind of face internet data the characteristics of data storage scheme, come solve read-write efficiency with
And storage problem.
The content of the invention
The present invention provides a kind of date storage method and system, to solve the above problems.
The present invention provides a kind of date storage method.The above method comprises the following steps：
The data of receiving front-end access, create the multiple index of the data and create the metadata of the data, and
The metadata is stored on multiple Metadata Service nodes；
The data are write on object storage device node, for accessing.
The present invention also provides a kind of data-storage system, including：Index creation unit, Metadata Service node, object are deposited
Device node is stored up, wherein, the index creation unit is connected with the Metadata Service node, the object storage device node
It is connected with the Metadata Service node；
Index creation unit, the data of receiving front-end access, creates the multiple index of the data；
Metadata Service node, creates the metadata of the data, and store the metadata；
Object storage device node, for storing the data, for accessing.
Pass through following scheme：The data of receiving front-end access, create the multiple index of data and create the metadata of data,
And metadata is stored on multiple Metadata Service nodes, write data on object storage device node, for accessing.Base
The continuous of big section is changed into metadata and the mode of data storage distributed management, and by a large amount of simultaneous streaming random writes
Write, greatly improve disk write-in bandwidth, support the concurrently write-in of large-scale data stream；Secondly, using two-stage index structure
To manage data metadata.Two-stage index structure design considerably reduces the metadata number that state monitoring and management device needs to manage
Amount, eliminates performance bottleneck, and can realize the second level retrieval of internet data.
Brief description of the drawings
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, this hair
Bright schematic description and description is used to explain the present invention, does not constitute inappropriate limitation of the present invention.In the accompanying drawings：
Fig. 1 show the date storage method process chart of the embodiment of the present invention 1；
Fig. 2 show the data storage system architecture figure of the embodiment of the present invention 2；
Fig. 3 show the data storage Organization Chart of the embodiment of the present invention 3；
Fig. 4 show the data storage organization structure chart of the embodiment of the present invention 4；
Fig. 5 show the disk pool structure chart of the embodiment of the present invention 5.
Embodiment
Describe the present invention in detail below with reference to accompanying drawing and in conjunction with the embodiments.It should be noted that not conflicting
In the case of, the feature in embodiment and embodiment in the application can be mutually combined.
Fig. 1 show the date storage method process chart of the embodiment of the present invention 1, comprises the following steps：
Step 102：The data of receiving front-end access, create the multiple index of the data and create first number of the data
According to, and the metadata is stored on multiple Metadata Service nodes.
Step 104：The data are write on object storage device node, for accessing.Wherein, the multiple index can be with
It is secondary index.
Realize for convenience and two-stage rope is devised in metadata load balancing and more efficient search efficiency, the present embodiment
Guiding structure, front end data enters after storage system, while writing on multiple metadata MDS (Metadata Service node), and ensures
At least two MDS are active states, and remaining MDS can await orders for standby patterns, when carrying out data query, by master
Node is shunted to the data of required inquiry so that have the effect of load balancing when MDS is used, and is come for mass data
Say, a part of query efficiency can be improved, and then recycle the advantage of secondary index, mass data is reduced, for metadata
The time of progress parallel query is needed, so as to further improve search efficiency.
Further, the data are write on object storage device node, including：
Grouping-hierarchy strategy is stored using high-low water level method, the memory space of the object storage device is divided into difference
Storage rank；
The data are stored according to hierarchical storage management strategy in different storage ranks.
Further, when the memory space occupancy is less than watermark threshold, if system load is more than preset value, prohibit
The only shift function of the data；
If system load is less than or equal to preset value, log-on data refresh operation.
High-low water level method based on memory space, high-low water level method requirement is supervised in real time to the use space of storage device
Control, judges whether high-performance storage device space reaches certain watermark threshold.System then making according to high-performance storage device
Data migration operation is performed with situation triggering.Support read operation request data can be upgraded according to last access time (by
Secondary storage pond is migrated to one-level storage pool) mechanism：For read operation request, if the data in last access time,
Then data are upgraded, otherwise not upgraded.The mechanism reduces the frequency of moving back of data, and reducing unnecessary data and moving back causes
System loss, while also improving the service life of high-performance storage device.
Further, date storage method can also include：
Redundancy processing is done to the data using correcting and eleting codes technology；
It is K blocks by the data cutting, M check block is calculated by encryption algorithm, utilizes in (K+M) individual data block
Any K data block can recover initial data.
By to entangle delete rule setting, the size of storage overhead can be controlled, space utilisation is effectively improved, compared to
The reliability that memory space in turn ensure that data is saved for replication policy.
Therefore, after system creation success, data are accessed for front end, first at metadata end, by the basic of data message
Index information data carry out it is secondary create index, and be stored in associated metadata is scattered on two and above node so that
Data when being read in front end, solve the metadata of mass data also extremely huge the problem of drag slow reading rate in itself；First number
After creating successfully, data start on write-in cluster each osd node, and grouping-hierarchy plan is stored using high-low water level method by cluster
Slightly, memory space can be divided into different storage ranks, different deposit is placed the data according to hierarchical storage management strategy
Store up in rank, meet diverse requirements of the user to processing speed, transmission bandwidth and memory capacity, it is rationally empty using storage to reach
Between, lifting storage system access performance, reduce the purpose of general plan cost.And after data determine storage packet, write-in
Osd data can delete strategy to data in memory node distribution storage according to set in advance entangle, and the presence of strategy is deleted by entangling, is made
The data volume that must be ultimately stored on memory node can save larger memory space relative to traditional many replication policies, simultaneously
It is not again its distributed high reliability.
Fig. 2 show the data storage system architecture figure of the embodiment of the present invention 2.
As shown in Fig. 2 data-storage system a kind of according to an embodiment of the invention, including：Index creation unit 202,
Metadata Service node 204, object storage device node 206, wherein, the index creation unit 202 takes with the metadata
Business node 204 is connected, and the object storage device node 206 is connected with the Metadata Service node 204；
Index creation unit 202, the data of receiving front-end access, creates the multiple index of the data；
Metadata Service node 204, creates the metadata of the data, and store the metadata；
Object storage device node 206, for storing the data, for accessing.
Further, the object storage device node 206 is additionally operable to store grouping-hierarchy strategy using high-low water level method,
The memory space of the object storage device is divided into different storage ranks, according to hierarchical storage management strategy by the number
According to being stored in different storage ranks.
Further, the object storage device node 206 when the memory space occupancy be less than watermark threshold when,
If system load is more than preset value, forbid the shift function of the data, if system load is less than or equal to preset value, start
Data refresh operation.
Wherein, the object storage device node 206 is additionally operable to do redundancy processing to the data using correcting and eleting codes technology,
And by the data cutting be K blocks, M check block is calculated by encryption algorithm, it is any in utilization (K+M) individual data block
K data block can recover initial data.
As shown in figure 3, the MDS (Metadata Service node) comprising secondary index can be used for storage metadata and scheduling work
Make, make system that there is the effect of load balancing.Monitor needs the status information of record storage server, OSD nodes
(Object-based Storage Device, object storage device) is a group internal storage capacity, by minimum storage server magnetic
Disk determines that storage cluster total capacity is all groups in cluster of memory capacity sum.The copy implementation of the main flows such as HDFS is led to
Frequently with the mode of dynamically distributes, a file actual storage node location is uncertain, typically three backups.The system
The packet storage mode of use more flexible, operability is stronger.For example, for emphasis data, it can be specified and deposited by keeper
Storage packet.When the storage server access pressure of a packet is larger, it can add storage server lift clothes in the group
Business ability, when internet front end data increases, can increase storage group and carry out extended storage capacity, realize that system linear extends.
Storage server preserves a large amount of unstructured datas and its index information, and client possesses globally unique identifier (SID).
In order to improve storage performance, convenient that data are retrieved and managed, the system in the present embodiment utilizes AS13000
System, employs the design philosophy for being stored separately data and its metadata, as shown in Figure 4.Metadata and data are stored in not
On same disc driver, it is ensured that the read-write requests of metadata do not interfere with the read-write to data storage.
Metadata load balancing and more efficient search efficiency are realized for convenience, devise two-stage index structure, it is preceding
End data enters after storage system, while writing on multiple nodes on metadata MDS, and ensures that at least two MDS are
Active states, remaining MDS can await orders for standby patterns, when carrying out data query, by master nodes to required inquiry
Data are shunted so that have the effect of load balancing when MDS is used, for mass data, can improve a part
Query efficiency, and then the advantage of secondary index is recycled, mass data is reduced, needs to carry out parallel query for metadata
Time, so as to further improve search efficiency.
In addition, the data-storage system in the present embodiment is based on AS13000 distributed file systems, using storage packet point
Memory space, can be divided into different storage ranks, difference is placed the data according to hierarchical storage management strategy by layer strategy
Storage rank on, diverse requirements of the user to processing speed, transmission bandwidth and memory capacity are met, to reach rationally using depositing
Store up space, lifting storage system access performance, reduce the purpose of general plan cost.Also support the height water based on memory space
Position method.The requirement of high-low water level method is monitored in real time to the use space of storage device, judges that high-performance storage device space is
It is no to reach certain watermark threshold.System then performs data migration operation according to the triggering of the service condition of high-performance storage device.
The Data Migration based on object size is also supported to control.Different size of object has different transmission rates.In general, greatly
The handling capacity of object is higher than small object.If not being controlled by the processing of sized subjects in data migration process, then reaching
During certain watermark threshold, the migration of a large amount of small objects can then have a strong impact on systematic difference performance and the use perception of client.
This hierarchical storage management strategy supports the Data Migration control based on object size, and its basic feature is to specify size by limitation
The object of scope is migrated, so as to realize that data migration efficiency optimizes, lifts the application performance of hierarchical stor.
Equally, the system also supports the data intelligence shift function based on system load.After system starts, timing can be passed through
Device constantly carries out system load judgement：When high-performance memory space occupancy is also not up to watermark threshold, if system is negative
Carry high, then the automatic data migration function of system will be prohibited, stored using the overall performance for ensureing storage system as high-performance
The performance of equipment；When system load is relatively low, system automatic start data refresh operation will need to spend in data migration process
Resource consumed when system load is relatively low.It so may insure the brush of the completion data in the case of more stable system load
Newly operate and then realize that the application performance of storage system is integrally lifted.
Using grading function, different types of storage medium, such as SSD hard disks, SAS must be configured with system
Hard disk, SATA hard disc.First have to set up different disk pools during configuration grading function, to distinguish storage media types, such as Fig. 5 institutes
Show (actual shipment configuration is more than 2 nodes).
In addition, the system can also use correcting and eleting codes technology, for data are done with certain redundancy, to increase the reliable of system
Property, the data cutting that will be stored is K blocks, then calculates M check block by encryption algorithm, utilizes (K+M) individual data block
In any K can just recover initial data.By deleting the setting of rule to entangling, the size of storage overhead can be controlled, is had
Effect improves space utilisation, compared to saving the reliability that memory space in turn ensure that data for replication policy.
The problem of linear expansion is difficult to when scale rises based on IP-SAN mass data storage systems for tradition, propose
Towards mass data distributed data-storage system DMUDS (Distributed Massive Unstructured,
DataStorage).DMUDS has abandoned traditional storage mode based on file, and the two-stage index index structure that DMUDS is used is only
Need MDS to store a small amount of metadata, and reduce the parallel query time, greatly reduce MDS burden, eliminate performance bottle
Neck；Two-stage index structure provides more efficient query performance, can preferably serve upper layer application.In addition,
The relation that the flexible storage server grouping-hierarchy modes of DMUDS and group memory storage node are backed up mutually, allows it can be preferably
Meet the extension of internet mass data system and fault-tolerant demand.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the invention, for the skill of this area
For art personnel, the present invention can have various modifications and variations.Within the spirit and principles of the invention, that is made any repaiies
Change, equivalent substitution, improvement etc., should be included in the scope of the protection.
Claims (10)
1. a kind of date storage method, it is characterised in that comprise the following steps：
The data of receiving front-end access, create the multiple index of the data and create the metadata of the data, and by institute
Metadata is stated to be stored on multiple Metadata Service nodes；
The data are write on object storage device node, for accessing.
2. according to the method described in claim 1, it is characterised in that write the data on object storage device node, bag
Include：
Grouping-hierarchy strategy is stored using high-low water level method, the memory space of the object storage device is divided into different deposit
Store up rank；
The data are stored according to hierarchical storage management strategy in different storage ranks.
3. method according to claim 2, it is characterised in that when the memory space occupancy is less than watermark threshold,
If system load is more than preset value, forbid the shift function of the data；
If system load is less than or equal to preset value, log-on data refresh operation.
4. according to the method described in claim 1, it is characterised in that also include：
Redundancy processing is done to the data using correcting and eleting codes technology；
It is K blocks by the data cutting, M check block is calculated by encryption algorithm, utilizes any in (K+M) individual data block
K data block can recover initial data.
5. method according to any one of claim 1 to 4, it is characterised in that the multiple index is secondary index.
6. a kind of data-storage system, it is characterised in that including：Index creation unit, Metadata Service node, object storage are set
Slave node, wherein, the index creation unit is connected with the Metadata Service node, the object storage device node and institute
State the connection of Metadata Service node；
Index creation unit, the data of receiving front-end access, creates the multiple index of the data；
Metadata Service node, creates the metadata of the data, and store the metadata；
Object storage device node, for storing the data, for accessing.
7. system according to claim 6, it is characterised in that the object storage device node is additionally operable to use height water
Position method storage grouping-hierarchy strategy, the memory space of the object storage device is divided into different storage ranks, according to point
The data are stored in different storage ranks by level memory management policy.
8. system according to claim 7, it is characterised in that the object storage device node is when the memory space
When occupancy is less than watermark threshold, if system load is more than preset value, forbid the shift function of the data, if system load
Less than or equal to preset value, then log-on data refresh operation.
9. system according to claim 6, it is characterised in that the object storage device node is additionally operable to use correcting and eleting codes
Technology does redundancy processing to the data, and by the data cutting is K blocks, and M check block is calculated by encryption algorithm,
Any K data block in (K+M) individual data block is utilized to recover initial data.
10. the system according to any one of claim 6 to 9, it is characterised in that the multiple index is secondary index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710473540.5A CN107291889A (en) | 2017-06-20 | 2017-06-20 | A kind of date storage method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710473540.5A CN107291889A (en) | 2017-06-20 | 2017-06-20 | A kind of date storage method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107291889A true CN107291889A (en) | 2017-10-24 |
Family
ID=60097802
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710473540.5A Pending CN107291889A (en) | 2017-06-20 | 2017-06-20 | A kind of date storage method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107291889A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107943867A (en) * | 2017-11-10 | 2018-04-20 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
| CN108803542A (en) * | 2018-08-01 | 2018-11-13 | 武汉索雅信息技术有限公司 | Industrial production data manages system |
| CN108874913A (en) * | 2018-05-29 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of storage custom data storage method, system and device |
| CN109062500A (en) * | 2018-07-05 | 2018-12-21 | 北京奇艺世纪科技有限公司 | A kind of metadata management server, data-storage system and date storage method |
| CN109086364A (en) * | 2018-07-19 | 2018-12-25 | 深圳市网心科技有限公司 | Data managing method, data management apparatus and storage medium |
| CN109885256A (en) * | 2019-01-23 | 2019-06-14 | 平安科技（深圳）有限公司 | A kind of date storage method based on data fragmentation, equipment and medium |
| CN111506251A (en) * | 2019-01-30 | 2020-08-07 | 杭州海康威视系统技术有限公司 | Data processing method, data processing device, SMR storage system and storage medium |
| CN111506570A (en) * | 2020-03-05 | 2020-08-07 | 百度在线网络技术（北京）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN111684779A (en) * | 2018-02-12 | 2020-09-18 | 国际商业机器公司 | Data migration in a hierarchical storage management system |
| CN112148513A (en) * | 2019-06-28 | 2020-12-29 | 北京百度网讯科技有限公司 | Dump method and device for single point service data and storage medium |
| CN112954264A (en) * | 2019-12-10 | 2021-06-11 | 浙江宇视科技有限公司 | Platform backup protection method and device |
| CN113672175A (en) * | 2021-08-09 | 2021-11-19 | 浙江大华技术股份有限公司 | Distributed object storage method, device and equipment and computer storage medium |
| CN114189752A (en) * | 2021-11-25 | 2022-03-15 | 杭州视在数科信息技术有限公司 | Video storage scheme based on object storage service |
| CN116954523A (en) * | 2023-09-20 | 2023-10-27 | 苏州元脑智能科技有限公司 | Storage system, data storage method, data reading method and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598495A (en) * | 2013-10-31 | 2015-05-06 | 南京中兴新软件有限责任公司 | Hierarchical storage method and system based on distributed file system |
| CN105677826A (en) * | 2016-01-04 | 2016-06-15 | 博康智能网络科技股份有限公司 | Resource management method for massive unstructured data |
| CN106231252A (en) * | 2016-07-29 | 2016-12-14 | 银江股份有限公司 | A kind of extensive video monitoring storage method |
| CN106570113A (en) * | 2016-10-25 | 2017-04-19 | 中国电力科学研究院 | Cloud storage method and system for mass vector slice data |
| CN106599158A (en) * | 2016-12-07 | 2017-04-26 | 国家海洋局第二海洋研究所 | Quick query method of typhoon sea area Argo information based on space-time dual approximate index |
-
2017
- 2017-06-20 CN CN201710473540.5A patent/CN107291889A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598495A (en) * | 2013-10-31 | 2015-05-06 | 南京中兴新软件有限责任公司 | Hierarchical storage method and system based on distributed file system |
| CN105677826A (en) * | 2016-01-04 | 2016-06-15 | 博康智能网络科技股份有限公司 | Resource management method for massive unstructured data |
| CN106231252A (en) * | 2016-07-29 | 2016-12-14 | 银江股份有限公司 | A kind of extensive video monitoring storage method |
| CN106570113A (en) * | 2016-10-25 | 2017-04-19 | 中国电力科学研究院 | Cloud storage method and system for mass vector slice data |
| CN106599158A (en) * | 2016-12-07 | 2017-04-26 | 国家海洋局第二海洋研究所 | Quick query method of typhoon sea area Argo information based on space-time dual approximate index |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107943867B (en) * | 2017-11-10 | 2021-11-23 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
| CN107943867A (en) * | 2017-11-10 | 2018-04-20 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
| CN111684779B (en) * | 2018-02-12 | 2023-02-03 | 国际商业机器公司 | Data migration in a hierarchical storage management system |
| CN111684779A (en) * | 2018-02-12 | 2020-09-18 | 国际商业机器公司 | Data migration in a hierarchical storage management system |
| CN108874913A (en) * | 2018-05-29 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of storage custom data storage method, system and device |
| CN109062500A (en) * | 2018-07-05 | 2018-12-21 | 北京奇艺世纪科技有限公司 | A kind of metadata management server, data-storage system and date storage method |
| CN109062500B (en) * | 2018-07-05 | 2021-11-19 | 北京奇艺世纪科技有限公司 | Metadata management server, data storage system and data storage method |
| CN109086364A (en) * | 2018-07-19 | 2018-12-25 | 深圳市网心科技有限公司 | Data managing method, data management apparatus and storage medium |
| CN108803542B (en) * | 2018-08-01 | 2020-09-18 | 武汉索雅信息技术有限公司 | Industrial production data management system |
| CN108803542A (en) * | 2018-08-01 | 2018-11-13 | 武汉索雅信息技术有限公司 | Industrial production data manages system |
| CN109885256A (en) * | 2019-01-23 | 2019-06-14 | 平安科技（深圳）有限公司 | A kind of date storage method based on data fragmentation, equipment and medium |
| CN109885256B (en) * | 2019-01-23 | 2022-07-08 | 平安科技（深圳）有限公司 | Data storage method, device and medium based on data slicing |
| CN111506251A (en) * | 2019-01-30 | 2020-08-07 | 杭州海康威视系统技术有限公司 | Data processing method, data processing device, SMR storage system and storage medium |
| CN112148513A (en) * | 2019-06-28 | 2020-12-29 | 北京百度网讯科技有限公司 | Dump method and device for single point service data and storage medium |
| CN112148513B (en) * | 2019-06-28 | 2023-08-01 | 北京百度网讯科技有限公司 | Method, device and storage medium for dumping single-point service data |
| CN112954264A (en) * | 2019-12-10 | 2021-06-11 | 浙江宇视科技有限公司 | Platform backup protection method and device |
| CN111506570A (en) * | 2020-03-05 | 2020-08-07 | 百度在线网络技术（北京）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN113672175A (en) * | 2021-08-09 | 2021-11-19 | 浙江大华技术股份有限公司 | Distributed object storage method, device and equipment and computer storage medium |
| CN114189752A (en) * | 2021-11-25 | 2022-03-15 | 杭州视在数科信息技术有限公司 | Video storage scheme based on object storage service |
| CN116954523A (en) * | 2023-09-20 | 2023-10-27 | 苏州元脑智能科技有限公司 | Storage system, data storage method, data reading method and storage medium |
| CN116954523B (en) * | 2023-09-20 | 2024-01-26 | 苏州元脑智能科技有限公司 | Storage system, data storage method, data reading method and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107291889A (en) | A kind of date storage method and system | |
| CN107943867B (en) | High-performance hierarchical storage system supporting heterogeneous storage | |
| CN107844269A (en) | A kind of layering mixing storage system and method based on uniformity Hash | |
| CN101520805B (en) | Distributed file system and file processing method thereof | |
| CN100520696C (en) | Date storage method and data storage system | |
| US8909887B1 (en) | Selective defragmentation based on IO hot spots | |
| CN103106152B (en) | Based on the data dispatching method of level storage medium | |
| CN103635900B (en) | Time-based data partitioning | |
| CN100419664C (en) | Incremental backup operations in storage networks | |
| US8271548B2 (en) | Systems and methods for using metadata to enhance storage operations | |
| CN103761053B (en) | A kind of data processing method and device | |
| CN104317736B (en) | A kind of distributed file system multi-level buffer implementation method | |
| US20050120058A1 (en) | File management apparatus, storage management system, storage management method, program, and recording medium | |
| CN106708427A (en) | Storage method suitable for key value pair data | |
| US20060218113A1 (en) | Method and system for shredding data within a data storage subsystem | |
| CN106662981A (en) | Storage device, program, and information processing method | |
| CN102508789A (en) | Grading storage method for system | |
| CN103455577A (en) | Multi-backup nearby storage and reading method and system of cloud host mirror image file | |
| CN102136003A (en) | Large-scale distributed storage system | |
| CN1770114A (en) | Copy operations in storage networks | |
| WO2015015550A1 (en) | Computer system and control method | |
| CN103095805A (en) | Cloud storage system of data intelligent and decentralized management | |
| CN101137981A (en) | Methods and apparatus for managing the storage of content in a file system | |
| CN104025054A (en) | Dynamic storage tiering in a virtual environment | |
| CN103037004A (en) | Implement method and device of cloud storage system operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171024 |
|
| RJ01 | Rejection of invention patent application after publication |