CN108920095A - A kind of data store optimization method and apparatus based on CRUSH - Google Patents
A kind of data store optimization method and apparatus based on CRUSH Download PDFInfo
- Publication number
- CN108920095A CN108920095A CN201810573067.2A CN201810573067A CN108920095A CN 108920095 A CN108920095 A CN 108920095A CN 201810573067 A CN201810573067 A CN 201810573067A CN 108920095 A CN108920095 A CN 108920095A
- Authority
- CN
- China
- Prior art keywords
- data
- crush
- target data
- target
- write operation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The data store optimization method based on CRUSH that the present invention provides a kind of, including：Classification storage is carried out to multiple target datas according to data characteristics；According to type belonging to the target data, CRUSH strategy is adjusted, write operation mapping relations are optimized；According to the write operation mapping relations after CRUSH strategy adjusted and optimization, by multiple target datas storage to corresponding disk；Data are distributed more balancedly thus, it is possible to realize, are operated to reduce the end client thread frequent switching and memory copying etc. during storage object, to reach optimization I/O operation, are greatly improved the purpose of disk writing speed.
Description
Technical field
The present invention relates to computer memory technical field more particularly to a kind of data store optimization methods based on CRUSH
And device.
Background technique
Nowadays, mobile Internet is in the prosperity period of great prosperity, has driven cloud computing, Internet of Things (IoT), artificial intelligence
(AI) etc. the rapid development of new industries, consequent is the generation of magnanimity unstructured data, and is being in explosive growth.
Such as shopping website, video website and immediate communication tool etc. that current people often access, rear end store and save sea
Video, picture resource and the access of user record log of amount etc..Traditional technical solution is usually more services
Device connects the storage of FC optical fiber (shared storage-type, and non-distributed) by SAN exchange network simultaneously, then externally provides service.
Using traditional technical solution, there are multiple drawbacks, as hardware cost expense is excessively high；The SAN of the multiple brands of maintenance management stores net
Network has difficulties；There may be Single Point of Faliures for hardware store；Storing I/O, there are bottlenecks etc..Because of any operation of terminal user
The I/O operation stored to bottom is all referred to, when number of users is more and more, when concurrent increasing, I/O operation will become entire
Maximum performance bottleneck in system architecture.How more cheap vast capacity storage is provided, and to support high-performance, Gao Ke
With and the distributed memory system of lateral dynamic expansion be current urgent need to resolve the problem of.
Ceph file system provides the file system service compatible with POSIX, is based on Ceph object storage cluster,
File in it is mapped to the object in Ceph storage cluster.Client can this it is file system mounted in kernel objects or
In user's space file system.
Interface is locally stored in order to support affairs in Ceph, introduces log (Journal) mechanism.All write operations are all
First record log is needed, then local file system is written, then writes PG Log.It is in simple terms exactly that a data require to write two
Time, log and local file system, however local file system such as ext3/ext4/xfs default is also all to write log.This
It results in the case where extensive continuous I/O operation, practical magnetic disc i/o show very high, but performance performance is generally.Because having
The I/O of half is to write journalizing.In addition, Ceph has that I/O path is tediously long during write operation, it is with osd
Example, an I/O need by message queue, OSDs, FileJournal, the multiple modules of interface are locally stored could complete, each
Queue and thread switching are directed between module, part of module will also carry out memory copying when handling IO, cause
Overall performance is not high.
Summary of the invention
The present invention needs frequent record log and crush algorithm policy to cause for existing based on Ceph distributed storage
The lower defect of magnetic disc i/o utilization rate provides a kind of data store optimization method and apparatus based on CRUSH, realizes more equal
Distribution data in weighing apparatus ground operate to reduce the end client thread frequent switching and memory copying etc. during storage object,
To reach optimization I/O operation, the purpose of disk writing speed is greatly improved.
On the one hand, the embodiment of the present invention provides a kind of data store optimization method based on CRUSH, includes the following steps：
Classification storage is carried out to multiple target datas according to data characteristics；
According to type belonging to the target data, CRUSH strategy is adjusted；
According to type belonging to the target data, optimize write operation mapping relations；
According to the write operation mapping relations after CRUSH strategy adjusted and optimization, by multiple target datas storage to phase
The disk answered.
In the data store optimization method of the invention based on CRUSH, according to data characteristics to multiple target datas
In the step for carrying out classification storage, if the target data is to have the data of strong consistency requirement to affairs, by institute
It states target data and is identified as first kind data；If the daily record data that the target data needs to record is less than preset value,
The target data is identified as Second Type data；If the target data does not need to write log, by the number of targets
According to being identified as third categorical data.
In the data store optimization method of the invention based on CRUSH, in the type according to belonging to the target data,
In the step for optimizing write operation mapping relations, if the target data is first kind data, write operation is recorded
It is saved in log recording；If the target data is Second Type data, write operation record is saved in log delay
In write-in record；If the target data is third categorical data, write operation record is updated onto Object.
In the data store optimization method of the invention based on CRUSH, in the type according to belonging to the target data,
In the step for adjusting CRUSH strategy, same type of target data is put into specified region, and according to the number of targets
According to purposes to specify copy set number and shelf position with important level.
Correspondingly, the present invention also provides a kind of data store optimization device based on CRUSH, including：
Categorization module, for carrying out classification storage to multiple target datas according to data characteristics；
Module is adjusted, for the type according to belonging to the target data, adjusts CRUSH strategy；
Optimization module optimizes write operation mapping relations for the type according to belonging to the target data；
Memory module, for the write operation mapping relations according to CRUSH strategy adjusted and after optimizing, by multiple targets
Data are stored to corresponding disk.
In the data store optimization device of the invention based on CRUSH, if the target data is to have by force to affairs
The data of coherence request, then the target data is identified as first kind data by the categorization module；If the target
The daily record data that data need to record is less than preset value, then the target data is identified as Second Type number by the categorization module
According to；If the target data does not need to write log, the target data is identified as third number of types by the categorization module
According to.
In the data store optimization device of the invention based on CRUSH, if the target data is first kind number
According to then write operation record is saved in log recording by the optimization module；If the target data is Second Type data,
Then write operation record is saved in log delay write-in record by the optimization module；If the target data is third type
Data, then the optimization module updates write operation record onto Object.
In the data store optimization device of the invention based on CRUSH, the adjustment module is by same type of target
Data are put into specified region, and according to the purposes of the target data to specify copy set number and rack with important level
Position.
Implement the embodiment of the present invention, has the advantages that：Data store optimization provided by the invention based on CRUSH
Method carries out classification storage to multiple target datas according to data characteristics；According to type belonging to the target data, adjustment
CRUSH strategy, optimizes write operation mapping relations；It, will according to the write operation mapping relations after CRUSH strategy adjusted and optimization
Multiple target datas are stored to corresponding disk；Data are distributed more balancedly thus, it is possible to realize, to reduce the end client
Thread frequent switching and memory copying etc. operate during storage object, to reach optimization I/O operation, greatly improve
The purpose of disk writing speed.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow chart of the data store optimization method provided in an embodiment of the present invention based on CRUSH；
Fig. 2 is the schematic diagram of the data store optimization device provided in an embodiment of the present invention based on CRUSH；
Fig. 3 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
The embodiment of the present application solves existing suspension mode and operates speed by providing a kind of method for connecting network and device
Degree is slow, complex for operation step；It is horizontal that operation accuracy rate is completely dependent on personnel's operation；It is not available the technology that mobile terminal is operated
Problem realizes the automation of the network connection control of internal network service system, mitigates operator's burden, improves whole effect
Rate adapts to the internet of network security quick and precisely connection requirement.
In order to make those skilled in the art more fully understand the present invention program, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
Description and claims of this specification and term " first ", " second ", " third " and " in above-mentioned attached drawing
Four " etc. are not use to describe a particular order for distinguishing different objects.In addition, term " includes " and " having " and it
Any deformation, it is intended that cover and non-exclusive include.Such as it contains the process, method of a series of steps or units, be
System, product or equipment are not limited to listed step or unit, but optionally further comprising the step of not listing or list
Member, or optionally further comprising other step or units intrinsic for these process, methods, product or equipment.
Below in conjunction with drawings and examples the present invention will be further explained explanation.
In the first embodiment of the data store optimization method of the invention based on CRUSH, according to data storage features,
It will require lower data to log rank is write (for example, being required without affairs, atomicity or using non-critical datas such as logs)
It can record a small amount of log or closing journal writing function；In addition, improving the algorithmic rule of Crush in Ceph, adjustment object is deposited
Put region, realization distributes data more balancedly, thus reduce the end client during storage object thread frequent switching with
And the operation such as memory copying, to reach optimization I/O operation, greatly improve disk writing speed.
As shown in Figure 1, the described method comprises the following steps：
Step S11, classification storage is carried out to multiple target datas according to data characteristics；
Step S12, the type according to belonging to the target data, adjustment CRUSH strategy；
Step S13, the type according to belonging to the target data optimizes write operation mapping relations；
Step S14, according to the write operation mapping relations after CRUSH strategy adjusted and optimization, by multiple target datas
Store corresponding disk.
In step s 11, the process of an object is saved according to Ceph, we can be first enterprising in the data characteristics of storage
The data (no affairs, atomicity requirement) that do not need to write log can only be recorded a small amount of log or closing journal note by line identifier
Recording function.Specifically, if the target data, which is, the data of strong consistency requirement to affairs, by the target data mark
Knowing is first kind data；If the daily record data that the target data needs to record is less than preset value, by the number of targets
According to being identified as Second Type data；If the target data does not need to write log, the target data is identified as third
Categorical data.
In step s 12, same type of target data is put into specified region, and according to the use of the target data
Way is to specify copy set number and shelf position with important level.By adjusting CRUSH strategy, Ke Yiti according to data type
For the harmony of data distribution.
In step s 13, if the target data is first kind data, write operation record is saved in log note
In record；If the target data is Second Type data, write operation record is saved in log delay write-in record；Such as
Target data described in fruit is third categorical data, then updates write operation record onto Object.Specifically, have to affairs strong by one
The first kind data (for example, transaction journal class data) that cause property requires map process File- by the default of ceph conservation object
>FileJournal->Object->PG->OSD->Disk completes storage；The second class to the data that need to only record a small amount of log
Type data (for example, the data such as certain operation notes of user) allow FileJournal write latency and do not influence to return the result,
Then press File->Delay write FileJournal->Object->PG->OSD->Disk mapping relations complete storage；To not
The third categorical data (such as nonessential application log) for needing to write the data (no affairs, atomicity requirement) of log, can
Closing journal writing function does not need for write operation record to be saved on the Journal of OSD, directly presses File->Object->
PG->OSD->The mapping relations of Disk complete object storage.
In step S14, by the write operation mapping relations after CRUSH strategy adjusted above and optimization, by target
Disk corresponding with its type is written in data.
Fig. 2 is the functional block diagram of the first embodiment of the data store optimization device of the invention based on CRUSH, such as Fig. 2
Shown, in the present embodiment, the data store optimization device 200 based on CRUSH includes：
Categorization module 110, for carrying out classification storage to multiple target datas according to data characteristics；
Module 120 is adjusted, for the type according to belonging to the target data, adjusts CRUSH strategy；
Optimization module 130 optimizes write operation mapping relations for the type according to belonging to the target data；
Memory module 140 will be multiple for the write operation mapping relations according to CRUSH strategy adjusted and after optimizing
Target data is stored to corresponding disk.
Specifically, there are the data of strong consistency requirement to affairs if the target data is, the categorization module will
The target data is identified as first kind data；If the daily record data that the target data needs to record is less than preset value,
Then the target data is identified as Second Type data by the categorization module；If the target data does not need to write log,
Then the target data is identified as third categorical data by the categorization module.
Specifically, if the target data is first kind data, write operation is recorded and is saved by the optimization module
Into log recording；If the target data is Second Type data, write operation record is saved in by the optimization module
In log delay write-in record；If the target data is third categorical data, the optimization module records write operation
It updates on Object.
Specifically, same type of target data is put into specified region by the adjustment module, and according to the target
The purposes of data is to specify copy set number and shelf position with important level.
In the first embodiment of the data store optimization device of the invention based on CRUSH, according to data storage features,
It will require lower data to log rank is write (for example, being required without affairs, atomicity or using non-critical datas such as logs)
It can record a small amount of log or closing journal writing function；In addition, improving the algorithmic rule of Crush in Ceph, adjustment object is deposited
Put region, realization distributes data more balancedly, thus reduce the end client during storage object thread frequent switching with
And the operation such as memory copying, to reach optimization I/O operation, greatly improve disk writing speed.
About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method
Embodiment in be described in detail, no detailed explanation will be given here.
Referring to Fig. 3, Fig. 3 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention, the electronic equipment
300, may include：
Processor 310, memory 320, communication interface 330 and bus 340；
The processor 310, the memory 320 and the communication interface 330 are connected and are completed by the bus 340
Mutual communication；
The memory 320 stores executable program code；
The processor 310 run by reading the executable program code stored in the memory 320 with it is described
The corresponding program of executable program code, for executing a kind of method for connecting network；Wherein, a kind of method for connecting network, packet
Include following steps：
Classification storage is carried out to multiple target datas according to data characteristics；
According to type belonging to the target data, CRUSH strategy is adjusted；
According to type belonging to the target data, optimize write operation mapping relations；
According to the write operation mapping relations after CRUSH strategy adjusted and optimization, by multiple target datas storage to phase
The disk answered.
As can be seen that using technical solution provided in an embodiment of the present invention, it is possible to reduce the end client is in storage object mistake
Thread frequent switching and memory copying etc. operate in journey, to reach optimization I/O operation, greatly improve disk writing speed
Purpose.
The embodiment of the invention also provides a kind of storage mediums, wherein the storage medium is described for storing application program
Application program for executing a kind of data store optimization method based on CRUSH described in the embodiment of the present invention at runtime.
The embodiment of the invention also provides a kind of application programs, wherein the application program for executing this hair at runtime
A kind of data store optimization method based on CRUSH described in bright embodiment.
It should be noted that for the various method embodiments described above, for simple description, therefore, it is stated as a series of
Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described because
According to the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know
It knows, the embodiments described in the specification are all preferred embodiments, and related actions and modules is not necessarily of the invention
It is necessary.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment
Point, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed device, it can be by another way
It realizes.For example, the apparatus embodiments described above are merely exemplary, such as the division of the unit, it is only a kind of
Logical function partition, there may be another division manner in actual implementation, such as multiple units or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be through some interfaces, the indirect coupling or communication connection of device or unit,
It can be electrical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes：USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program code
Medium.
The above, above embodiments are only to illustrate technical solution of the present invention, rather than its limitations；Although referring to aforementioned
Invention is explained in detail for embodiment, wherein those skilled in the art should understand that：It still can be right
Technical solution documented by foregoing embodiments is modified or equivalent replacement of some of the technical features；And this
It modifies or replaces, the range for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should be included within scope of the presently claimed invention.
Claims (8)
1. a kind of data store optimization method based on CRUSH, which is characterized in that include the following steps：
Classification storage is carried out to multiple target datas according to data characteristics；
According to type belonging to the target data, CRUSH strategy is adjusted；
According to type belonging to the target data, optimize write operation mapping relations；
According to the write operation mapping relations after CRUSH strategy adjusted and optimization, multiple target datas are stored to corresponding
Disk.
2. the data store optimization method according to claim 1 based on CRUSH, which is characterized in that according to data
Feature carries out multiple target datas in the step of classification storage, if the target data is to have strong consistency to affairs
It is required that data, then the target data is identified as first kind data；If the target data needs the log recorded
Data are less than preset value, then the target data are identified as Second Type data；If the target data does not need to write day
The target data is then identified as third categorical data by will.
3. the data store optimization method according to claim 2 based on CRUSH, which is characterized in that according to
Type belonging to target data, optimize write operation mapping relations the step in, if the target data be the first kind
Write operation record is then saved in log recording by data；If the target data is Second Type data, by write operation
Record is saved in log delay write-in record；If the target data is third categorical data, more by write operation record
Newly on Object.
4. the data store optimization method according to claim 1 based on CRUSH, which is characterized in that according to
Type belonging to target data, adjust CRUSH strategy the step in, same type of target data is put into specified area
Domain, and according to the purposes of the target data to specify copy set number and shelf position with important level.
5. a kind of data store optimization device based on CRUSH, which is characterized in that including：
Categorization module, for carrying out classification storage to multiple target datas according to data characteristics；
Module is adjusted, for the type according to belonging to the target data, adjusts CRUSH strategy；
Optimization module optimizes write operation mapping relations for the type according to belonging to the target data；
Memory module, for the write operation mapping relations according to CRUSH strategy adjusted and after optimizing, by multiple target datas
Store corresponding disk.
6. the data store optimization device according to claim 5 based on CRUSH, which is characterized in that if the mesh
Marking data is to have the data of strong consistency requirement to affairs, then the target data is identified as the first kind by the categorization module
Data；If the daily record data that the target data needs to record is less than preset value, the categorization module is by the number of targets
According to being identified as Second Type data；If the target data does not need to write log, the categorization module is by the number of targets
According to being identified as third categorical data.
7. the data store optimization device according to claim 6 based on CRUSH, which is characterized in that if the mesh
Mark data are first kind data, then write operation record is saved in log recording by the optimization module；If the target
Data are Second Type data, then write operation record is saved in log delay write-in record by the optimization module；If institute
Stating target data is third categorical data, then the optimization module updates write operation record onto Object.
8. the data store optimization device according to claim 5 based on CRUSH, which is characterized in that the adjustment mould
Same type of target data is put into specified region by block, and according to the purposes of the target data with specified with important level
Copy set number and shelf position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810573067.2A CN108920095B (en) | 2018-06-06 | 2018-06-06 | Data storage optimization method and device based on CRUSH |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810573067.2A CN108920095B (en) | 2018-06-06 | 2018-06-06 | Data storage optimization method and device based on CRUSH |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108920095A true CN108920095A (en) | 2018-11-30 |
| CN108920095B CN108920095B (en) | 2021-06-29 |
Family
ID=64418413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810573067.2A Active CN108920095B (en) | 2018-06-06 | 2018-06-06 | Data storage optimization method and device based on CRUSH |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108920095B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110119251A (en) * | 2019-05-14 | 2019-08-13 | 北京杰思安全科技有限公司 | A kind of buffer control method and system based on disk big flow |
| CN111026337A (en) * | 2019-12-30 | 2020-04-17 | 中科星图股份有限公司 | Distributed storage method based on machine learning and ceph thought |
| CN112379825A (en) * | 2019-09-24 | 2021-02-19 | 北京城建设计发展集团股份有限公司 | Distributed data storage method and device based on data feature sub-pools |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102938784A (en) * | 2012-11-06 | 2013-02-20 | 无锡江南计算技术研究所 | Method and system used for data storage and used in distributed storage system |
| CN103064761A (en) * | 2012-12-24 | 2013-04-24 | 华为技术有限公司 | Data synchronization method, device and system |
| CN105468297A (en) * | 2015-11-18 | 2016-04-06 | 临沂大学 | Quick synchronization method for master and slave device data in cloud storage system |
| CN105573679A (en) * | 2015-12-18 | 2016-05-11 | 国云科技股份有限公司 | Method suitable for storage pool resource mapping rule of distributed storage system |
| CN106033322A (en) * | 2015-03-17 | 2016-10-19 | 北京元心科技有限公司 | Method and device for data storage |
| CN106502587A (en) * | 2016-10-19 | 2017-03-15 | 华为技术有限公司 | Data in magnetic disk management method and magnetic disk control unit |
| CN106533967A (en) * | 2016-12-08 | 2017-03-22 | 北京中安智达科技有限公司 | Data transmission method capable of customizing load balance strategy |
| CN106708427A (en) * | 2016-11-17 | 2017-05-24 | 华中科技大学 | Storage method suitable for key value pair data |
| CN107203341A (en) * | 2017-05-23 | 2017-09-26 | 建荣半导体（深圳）有限公司 | Date storage method, device and flash chip based on flash memory |
| CN107453948A (en) * | 2017-07-28 | 2017-12-08 | 北京邮电大学 | The storage method and system of a kind of network measurement data |
| CN107491529A (en) * | 2017-08-18 | 2017-12-19 | 华为技术有限公司 | A kind of snapshot delet method and node |
| CN107979490A (en) * | 2017-11-17 | 2018-05-01 | 北京联想超融合科技有限公司 | The recording method of daily record data and server cluster |
-
2018
- 2018-06-06 CN CN201810573067.2A patent/CN108920095B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102938784A (en) * | 2012-11-06 | 2013-02-20 | 无锡江南计算技术研究所 | Method and system used for data storage and used in distributed storage system |
| CN103064761A (en) * | 2012-12-24 | 2013-04-24 | 华为技术有限公司 | Data synchronization method, device and system |
| CN106033322A (en) * | 2015-03-17 | 2016-10-19 | 北京元心科技有限公司 | Method and device for data storage |
| CN105468297A (en) * | 2015-11-18 | 2016-04-06 | 临沂大学 | Quick synchronization method for master and slave device data in cloud storage system |
| CN105573679A (en) * | 2015-12-18 | 2016-05-11 | 国云科技股份有限公司 | Method suitable for storage pool resource mapping rule of distributed storage system |
| CN106502587A (en) * | 2016-10-19 | 2017-03-15 | 华为技术有限公司 | Data in magnetic disk management method and magnetic disk control unit |
| CN106708427A (en) * | 2016-11-17 | 2017-05-24 | 华中科技大学 | Storage method suitable for key value pair data |
| CN106533967A (en) * | 2016-12-08 | 2017-03-22 | 北京中安智达科技有限公司 | Data transmission method capable of customizing load balance strategy |
| CN107203341A (en) * | 2017-05-23 | 2017-09-26 | 建荣半导体（深圳）有限公司 | Date storage method, device and flash chip based on flash memory |
| CN107453948A (en) * | 2017-07-28 | 2017-12-08 | 北京邮电大学 | The storage method and system of a kind of network measurement data |
| CN107491529A (en) * | 2017-08-18 | 2017-12-19 | 华为技术有限公司 | A kind of snapshot delet method and node |
| CN107979490A (en) * | 2017-11-17 | 2018-05-01 | 北京联想超融合科技有限公司 | The recording method of daily record data and server cluster |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110119251A (en) * | 2019-05-14 | 2019-08-13 | 北京杰思安全科技有限公司 | A kind of buffer control method and system based on disk big flow |
| CN110119251B (en) * | 2019-05-14 | 2023-01-03 | 北京杰思安全科技有限公司 | Buffer control method and system based on disk large flow |
| CN112379825A (en) * | 2019-09-24 | 2021-02-19 | 北京城建设计发展集团股份有限公司 | Distributed data storage method and device based on data feature sub-pools |
| CN112379825B (en) * | 2019-09-24 | 2021-07-06 | 北京城建设计发展集团股份有限公司 | Distributed data storage method and device based on data feature sub-pools |
| CN111026337A (en) * | 2019-12-30 | 2020-04-17 | 中科星图股份有限公司 | Distributed storage method based on machine learning and ceph thought |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108920095B (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105144121B (en) | Cache content addressable data block is for Storage Virtualization | |
| US10579272B2 (en) | Workload aware storage platform | |
| US9460008B1 (en) | Efficient garbage collection for a log-structured data store | |
| US9342574B2 (en) | Distributed storage system and distributed storage method | |
| US10482062B1 (en) | Independent evictions from datastore accelerator fleet nodes | |
| US10853193B2 (en) | Database system recovery using non-volatile system memory | |
| CN104111804A (en) | Distributed file system | |
| CN103139300A (en) | Virtual machine image management optimization method based on data de-duplication | |
| CN106202416B (en) | Table data write method and device, table data read method and device | |
| CN109697016A (en) | Method and apparatus for improving the storage performance of container | |
| CN104270412A (en) | Three-level caching method based on Hadoop distributed file system | |
| CN108920095A (en) | A kind of data store optimization method and apparatus based on CRUSH | |
| CN109446222A (en) | A kind of date storage method of Double buffer, device and storage medium | |
| US11080207B2 (en) | Caching framework for big-data engines in the cloud | |
| CN111309266B (en) | Distributed storage metadata system log optimization system and method based on ceph | |
| CN109767274B (en) | Method and system for carrying out associated storage on massive invoice data | |
| CN109359063A (en) | Caching replacement method, storage equipment and storage medium towards storage system software | |
| CN107864215B (en) | Peer-to-peer network file system, access control/management method/system, and terminal | |
| CN103559247B (en) | A kind of data service handling method and device | |
| CN117608856A (en) | Memory expansion method, system, terminal and storage medium for NVMe (network video Me) acceleration card | |
| WO2024021470A1 (en) | Cross-region data scheduling method and apparatus, device, and storage medium | |
| CN113343045B (en) | Data caching method and network equipment | |
| US11341163B1 (en) | Multi-level replication filtering for a distributed database | |
| WO2019143967A1 (en) | Methods for automated artifact storage management and devices thereof | |
| US11042665B2 (en) | Data connectors in large scale processing clusters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: 2519 Dongfang Science and Technology Building, No. 16 Keyuan Road, Science and Technology Park Community, Yuehai Street, Nanshan District, Shenzhen, Guangdong Province, 518000 Patentee after: SHENZHEN MASLONG INFORMATION TECHNOLOGY Co.,Ltd. Address before: 518000 Room 502, building 8, Shenzhen Software Park, Central District, high tech Zone, Shenzhen City, Guangdong Province Patentee before: SHENZHEN MASLONG INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder |