CN109254732A - A kind of reading/writing method of storing data, device and relevant device - Google Patents
A kind of reading/writing method of storing data, device and relevant device Download PDFInfo
- Publication number
- CN109254732A CN109254732A CN201811011202.0A CN201811011202A CN109254732A CN 109254732 A CN109254732 A CN 109254732A CN 201811011202 A CN201811011202 A CN 201811011202A CN 109254732 A CN109254732 A CN 109254732A
- Authority
- CN
- China
- Prior art keywords
- data
- storing data
- osd
- reading
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of reading/writing methods of storing data, receive the pending order that client is sent including OSDC；Command type is determined according to pending order；If writing commands, data to be stored is obtained according to writing commands, data to be stored is issued in predetermined quantity OSD and is saved；If reading order, read requests are obtained according to reading order, read requests are issued in predetermined quantity OSD, so that OSD extracts storing data according to read requests；It receives the data that each OSD is returned and saves information or storing data, and be back to client；The reading/writing method of the storing data can effectively reduce the transmission path of storing data relevant information, reduce bandwidth usage, the read-write efficiency of storing data is further improved, to improve the overall performance of distributed memory system；Disclosed herein as well is a kind of read-write equipment of storing data, system and computer readable storage mediums, it may have above-mentioned beneficial effect.
Description
Technical field
This application involves computer memory technical field, in particular to a kind of reading/writing method of storing data further relates to one
Read-write equipment, system and the computer readable storage medium of kind storing data.
Background technique
With the development of internet and the increase of Internet user, the data in internet also sharply expand, and generate
Data it is ten hundreds of, mass data storage is come into being as a result, wherein distributed memory system is mass data storage
An important technology in technology, it is stored data on multiple memory nodes of physical dispersion, and then to these nodes
Resource carries out unified management and distribution, and provides user and access file system interface, and distributed type assemblies make full use of distribution
Superiority, efficiently solve the storage and problem analysis of big data.Wherein, in distributed memory system the storage of data with
Reading becomes its key problem in technology.
In distributed memory system, storing data deposits in OSD (Object Storage Devices, object storage
Equipment) on cluster, client passes through OSDC (Object Storage Devices Client, object storage device client)
Realize the access to OSD cluster.Specifically, referring to FIG. 1, Fig. 1 is a kind of read-write system of storing data in the prior art
Structural schematic diagram be primarily based on client when user needs storing data for data to be stored and be sent to OSDC, then by
OSDC is sent to main OSD, be finally forwarded to by main OSD it is each from OSD, further, when each OSD completes data to be stored
After storage, information can be saved according to the transmission path backtracking of data to be stored to client by corresponding；When user needs
It is similar with the storing process of above-mentioned data to be stored when reading the storing data in distributed memory system, it is primarily based on visitor
Family end issues read requests to OSDC, is being sent to main OSD by OSDC, be finally forwarded to by main OSD it is each from OSD, into one
Step, it is each to extract corresponding storing data according to read requests from OSD, and by storing data according to the transmission of read requests
Path Yuan Lu is transmitted to client.
Storage and reading as a result, based on data in above-mentioned distributed memory system is as can be seen that either treat storage
Data are stored, or are read out to the storing data in system, corresponding information, i.e. data to be stored or reading is asked
Multiple forwarding can be undergone by asking, as shown in Figure 1, its related data information can be at most forwarded three times, so as to cause I/O path drawing
It is long, extra bandwidth is occupied, the readwrite performance decaying of data has been eventually led to, has reduced the read-write efficiency of storing data, into
One step affects the overall performance of cluster.
Therefore, the transmission path of storing data relevant information how is effectively reduced, bandwidth usage is reduced, further increases
The read-write efficiency of storing data, so that the overall performance for improving distributed memory system is that those skilled in the art are urgently to be resolved
Problem.
Summary of the invention
The purpose of the application is to provide a kind of reading/writing method of storing data, and the reading/writing method of the storing data can effectively subtract
The transmission path of few storing data relevant information, reduces bandwidth usage, further improves the read-write efficiency of storing data, from
And improve the overall performance of distributed memory system；The another object of the application is to provide a kind of read-write dress of storing data
It sets, system and computer readable storage medium, it may have above-mentioned beneficial effect.
In order to solve the above technical problems, this application provides a kind of reading/writing method of storing data, the storing data
Reading/writing method includes:
OSDC receives the pending order that client is sent；
Command type is determined according to the pending order；
When the pending order is writing commands, data to be stored is obtained according to said write order, and will be described
Data to be stored is issued in predetermined quantity OSD, so that the OSD saves the data to be stored；
When the pending order is reading order, read requests are obtained according to the reading order, and by the reading
Request is taken to be issued in the predetermined quantity OSD, so that the OSD extracts storing data according to the read requests；
It receives the data that each OSD is returned and saves information or the storing data；
The data are saved into information or the storing data is back to the client.
Preferably, after the acquisition data to be stored according to said write order, further includes:
The data to be stored is encoded, the storing data is obtained；
After the storing data for then receiving each OSD return, further includes:
The storing data is decoded, decoded storing data is obtained.
It is preferably, described that the read requests are issued in the predetermined quantity OSD, comprising:
Algorithm fractionation is carried out to the read requests, obtains the sub- read requests of the predetermined quantity；
Each sub- read requests are issued in each OSD.
Preferably, described that algorithm fractionation is carried out to the read requests, obtain the sub- read requests of the predetermined quantity, packet
It includes:
Algorithm fractionation is carried out to the read requests by EC algorithm, obtains the sub- read requests of the predetermined quantity.
Preferably, after the data preservation information for receiving each OSD return, further includes:
Information, which is saved, according to the data generates preservation information list.
Preferably, after the storing data for receiving each OSD return, further includes:
Data information list is generated according to the storing data.
In order to solve the above technical problems, being applied to OSDC, packet this application provides a kind of read-write equipment of storing data
It includes:
Order receiver module, for receiving the pending order of client transmission；
Determination type module, for determining command type according to the pending order；
Data memory module, for when it is described it is pending order be writing commands when, according to said write order obtain to
Storing data, and the data to be stored is issued in predetermined quantity OSD, so that the OSD is to the data to be stored
It is saved；
Data read module, for being read according to the reading order when the pending order is reading order
Request is taken, and the read requests are issued in the predetermined quantity OSD, so that the OSD is according to the read requests
Extract storing data；
Information receiving module saves information or the storing data for receiving the data that each OSD is returned；
Information return module, for data preservation information or the storing data to be back to the client.
Preferably, the read-write equipment of the storing data further include:
First information summarizing module generates preservation information list for saving information according to the data.
In order to solve the above technical problems, this application provides a kind of read-write system of storing data, the storing data
Read-write system includes:
Client for received pending order to be issued to OSDC, and receives the data that the OSDC is returned and saves
Information or storing data；
The OSDC, for receiving the pending order；Command type is determined according to the pending order；When described
When pending order is writing commands, data to be stored is obtained according to said write order, and the data to be stored is issued
Into predetermined quantity OSD, so that the OSD saves the data to be stored；When the pending order is reading
When order, read requests are obtained according to the reading order, and the read requests are issued to the predetermined quantity OSD
In, so that the OSD extracts storing data according to the read requests；It receives the data that each OSD is returned and saves information
Or the storing data；The data are saved into information or the storing data is back to the client；
The predetermined quantity OSD for saving the data to be stored, and data preservation information is back to described
OSDC；Or the storing data is extracted according to the read requests, and the storing data is back to the OSDC.
In order to solve the above technical problems, this application provides a kind of computer readable storage medium, it is described computer-readable
Computer program is stored on storage medium, the computer program realizes that any one of the above stores number when being executed by processor
According to reading/writing method the step of.
A kind of reading/writing method of storing data provided herein receives the pending life that client is sent including OSDC
It enables；Command type is determined according to the pending order；When the pending order is writing commands, ordered according to said write
Enable and obtain data to be stored, and the data to be stored is issued in predetermined quantity OSD so that the OSD to it is described to
Storing data is saved；When the pending order is reading order, read requests are obtained according to the reading order, and
The read requests are issued in the predetermined quantity OSD, so that the OSD extracts storage number according to the read requests
According to；It receives the data that each OSD is returned and saves information or the storing data；The data are saved into information or described are deposited
Storage data are back to the client.
As it can be seen that the reading/writing method of storing data provided herein compared with the existing technology believes storing data correlation
Breath is split and the process of forwarding is advanced to the side OSDC, i.e., is directly communicated to each relevant OSD by OSDC, realize data to be stored
Storage and system in storing data reading, remove the logic of main OSD forwarding request, and eliminate the concept of main OSD.Due to going
The process for having fallen main OSD forwarding, is directly communicated by OSDC with OSD, so the forwarding of storing data relevant information is by most three times
Become at most twice, meanwhile, either storing process or reading process, relevant information be need it is round-trip, thus, it is possible to
The transmission for regarding information as has been become four times from six times, and therefore, the reading/writing method of this kind of storing data substantially reduces storing data
The transmission path of relevant information, effectively reduces bandwidth usage, further improves the read-write efficiency of storing data, improves
The overall performance of distributed memory system.
Read-write equipment, system and the computer readable storage medium of a kind of storing data provided herein, also have
There is above-mentioned beneficial effect, details are not described herein.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of structural schematic diagram of the read-write system of storing data in the prior art；
Fig. 2 is a kind of flow diagram of the reading/writing method of storing data provided herein；
Fig. 3 is a kind of flow diagram of the reading/writing method of another storing data provided herein；
Fig. 4 is a kind of flow diagram of reading/writing method of storing data again provided herein；
Fig. 5 is a kind of structural schematic diagram of the read-write equipment of storing data provided herein；
Fig. 6 is the structural schematic diagram of the read-write equipment of another storing data provided herein；
Fig. 7 is a kind of structural schematic diagram of the read-write system of storing data provided herein.
Specific embodiment
The core of the application is to provide a kind of reading/writing method of storing data, and the reading/writing method of the storing data can effectively subtract
The transmission path of few storing data relevant information, reduces bandwidth usage, further improves the read-write efficiency of storing data, from
And improve the overall performance of distributed memory system；Another core of the application is to provide a kind of read-write dress of storing data
It sets, system and computer readable storage medium, it may have above-mentioned beneficial effect.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is
Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art
All other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Referring to FIG. 2, Fig. 2 is a kind of flow diagram of the reading/writing method of storing data provided herein, this is deposited
Storage data reading/writing method may include:
S101:OSDC receives the pending order that client is sent；
Specifically, can be based on client will when needing that the data in distributed memory system are read out or are written
Corresponding pending order is sent in OSDC, is performed corresponding processing by OSDC to pending order, such as subsequent related data
Fractionation, forwarding of information etc..
S102: command type is determined according to pending order；
Specifically, can determine its command type according to the pending order after receiving pending order, that is, determine institute
The operation that need to be executed is storing data or reads data, corresponding, and mentioned order type may include writing commands and reading life
It enables, wherein writing commands are operated corresponding to storing data, and reading order corresponds to read data operation.When pending order is
When writing commands, inside may include data that corresponding user needs to store, i.e., above-mentioned data to be stored；When pending order
When for reading order, inside may include corresponding read requests.
S103: when pending order is writing commands, data to be stored is obtained according to writing commands, and will number be stored
According to being issued in predetermined quantity OSD, so that OSD saves data to be stored；
Specifically, illustrating that user needs to be implemented storing data if the pending order received is writing commands
Therefore operation first can obtain corresponding data to be stored according to the writing commands；Further, data to be stored is issued to respectively
It is saved in a OSD.Wherein, above-mentioned OSD can be OSD, i.e., above-mentioned predetermined quantity OSD all in system, can also be with
Part OSD therein；Further, since OSD quantity is not unique, for above-mentioned data to be stored, it is issued at it
Before, it can be split, for example, average fractionation can be carried out according to the quantity of OSD, data to be stored is averagely distributed to each
In OSD；It can also be split, same type of data to be stored is issued to same according to the type of data to be stored
In OSD, different types of data to be stored is stored in different OSD, and therefore, for its specific fractionation mode, the application is not
It is specifically limited.
S104: when pending order is reading order, read requests are obtained according to reading order, and will be under read requests
It is sent in predetermined quantity OSD, so that OSD extracts storing data according to read requests；
Specifically, illustrating that user needs to be implemented if the pending order received is reading order and reading data behaviour
Make, therefore, corresponding read requests first can be obtained according to reading order, and reading order is issued in each OSD；Into one
Step, each OSD can extract the storing data of itself storage after receiving read requests according to the read requests.Likewise,
Before read requests issue, it can be split, to be further issued in each OSD.For example, it is assumed that distributed storage
In advance according to the type storing data of data in system, when needing to read image data information and lteral data information therein
When, then read requests can be split as to different types of read requests, as image data information read requests and lteral data are believed
Read requests are ceased, and the two is issued to respectively in corresponding OSD, can be obtained required storing data.
Preferably, above-mentioned that read requests are issued in predetermined quantity OSD, it may include: that read requests are calculated
Method is split, and obtains the sub- read requests of predetermined quantity；Each sub- read requests are issued in each OSD.
Specifically, this application provides a kind of method for splitting of specific read requests, it specifically can be by splitting accordingly
Algorithm is realized, i.e., carries out algorithm fractionation to the read requests received, is obtained the sub- read requests of predetermined quantity, is needed exist for
Bright, the present embodiment is to be split according to the quantity of OSD to read requests, i.e., read requests is split as predetermined quantity
A sub- read requests, the sub- read requests of the predetermined quantity and OSD are corresponded, but in practical operation operation, above-mentioned fractionation
The quantity of the sub- read requests obtained may be lower than the quantity of OSD, have no effect on the implementation of the technical program.
Preferably, above-mentioned that algorithm fractionation is carried out to read requests, the sub- read requests of predetermined quantity are obtained, may include:
Algorithm fractionation is carried out to read requests by EC (Erasure Coding, correcting and eleting codes) algorithm, predetermined quantity height is obtained and reads
Request.
Specifically, this step provides a kind of specific fractionation algorithm, i.e., above-mentioned EC algorithm.EC algorithm is before one kind to mistake
Accidentally correction technology, main application avoid the loss of packet in network transmission, are applied in distributed memory system, can have
The reliability of raising system is imitated, specifically, correcting and eleting codes can obtain higher data reliability with smaller data redudancy.
S105: the data that each OSD is returned are received and save information or storing data；
Specifically, each OSDC is after executing the step S103, it will generates when pending order is writing commands
Corresponding data save information, at this point, the data can be saved information is back to OSDC；When pending order is reading order
When, each OSDC is after executing the step S104, it will storing data needed for obtaining user is extracted, at this point, can should
Storing data is back to OSDC.
Wherein, since the quantity of OSD in distributed memory system is not unique, information is saved for above-mentioned data
Or the return course of storing data, the application can directly be returned without specifically limiting after OSD completes itself corresponding task
It returns above-mentioned data and saves information or storing data to OSDC, information time of return can also be preset, i.e., receive and write from OSD
Start timing when entering order or reading order, when reaching above- mentioned information time of return, each OSD can simultaneously be obtained itself
Data save information or storing data and be back to OSDC.Further, if OSDC is still when above- mentioned information time of return reaches
Corresponding return information is not received, at this time, then it is assumed that OSD fails when executing corresponding task, returns to S101 and restarts
?.
S106: data are saved into information or storing data is back to client.
Specifically, this step is intended to OSDC after obtaining data based on S105 and saving information or storing data, further
It is back to client to be shown, to be fed back to user.Wherein, it may include that data to be stored is being deposited that data, which save in information,
Relevant information during storage, such as the location information and its type of data to be stored；Storing data is number needed for user
According to directly being shown.The read-write capability of storing data in distributed memory system is realized as a result,.
The reading/writing method of storing data provided herein compared with the existing technology tears storing data relevant information open
Divide and the process of forwarding is advanced to the side OSDC, i.e., is directly communicated to each relevant OSD by OSDC, realize depositing for data to be stored
The reading of storing data in storage and system, removes the logic of main OSD forwarding request, and eliminates the concept of main OSD.Due to eliminating
The process of main OSD forwarding, is directly communicated by OSDC with OSD, so the forwarding of storing data relevant information from most becoming three times
At most twice, meanwhile, either storing process or reading process, relevant information are needed back and forth, thus, it is possible to regard as
The transmission of information has been become four times from six times, and therefore, the reading/writing method of this kind of storing data substantially reduces storing data correlation
The transmission path of information, effectively reduces bandwidth usage, further improves the read-write efficiency of storing data, improves distribution
The overall performance of formula storage system.
On the basis of above-mentioned each embodiment, referring to FIG. 3, Fig. 3 is that another data provided herein store
The flow diagram of method.
As a kind of preferred embodiment, after the above-mentioned acquisition data to be stored according to writing commands, can also include:
S204: encoding data to be stored, obtains storing data；
After then receiving the storing data that each OSD is returned, can also include:
S207: being decoded storing data, obtains decoded storing data.
Specifically, being obtained to be further ensured that safety of the data in transmission process when being based on writing commands in S204
After data to be stored, coded treatment further can be carried out to it, the data to be stored after being encoded, i.e., above-mentioned storing data,
And enter S205, data write-in is further realized, data is completed and saves.Wherein, for above-mentioned cataloged procedure, referring to the prior art
In any one coding techniques, the application is not construed as limiting this.
Further, it is encoded in storing process due to data to be stored, each OSD is received in S206
The storing data of return is the data after encoding, therefore can be decoded to the storing data, decoded further to obtain
Storing data.Wherein, for above-mentioned decoding process, it equally can refer to any one decoding process in the prior art, but need
Illustrate, above-mentioned cataloged procedure and decoding process should be mutual corresponding.
For the other methods part of the present embodiment, referring to a upper specific embodiment, details are not described herein by the application.
The reading/writing method of storing data provided by the embodiment of the present application, before being stored to storing data, to it
It performs the encoding operation, correspondingly, after obtaining storing data in OSD, then operation is decoded to it, obtain required storage
The safety of storing data has been effectively ensured in data.
On the basis of above-mentioned each embodiment, referring to FIG. 4, Fig. 4 is that another data provided herein store
The flow diagram of method.
As a kind of preferred embodiment, after the data that the above-mentioned each OSD of reception is returned save information, can also include:
Information, which is saved, according to data generates preservation information list.
Specifically, being checking convenient for user, each data that each OSD is returned can be saved in the form of a list
Information is shown, can specifically be summarized by the way that the data of acquisition are saved information, and corresponding preservation information list is obtained,
It saves in information list, OSD and the storing data itself saved one-to-one correspondence can be listed, easily facilitate user and carry out information
It checks, when user needs to read data, the storage location of storing data needed for being determined according to the preservation information list, more
Add the reading convenient for later period storing data.
Further, it is preferred that after the storing data that the above-mentioned each OSD of reception is returned, can also include:
Data information list is generated according to storing data.
Specifically, it is similarly checking convenient for user, each storage number that each OSD can be returned in the form of a list
According to being shown, specifically corresponding data information list can be obtained, is believed in data by summarizing the storing data of acquisition
It ceases in list, OSD and the storing data itself extracted one-to-one correspondence can be listed, easily facilitate user's progress data and check.
Wherein, for above-mentioned preservation information list or data information list, the form of missing list can be used,
Its content may include object ID, current version number, the version number newly operated, the offset length and attribute information newly operated
Deng, so as to it is subsequent carry out submission data, time-out, repeating transmission etc. operation.
For the other methods part of the present embodiment, referring to a upper specific embodiment, details are not described herein by the application.
The reading/writing method of storing data provided by the embodiment of the present application, in the form of a list in distributed memory system
Reading data information shown and stored, easily facilitate user and check and acquisition of information.
To solve the above problems, referring to FIG. 5, Fig. 5 is a kind of read-write equipment of storing data provided herein
The read-write equipment of structural schematic diagram, the storing data is applied to above-mentioned OSDC, it may include:
Order receiver module 10, for receiving the pending order of client transmission；
Determination type module 20, for determining command type according to pending order；
Data memory module 30, for obtaining number to be stored according to writing commands when pending order is writing commands
According to, and data to be stored is issued in predetermined quantity OSD, so that OSD saves data to be stored；
Data read module 40, for obtaining read requests according to reading order when pending order is reading order,
And read requests are issued in predetermined quantity OSD, so that OSD extracts storing data according to read requests；
Information receiving module 50 saves information or storing data for receiving the data that each OSD is returned；
Information return module 60, for data preservation information or storing data to be back to client.
As a kind of preferred embodiment, above-mentioned data memory module 30 can also be used to encode data to be stored, obtain
Obtain storing data；Then above-mentioned data read module 40 can also be used to be decoded storing data, obtain decoded storage number
According to.
On the basis of above-mentioned each embodiment, referring to FIG. 6, Fig. 6 is another storing data provided herein
Read-write equipment structural schematic diagram.
As a kind of preferred embodiment, the read-write equipment of the storing data can also include:
First information summarizing module 70 generates preservation information list for saving information according to data.
As a kind of preferred embodiment, the read-write equipment of the storing data can also include:
Second information summarizing module 80, for generating data information list according to storing data.
Above method embodiment is please referred to for the introduction of device provided by the present application, this will not be repeated here by the application.
To solve the above problems, referring to FIG. 7, Fig. 7 is a kind of read-write system of storing data provided herein
Structural schematic diagram, the read-write system of the storing data can include:
Client 1, for received pending order to be issued to OSDC2, and the data for receiving OSDC2 return save letter
Breath or storing data；
OSDC2, for receiving pending order；Command type is determined according to pending order；When pending order is to write
When entering order, data to be stored is obtained according to writing commands, and data to be stored is issued in predetermined quantity OSD3, so that
OSD3 saves data to be stored；When pending order is reading order, read requests are obtained according to reading order, and
Read requests are issued in predetermined quantity OSD3, so that OSD3 extracts storing data according to read requests；It receives each
The data that OSD3 is returned save information or storing data；Data are saved into information or storing data is back to client 1；
Predetermined quantity OSD3 is back to OSDC2 for saving data to be stored, and by data preservation information；Or according to
Read requests extract storing data, and storing data is back to OSDC2.
As a kind of preferred embodiment, above-mentioned OSDC2 is also used to encode data to be stored, obtains storing data；
And storing data is decoded, obtain decoded storing data.
As a kind of preferred embodiment, above-mentioned OSDC2, which is also used to save information according to data to generate, saves information list.
As a kind of preferred embodiment, above-mentioned OSDC2 is also used to generate data information list according to storing data.
Above method embodiment is please referred to for the introduction of system provided by the present application, this will not be repeated here by the application.
To solve the above problems, present invention also provides a kind of computer readable storage medium, the computer-readable storage
It is stored with computer program on medium, can be realized when computer program is executed by processor following rapid:
OSDC receives the pending order that client is sent；Command type is determined according to pending order；When pending life
When order is writing commands, data to be stored is obtained according to writing commands, and data to be stored is issued to predetermined quantity OSD
In, so that OSD saves data to be stored；When pending order is reading order, read according to reading order
Request, and read requests are issued in predetermined quantity OSD, so that OSD extracts storing data according to read requests；It receives each
The data that a OSD is returned save information or storing data；Data are saved into information or storing data is back to client.
The computer readable storage medium may include: USB flash disk, mobile hard disk, read-only memory (Read-Only
Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. is various to deposit
Store up the medium of program code.
Above method embodiment is please referred to for the introduction of computer readable storage medium provided by the present application, the application exists
This is not repeated them here.
Each embodiment is described in a progressive manner in specification, the highlights of each of the examples are with other realities
The difference of example is applied, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
Speech, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part illustration
?.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Above to the reading/writing method of storing data provided herein, device, system and computer-readable storage medium
Matter is described in detail.Specific examples are used herein to illustrate the principle and implementation manner of the present application, above
The explanation of embodiment is merely used to help understand the present processes and its core concept.It should be pointed out that for the art
Those of ordinary skill for, under the premise of not departing from the application principle, can also to the application carry out it is several improvement and repair
Decorations, these improvement and modification also fall into the protection scope element of the claim of this application.
Claims (10)
1. a kind of reading/writing method of storing data characterized by comprising
OSDC receives the pending order that client is sent；
Command type is determined according to the pending order；
When the pending order is writing commands, data to be stored is obtained according to said write order, and by described wait deposit
Data distributing is stored up into predetermined quantity OSD, so that the OSD saves the data to be stored；
When the pending order is reading order, read requests are obtained according to the reading order, and the reading is asked
It asks and is issued in the predetermined quantity OSD, so that the OSD extracts storing data according to the read requests；
It receives the data that each OSD is returned and saves information or the storing data；
The data are saved into information or the storing data is back to the client.
2. reading/writing method as described in claim 1, which is characterized in that described to obtain data to be stored according to said write order
Later, further includes:
The data to be stored is encoded, the storing data is obtained；
After the storing data for then receiving each OSD return, further includes:
The storing data is decoded, decoded storing data is obtained.
3. reading/writing method as described in claim 1, which is characterized in that described that the read requests are issued to the predetermined number
It measures in an OSD, comprising:
Algorithm fractionation is carried out to the read requests, obtains the sub- read requests of the predetermined quantity；
Each sub- read requests are issued in each OSD.
4. reading/writing method as claimed in claim 3, which is characterized in that it is described that algorithm fractionation is carried out to the read requests, it obtains
Obtain the sub- read requests of the predetermined quantity, comprising:
Algorithm fractionation is carried out to the read requests by EC algorithm, obtains the sub- read requests of the predetermined quantity.
5. the reading/writing method as described in Claims 1-4 any one, which is characterized in that receive the number that each OSD is returned
After preservation information, further includes:
Information, which is saved, according to the data generates preservation information list.
6. reading/writing method as claimed in claim 5, which is characterized in that receive the storing data that each OSD is returned
Later, further includes:
Data information list is generated according to the storing data.
7. a kind of read-write equipment of storing data, which is characterized in that be applied to OSDC, comprising:
Order receiver module, for receiving the pending order of client transmission；
Determination type module, for determining command type according to the pending order；
Data memory module, for being obtained wait store according to said write order when the pending order is writing commands
Data, and the data to be stored is issued in predetermined quantity OSD, so that the OSD carries out the data to be stored
It saves；
Data read module, for being obtained to read according to the reading order and be asked when the pending order being reading order
It asks, and the read requests is issued in the predetermined quantity OSD, so that the OSD is extracted according to the read requests
Storing data；
Information receiving module saves information or the storing data for receiving the data that each OSD is returned；
Information return module, for data preservation information or the storing data to be back to the client.
8. read-write equipment as claimed in claim 7, which is characterized in that further include:
First information summarizing module generates preservation information list for saving information according to the data.
9. a kind of read-write system of storing data characterized by comprising
Client for received pending order to be issued to OSDC, and receives the data that the OSDC is returned and saves information
Or storing data；
The OSDC, for receiving the pending order；Command type is determined according to the pending order；When described wait hold
When line command is writing commands, data to be stored is obtained according to said write order, and the data to be stored is issued to pre-
In fixed number amount OSD, so that the OSD saves the data to be stored；When the pending order is reading order
When, read requests are obtained according to the reading order, and the read requests are issued in the predetermined quantity OSD, with
The OSD is set to extract storing data according to the read requests；It receives the data that each OSD is returned and saves information or described
Storing data；The data are saved into information or the storing data is back to the client；
The predetermined quantity OSD is back to the OSDC for saving the data to be stored, and by data preservation information；
Or the storing data is extracted according to the read requests, and the storing data is back to the OSDC.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the reading of the storing data as described in claim 1 to 6 any one when the computer program is executed by processor
The step of write method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811011202.0A CN109254732A (en) | 2018-08-31 | 2018-08-31 | A kind of reading/writing method of storing data, device and relevant device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811011202.0A CN109254732A (en) | 2018-08-31 | 2018-08-31 | A kind of reading/writing method of storing data, device and relevant device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109254732A true CN109254732A (en) | 2019-01-22 |
Family
ID=65050476
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811011202.0A Pending CN109254732A (en) | 2018-08-31 | 2018-08-31 | A kind of reading/writing method of storing data, device and relevant device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109254732A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112346667A (en) * | 2021-01-07 | 2021-02-09 | 广东睿江云计算股份有限公司 | Ceph storage osd reading balancing method and system |
| CN112486413A (en) * | 2020-11-27 | 2021-03-12 | 杭州朗和科技有限公司 | Data reading method, device, medium and computing equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106375404A (en) * | 2016-08-30 | 2017-02-01 | 华为技术有限公司 | Data storage control method and apparatus, data storage method and apparatus, and data acquisition method and apparatus |
| CN106648832A (en) * | 2016-12-09 | 2017-05-10 | 武汉烽火信息集成技术有限公司 | Virtual machine resource utilization improving device and method thereof |
| US20170139840A1 (en) * | 2015-11-12 | 2017-05-18 | International Business Machines Corporation | Memory mapping for object-based storage devices |
| CN107577543A (en) * | 2017-09-18 | 2018-01-12 | 郑州云海信息技术有限公司 | Data read-write method, device, storage system and computer-readable recording medium |
-
2018
- 2018-08-31 CN CN201811011202.0A patent/CN109254732A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170139840A1 (en) * | 2015-11-12 | 2017-05-18 | International Business Machines Corporation | Memory mapping for object-based storage devices |
| CN106375404A (en) * | 2016-08-30 | 2017-02-01 | 华为技术有限公司 | Data storage control method and apparatus, data storage method and apparatus, and data acquisition method and apparatus |
| CN106648832A (en) * | 2016-12-09 | 2017-05-10 | 武汉烽火信息集成技术有限公司 | Virtual machine resource utilization improving device and method thereof |
| CN107577543A (en) * | 2017-09-18 | 2018-01-12 | 郑州云海信息技术有限公司 | Data read-write method, device, storage system and computer-readable recording medium |
Non-Patent Citations (2)
| Title |
|---|
| 云中不知人: "ceph PG故障排除", 《HTTPS://BLOG.51CTO.COM/LITTLEFIVE/1894197》 * |
| 陈小跑: "Ceph源码解析：读写流程", 《HTTPS://WWW.CNBLOGS.COM/CHENXIANPAO/P/5572859.HTML》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112486413A (en) * | 2020-11-27 | 2021-03-12 | 杭州朗和科技有限公司 | Data reading method, device, medium and computing equipment |
| CN112486413B (en) * | 2020-11-27 | 2022-08-05 | 杭州朗和科技有限公司 | Data reading method, device, medium and computing equipment |
| CN112346667A (en) * | 2021-01-07 | 2021-02-09 | 广东睿江云计算股份有限公司 | Ceph storage osd reading balancing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10152501B2 (en) | Rollover strategies in a n-bit dictionary compressed column store | |
| CN108874688B (en) | Message data caching method and device | |
| CN107643880A (en) | The method and device of file data migration based on distributed file system | |
| CN102831088A (en) | Data migration method and device based on mixing memory | |
| CN109308280A (en) | Data processing method and relevant device | |
| CN103593257B (en) | A kind of data back up method and device | |
| CN105446893A (en) | Data storage method and device | |
| US11442636B2 (en) | Method, apparatus for managing the redundant array of independent disks (RAID) and related computer readable medium | |
| CN109033359A (en) | A kind of method of multi-process secure access sqlite | |
| CN107608627B (en) | Remote data hierarchical storage method, electronic equipment and storage medium | |
| CN107623722A (en) | A kind of remote data caching method, electronic equipment and storage medium | |
| CN103150149A (en) | Method and device for processing redo data of database | |
| CN109254732A (en) | A kind of reading/writing method of storing data, device and relevant device | |
| CN107678918A (en) | The OSD heartbeat mechanisms method to set up and device of a kind of distributed file system | |
| CN105138284A (en) | Virtual machine disk mirror image synchronization operation optimization system and method | |
| CN109471843A (en) | A kind of metadata cache method, system and relevant apparatus | |
| CN105302489B (en) | A kind of remote embedded accumulator system of heterogeneous polynuclear and method | |
| CN103369038A (en) | PaaS (platform as a service) management platform and method | |
| CN111061690A (en) | RAC-based database log file reading method and device | |
| CN111007987A (en) | Memory management method, system, terminal and storage medium for raid io | |
| CN111008181A (en) | Method, system, terminal and storage medium for switching storage strategies of distributed file system | |
| CN108932158A (en) | A kind of method, apparatus and MDS server of meta data server issued transaction | |
| CN111444046B (en) | Data recovery method and distributed data recovery system | |
| CN104899158A (en) | Memory access optimization method and memory access optimization device | |
| CN109960474A (en) | Data-updating method, device, equipment and medium based on automatic simplify configuration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190122 |
|
| RJ01 | Rejection of invention patent application after publication |