CN108874310A - The read buffer method, apparatus and computer readable storage medium of distributed storage - Google Patents
The read buffer method, apparatus and computer readable storage medium of distributed storage Download PDFInfo
- Publication number
- CN108874310A CN108874310A CN201810523394.7A CN201810523394A CN108874310A CN 108874310 A CN108874310 A CN 108874310A CN 201810523394 A CN201810523394 A CN 201810523394A CN 108874310 A CN108874310 A CN 108874310A
- Authority
- CN
- China
- Prior art keywords
- read
- offset
- reading
- read request
- last time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the read buffer method, apparatus and computer readable storage medium of a kind of distributed storage, this method includes：After obtaining this read request, extracts this reading offset of the corresponding file of this read request and record；If the corresponding file of this read request file corresponding with last time read request is identical, judge that this reads whether offset read in the corresponding threshold range of offset in last time；If so, initiating to pre-read；The present invention is by judging that this reads offset whether in the corresponding threshold range of last time reading offset, the Rule of judgment that triggering is pre-read is expanded, it can normally be pre-read when there is the not exclusively scene of sequence reading, enhance the stability of front end reading performance, improve user experience.
Description
Technical field
The present invention relates to distributed storage application field, in particular to the read buffer method, apparatus of a kind of distributed storage
And computer readable storage medium.
Background technique
With the development of modern society's science and technology, the distribution that data dispersion is stored in more independent equipment based on network
Formula storage system is widely used.In distributed memory system, in order to improve the readwrite performance of front-end client, it is
The concept that caching can be introduced when system design marks off one in memory piece of region and is used to accommodate the read-write requests data issued,
In particular for read operation, the data that storage is pre-read from rear end can be improved cache hit rate, greatly improve reading performance.
In the prior art, read buffer mechanism traditional in distributed memory system is designed only for common situation, if reading
If file is not exclusively read in sequence, then pre-reading will fail, and influence front end reading performance, be unfavorable for user experience.Therefore,
How existing read buffer mechanism is improved, enhance the stability of front end reading performance, improve user experience, is existing
Modern urgent problem.
Summary of the invention
The object of the present invention is to provide the read buffer method, apparatus and computer-readable storage medium of a kind of distributed storage
Matter enhances the stability of front end reading performance on the basis of current ahead mechanism, to expand the Rule of judgment that triggering is pre-read,
Improve user experience.
In order to solve the above technical problems, the present invention provides a kind of read buffer method of distributed storage, including：
After obtaining this read request, extracts this reading offset of the corresponding file of this described read request and record；
If the corresponding file of described this read request file corresponding with last time read request is identical, this described reading of judgement
Whether offset read in the corresponding threshold range of offset in last time；
If so, initiating to pre-read.
Optionally, after described this read request of acquisition, further include：
It extracts this reading length of the corresponding file of this described read request and records.
Optionally, whether this reads offset in the corresponding threshold range of last time reading offset described in the judgement
It is interior, including：
Described this of judgement reads the difference whether offset is more than or equal to last time reading offset and first threshold, and is less than
Offset was read equal to the last time, last time reads the sum of length and second threshold；
If so, executing the step of initiation is pre-read.
Optionally, after extracting the preceding reading offset and this reading length and recording, further include：
It, will this described reading if the corresponding file of described this read request file corresponding from the last time read request is different
Request is sent to rear end storage, to obtain the corresponding data of this described read request.
Optionally, this method further includes：
This reads offset according to record and described this reads length, calculate generate or update it is described this
The reading code stream of the corresponding file of read request；
According to the reading for reading code stream All Files corresponding with whole read requests of the corresponding file of this described read request
The ratio of code fetch stream is that the corresponding file of this described read request distributes corresponding capacity in the caching of preset capacity.
The present invention also provides a kind of read buffer devices of distributed storage, including：
Logging modle is extracted, after obtaining this read request, extracts this of the corresponding file of this described read request
It reads offset and records；
If judgment module is sentenced identical for the corresponding file of this described read request file corresponding with last time read request
Disconnected described this reads whether offset read in the corresponding threshold range of offset in last time；
Pre- read through model, if reading offset in the threshold range for described this, initiation is pre-read.
Optionally, the extraction logging modle, including：
Record sub module is extracted, after obtaining this read request, extracts the institute of the corresponding file of this described read request
This is stated to read offset and this reading length and record.
Optionally, the judgment module, including：
Judging submodule, for judging that described this reads whether offset is more than or equal to last time reading offset and first
The difference of threshold value, and it is less than or equal to last time reading offset, last time reading the sum of length and second threshold；If so, to institute
It states pre- read through model and sends enabling signal.
Optionally, which further includes：
Computing module, for this to read offset according to record and described this reads length, calculating generation
Or update the reading code stream of the corresponding file of this described read request；
Distribution module is cached, for the reading code stream and whole read request pair according to the corresponding file of this described read request
The ratio of the reading code stream for the All Files answered is the corresponding file distribution of this described read request in the caching of preset capacity
Corresponding capacity.
In addition, being deposited on the computer readable storage medium the present invention also provides a kind of computer readable storage medium
Computer program is contained, the computer program realizes distributed storage as described in any one of the above embodiments when being executed by processor
The step of read buffer method.
A kind of read buffer method of distributed storage provided by the present invention, including：After obtaining this read request, this is extracted
This of the corresponding file of secondary read request reads offset and records；If the corresponding file of this read request and last time read request pair
The file answered is identical, judges that this reads whether offset read in the corresponding threshold range of offset in last time；If so, hair
It rises and pre-reads；
As it can be seen that the present invention is by judging that whether this reads offset in the corresponding threshold range of last time reading offset
It is interior, the Rule of judgment that triggering is pre-read has been expanded, can normally have been pre-read, be increased when there is the not exclusively scene of sequence reading
The strong stability of front end reading performance, improves user experience.In addition, the present invention also provides a kind of readings of distributed storage
Buffer storage and computer readable storage medium equally have above-mentioned beneficial effect.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of the read buffer method of distributed storage provided by the embodiment of the present invention；
Fig. 2 is the flow chart of the read buffer method of another kind distributed storage provided by the embodiment of the present invention；
Fig. 3 is that the triggering of the read buffer method of another kind distributed storage provided by the embodiment of the present invention pre-reads displaying
Figure；
Fig. 4 is that the caching of the read buffer method of another kind distributed storage provided by the embodiment of the present invention adjusts process
Figure；
Fig. 5 is the internal storage structure signal of the read buffer method of another kind distributed storage provided by the embodiment of the present invention
Figure；
Fig. 6 is a kind of structure chart of the read buffer device of distributed storage provided by the embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Referring to FIG. 1, Fig. 1 is a kind of process of the read buffer method of distributed storage provided by the embodiment of the present invention
Figure.This method may include：
Step 101：After obtaining this read request, extracts this reading offset of the corresponding file of this read request and remember
Record.
Wherein, this step can be to store the front-end client (processor) docked with rear end in distributed memory system,
After receiving the read request that front end issues, the reading offset of the corresponding file of the read request is extracted.
It is understood that this read request in this step can be front-end client currently from one of front end receiver
Read request.That is, extracting after this step can receive each read request for front-end client and recording each read request
The reading offset of corresponding file.For extracting this reading offset of the corresponding file of this read request in this step simultaneously
The concrete mode of record, can by designer according to practical scene and user demand self-setting, such as can using with it is existing
The similar mode of technology is realized.As long as front-end client can extract the corresponding file of the read request after receiving read request
Reading offset.The present embodiment does not do any restrictions to this.
It should be noted that the purpose of this step can be to read offset by this of extraction, to sentence in step 102
Breaking, this reads whether offset prepares in the corresponding threshold range of last time reading offset
Step 102：If the corresponding file of this read request file corresponding with last time read request is identical, this reading is judged
Whether offset read in the corresponding threshold range of offset in last time；If so, entering step 103.
Wherein, the last time read request in this step can be front-end client before receiving this read request, received
One read request.Corresponding, it can be front-end client after receiving last time read request that last time, which reads offset, pass through step
The reading offset of rapid 101 extraction and the corresponding file of last time read request recorded.
It is understood that the purpose of this step can be corresponding with last time read request in the corresponding file of this read request
The identical situation of file under, read whether offset in last time read the corresponding threshold range of offset by judging this
It is interior, it is determined whether triggering is pre-read, and to expand the Rule of judgment that triggering is pre-read, tolerates the small size shake occurred when file is read, from
And pre-read front-end client normally when there is the not exclusively scene of sequence reading.
Specifically, read the corresponding threshold range of offset for the last time in this step, can by designer according to
Practical scene and user demand self-setting can such as set the threshold range to last time reading offset and subtract a threshold value
Offset was read with last time and adds the threshold value, that is, judged that whether this reads offset in the corresponding threshold value of last time reading offset
In range, it can read whether the absolute value of the difference of offset and last time reading offset is less than or equal to the threshold to judge this
Value；It can also set the threshold range to last time reading offset and subtract a threshold value and last time reading offset plus another
A threshold value；Can also set the threshold range to last time read the difference of offset and first threshold and last time read offset,
Last time reads between the sum of length and second threshold, that is, judges whether this reads offset corresponding in last time reading offset
In threshold range, it can read whether offset is more than or equal to the difference that last time reads offset and first threshold to judge this,
And it is less than or equal to last time reading offset, last time reading the sum of length and second threshold.The present embodiment does not do any restrictions to this.
Wherein, it can be front-end client after receiving last time read request that last time, which reads length, extract and record
The reading length of the corresponding file of last time read request.The acquisition modes for reading length for last time can voluntarily be set by designer
It sets, such as can extract and record each when step 101 is extracted and records the reading offset of the corresponding file of each read request
The reading length of the corresponding file of read request.As long as can read whether offset is more than or equal to last time reading partially judging this
The difference of shifting amount and first threshold, and be less than or equal to before last time reading offset, last time reading the sum of length and second threshold, it obtains
Last time was taken to read length, the present embodiment does not do any restrictions to this.
It should be noted that this step is identical in the corresponding file of this read request file corresponding with last time read request
In the case where the judgement that carries out, the whether identical tool of corresponding for this read request file file corresponding with last time read request
Body deterministic process and judge process, by designer's self-setting, can such as be realized using mode similar with the prior art,
It wants to guarantee that this step is the judgement carried out in the case where last time read request corresponds to same file with this read request, this reality
It applies example and does not do any restrictions to this.
Corresponding, corresponding for this read request file different situation of file corresponding with last time read request can
By using in a manner of similar with the prior art, by front-end client according to this read request from rear end, storage obtains corresponding number
According to.If the corresponding file of this read request file corresponding from last time read request is different, front-end client can be read this
Request is sent to rear end storage, to obtain the corresponding data of this read request.
Specifically, offset is read not in the case of reading in the corresponding threshold range of offset last time for this, it can
To read with being considered jumping characteristic, unsuitable initiation pre-reads occupied bandwidth and space, does not trigger and pre-reads, and can use and existing skill
The similar mode of art, by this read request of the direct basis of front-end client, from rear end, storage obtains corresponding data.
Step 103：Initiation is pre-read.
Wherein, the purpose of the present embodiment can be to expand the Rule of judgment that triggering is pre-read by step 102, and tolerance file is read
The small size shake occurred when taking, so that it is normal pre- to carry out front-end client when there is the not exclusively scene of sequence reading
It reads.It, can be by designer according to practical field for initiating the detailed process pre-read and mode in this step after triggering is pre-read
Scape and user demand are voluntarily configured using mode similar with the prior art, can read offset and this to this
Data after reading length in threshold value are pre-read.Any restrictions are not done in this implementation to this.
In the present embodiment, the embodiment of the present invention is by judging that it is whether corresponding in last time reading offset that this reads offset
Threshold range in, expanded the Rule of judgment that pre-reads of triggering, can have been carried out just when there is scene that not exclusively sequence is read
Normal pre-reads, and enhances the stability of front end reading performance, improves user experience.
Referring to FIG. 2, Fig. 2 is the process of the read buffer method of another kind distributed storage provided by the embodiment of the present invention
Figure.This method may include：
Step 201：After obtaining this read request, this reading offset and sheet of the corresponding file of this read request are extracted
Secondary reading length simultaneously records.
Specifically, as shown in figure 3, this step can connect to store the front-end client (processor) docked with rear end
Front end is received after this read request issued, extract the reading offset (offset ') of the corresponding file of the read request and is read
Length (len ').
Step 202：If the corresponding file of this read request file corresponding with last time read request is identical, this reading is judged
Whether offset is more than or equal to the difference for last time reading offset and first threshold, and is less than or equal to last time reading offset, last time
Read the sum of length and second threshold；If so, entering step 103.
Wherein, as shown in figure 3, it can be front-end client in receiving front-end that last time, which reads offset and last time reading length,
After the read request that last time issues, extracts the reading offset (offset) of the corresponding file of the read request and read length
(len)。
It is understood that the setting of the specific value for first threshold and second threshold in this step, Ke Yiyou
First threshold and second threshold such as can be disposed as 1MB according to practical scene and user demand self-setting by designer.
The present embodiment does not do any restrictions to this.
Specifically, as shown in figure 3, when first threshold and second threshold are 1MB, offset-1MB<offset'<
Offset+len+1MB meets in this step and pre-reads triggering judgement, can be pre-read by step 203.
Step 203：Offset is read to this and this reads the data in the later third threshold value of the sum of length and carries out
It pre-reads.
Specifically, the setting of the specific value for the third threshold value in this step, it can be by designer according to practical
Scene and user demand self-setting such as can set 4MB for third threshold value, as shown in figure 3, can for offset '+
4MB file data initiation after len ' is pre-read, so as to the direct hit of subsequent data request, improving performance.The present embodiment pair
This does not do any restrictions.
In the present embodiment, the embodiment of the present invention reads whether offset is more than or equal to last time reading offset by judging this
The difference of amount and first threshold, and it is less than or equal to last time reading offset, last time reading the sum of length and second threshold, further open up
The Rule of judgment that triggering is pre-read has been opened up, has defined the range slightly shaken occurred when tolerance file is read, has enhanced front end reading
The stability for taking performance, improves user experience.
Based on any of the above-described embodiment, in order to ensure making full use of for caching, it is unbalanced to prevent EMS memory occupation, influences
The case where front-end business performance, occurs, and method provided by the present embodiment can also include that caching dynamic as shown in Figure 4 adjusts
Process.Specifically, referring to FIG. 4, Fig. 4 is the read buffer method of another kind distributed storage provided by the embodiment of the present invention
Caching adjust flow chart.This method may include：
Step 301：Offset and this reading length are read according to this of record, calculates to generate or update this reading and ask
Seek the reading code stream of corresponding file.
Wherein, this step can for when the corresponding file of this read request there is no read code stream when, i.e., before reading
When requesting corresponding with this read request file difference of corresponding file, processor reads offset and this reading according to this
Length calculates the reading code stream for generating the corresponding file of this read request；When there are readout codes for the corresponding file of this read request
When stream, i.e., when the corresponding file of this read request file corresponding with read requests before is identical, processor is according to this reading
Offset and this reading length are taken, the reading code stream for updating the corresponding file of this read request is calculated.
Specifically, generating or updating the specific of the reading code stream of the corresponding file of this read request for calculating in this step
Mode can be such as configured, the present embodiment pair by designer's self-setting using mode similar with the prior art
This does not do any restrictions.
Step 302：According to the reading code stream of the corresponding file of this read request All Files corresponding with whole read requests
Reading code stream ratio, distribute corresponding capacity in the caching of preset capacity for the corresponding file of this read request.
It is understood that in this step the corresponding All Files of whole read request reading code stream, can be all to read
Request the reading code stream of corresponding all different files, the i.e. quantity of reading code stream all differences corresponding with whole read requests
File quantity it is identical, the corresponding identical file of multiple read requests only corresponds to a reading code stream.
Specifically, as shown in figure 5, this step can calculate or update this read request by step 301 for processor
After the reading code stream v1 of corresponding file, counting the reading code streams of All Files, (v1+v2+ ... vn, n are the number of All Files
Amount), the caching of preset capacity (memory_cache) is proportionally distributed, is this read request such as in the caching of preset capacity
The cache size of corresponding file distribution can be memory_cache*v1/ (v1+v2+ ... vn).
It should be noted that the division of the specific cache size (capacity) for the caching of preset capacity, it can be by designing
Personnel or user's self-setting, the present embodiment do not do any restrictions to this.
Corresponding, this step can also include to the alternative document in All Files in addition to the corresponding file of this read request
Cache size set-up procedure.The present embodiment does not do any restrictions to this.
In the present embodiment, the embodiment of the present invention according to the reading code stream of the corresponding file of this read request with whole by reading
The ratio for requesting the reading code stream of corresponding All Files is the corresponding file point of this read request in the caching of preset capacity
With corresponding capacity, the spatial cache that each file occupies can be dynamically distributed, the dynamic adjustment of caching is realized, protects to the maximum extent
It is unbalanced to avoid the occurrence of EMS memory occupation for card caching service efficiency, and then the case where influence front-end business performance occurs.
Referring to FIG. 6, Fig. 6 is a kind of structure of the read buffer device of distributed storage provided by the embodiment of the present invention
Figure.The apparatus may include：
Logging modle 100 is extracted, after obtaining this read request, extracts this reading of the corresponding file of this read request
It takes offset and records；
Judgment module 200, if identical for the corresponding file of this read request file corresponding with last time read request, judgement
This reads whether offset read in the corresponding threshold range of offset in last time；
Pre- read through model 300, if reading offset in threshold range for this, initiation is pre-read.
Optionally, logging modle 100 is extracted, may include：
Record sub module is extracted, after obtaining this read request, extracts this reading of the corresponding file of this read request
It takes offset and this reading length and records.
Optionally, judgment module 200, including：
Judging submodule, for judging that this reads whether offset is more than or equal to last time reading offset and first threshold
Difference, and be less than or equal to last time read offset, last time read the sum of length and second threshold；If so, being sent out to pre- read through model
Send enabling signal.
Optionally, which can also include：
Sending module, if not for the corresponding file of this described read request file corresponding with the last time read request
Together, this described read request is sent to rear end storage, to obtain the corresponding data of this described read request.
Optionally, which can also include：
Computing module calculates for reading offset and this reading length according to this of record and generates or update this
The reading code stream of the corresponding file of secondary read request；
Distribution module is cached, for corresponding with whole read requests according to the reading code stream of the corresponding file of this read request
The ratio of the reading code stream of All Files, distributes corresponding appearance in the caching of preset capacity for the corresponding file of this read request
Amount.
In the present embodiment, the embodiment of the present invention judges that whether this reads offset in last time reading by judgment module 200
It takes in the corresponding threshold range of offset, has expanded the Rule of judgment that triggering is pre-read, can occur what not exclusively sequence was read
It is normally pre-read when scene, enhances the stability of front end reading performance, improve user experience.
The present invention also provides a kind of computer readable storage mediums, have computer program thereon, the computer program
The step of being performed the read buffer method that distributed storage provided by any of the above-described embodiment may be implemented.The storage medium
May include：USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random
Access Memory, RAM), the various media that can store program code such as magnetic or disk.
Each embodiment is described in a progressive manner in specification, the highlights of each of the examples are with other realities
The difference of example is applied, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment and
For computer readable storage medium, since it is corresponded to the methods disclosed in the examples, so be described relatively simple, it is related
Place is referring to method part illustration.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Above to the read buffer method, apparatus of distributed storage provided by the present invention and computer readable storage medium into
It has gone and has been discussed in detail.Used herein a specific example illustrates the principle and implementation of the invention, the above implementation
The explanation of example is merely used to help understand method and its core concept of the invention.It should be pointed out that for the general of the art
, without departing from the principle of the present invention, can be with several improvements and modifications are made to the present invention for logical technical staff, this
A little improvement and modification are also fallen within the protection scope of the claims of the present invention.
Claims (10)
1. a kind of read buffer method of distributed storage, which is characterized in that including：
After obtaining this read request, extracts this reading offset of the corresponding file of this described read request and record；
If the corresponding file of described this read request file corresponding with last time read request is identical, described this of judgement reads offset
Whether amount read in the corresponding threshold range of offset in last time；
If so, initiating to pre-read.
2. the read buffer method of distributed storage according to claim 1, which is characterized in that described this read request of acquisition
Afterwards, further include：
It extracts this reading length of the corresponding file of this described read request and records.
3. the read buffer method of distributed storage according to claim 2, which is characterized in that this is read described in the judgement
Take whether offset read in the corresponding threshold range of offset in last time, including：
Described this of judgement reads the difference whether offset is more than or equal to last time reading offset and first threshold, and is less than or equal to
The last time reads offset, last time reads the sum of length and second threshold；
If so, executing the step of initiation is pre-read.
4. the read buffer method of distributed storage according to claim 2, which is characterized in that extract preceding read and deviate
After amount and this reading length and record, further include：
It, will this described read request if the corresponding file of described this read request file corresponding from the last time read request is different
It is sent to rear end storage, to obtain the corresponding data of this described read request.
5. according to the read buffer method of the described in any item distributed storages of claim 2 to 4, which is characterized in that further include：
This reads offset according to record and described this reads length, calculates generation or updates this described reading and asks
Seek the reading code stream of corresponding file；
According to the readout code for reading code stream All Files corresponding with whole read requests of the corresponding file of this described read request
The ratio of stream is that the corresponding file of this described read request distributes corresponding capacity in the caching of preset capacity.
6. a kind of read buffer device of distributed storage, which is characterized in that including：
Logging modle is extracted, after obtaining this read request, extracts this reading of the corresponding file of this described read request
Offset simultaneously records；
If judgment module judges institute identical for the corresponding file of this described read request file corresponding with last time read request
It states this and reads whether offset read in the corresponding threshold range of offset in last time；
Pre- read through model, if reading offset in the threshold range for described this, initiation is pre-read.
7. the read buffer device of distributed storage according to claim 6, which is characterized in that the extraction logging modle,
Including：
Record sub module is extracted, after obtaining this read request, described of the corresponding file of this described read request of extraction
Secondary reading offset and this reading length simultaneously record.
8. the read buffer device of distributed storage according to claim 7, which is characterized in that the judgment module, including：
Judging submodule, for judging that described this reads whether offset is more than or equal to last time reading offset and first threshold
Difference, and be less than or equal to the last time read offset, last time read the sum of length and second threshold；If so, to described pre-
Read through model sends enabling signal.
9. the read buffer device of distributed storage according to any one of claims 7 to 9, which is characterized in that further include：
Computing module, for according to record this read offset and it is described this read length, calculate generate or more
The reading code stream of the corresponding file of this new described read request；
Distribution module is cached, for corresponding with whole read requests according to the reading code stream of the corresponding file of this described read request
The ratio of the reading code stream of All Files is that the corresponding file distribution of this described read request corresponds in the caching of preset capacity
Capacity.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the reading such as distributed storage described in any one of claim 1 to 5 when the computer program is executed by processor
The step of caching method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810523394.7A CN108874310A (en) | 2018-05-28 | 2018-05-28 | The read buffer method, apparatus and computer readable storage medium of distributed storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810523394.7A CN108874310A (en) | 2018-05-28 | 2018-05-28 | The read buffer method, apparatus and computer readable storage medium of distributed storage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108874310A true CN108874310A (en) | 2018-11-23 |
Family
ID=64335279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810523394.7A Pending CN108874310A (en) | 2018-05-28 | 2018-05-28 | The read buffer method, apparatus and computer readable storage medium of distributed storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108874310A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110502498A (en) * | 2019-08-16 | 2019-11-26 | 济南浪潮数据技术有限公司 | Distributed file system file pre-reading method and system |
| CN111625503A (en) * | 2020-05-29 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Method and equipment for locally and randomly pre-reading file of distributed file system |
| CN112162956A (en) * | 2020-09-11 | 2021-01-01 | 北京浪潮数据技术有限公司 | Skip read pre-reading method, device, equipment and storage medium |
| CN113609093A (en) * | 2021-06-30 | 2021-11-05 | 济南浪潮数据技术有限公司 | Reverse order reading method, system and related device of distributed file system |
| WO2021238265A1 (en) * | 2020-05-28 | 2021-12-02 | 广东浪潮智慧计算技术有限公司 | File pre-reading method, apparatus and device, and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1280063A2 (en) * | 2001-07-27 | 2003-01-29 | Fujitsu Limited | Cache control methods and apparatus for hard disk drives |
| CN103544076A (en) * | 2012-07-13 | 2014-01-29 | 阿里巴巴集团控股有限公司 | Data backup method and device |
| CN103795781A (en) * | 2013-12-10 | 2014-05-14 | 西安邮电大学 | Distributed cache model based on file prediction |
| CN105426127A (en) * | 2015-11-13 | 2016-03-23 | 浪潮(北京)电子信息产业有限公司 | File storage method and apparatus for distributed cluster system |
| CN106776759A (en) * | 2016-11-17 | 2017-05-31 | 郑州云海信息技术有限公司 | The small documents pre-head method and system of distributed file system |
| CN107340978A (en) * | 2017-07-18 | 2017-11-10 | 郑州云海信息技术有限公司 | One kind storage pre-head method, device and storage system |
-
2018
- 2018-05-28 CN CN201810523394.7A patent/CN108874310A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1280063A2 (en) * | 2001-07-27 | 2003-01-29 | Fujitsu Limited | Cache control methods and apparatus for hard disk drives |
| CN103544076A (en) * | 2012-07-13 | 2014-01-29 | 阿里巴巴集团控股有限公司 | Data backup method and device |
| CN103795781A (en) * | 2013-12-10 | 2014-05-14 | 西安邮电大学 | Distributed cache model based on file prediction |
| CN105426127A (en) * | 2015-11-13 | 2016-03-23 | 浪潮(北京)电子信息产业有限公司 | File storage method and apparatus for distributed cluster system |
| CN106776759A (en) * | 2016-11-17 | 2017-05-31 | 郑州云海信息技术有限公司 | The small documents pre-head method and system of distributed file system |
| CN107340978A (en) * | 2017-07-18 | 2017-11-10 | 郑州云海信息技术有限公司 | One kind storage pre-head method, device and storage system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110502498A (en) * | 2019-08-16 | 2019-11-26 | 济南浪潮数据技术有限公司 | Distributed file system file pre-reading method and system |
| WO2021238265A1 (en) * | 2020-05-28 | 2021-12-02 | 广东浪潮智慧计算技术有限公司 | File pre-reading method, apparatus and device, and storage medium |
| CN111625503A (en) * | 2020-05-29 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Method and equipment for locally and randomly pre-reading file of distributed file system |
| CN111625503B (en) * | 2020-05-29 | 2022-11-04 | 苏州浪潮智能科技有限公司 | Method and equipment for locally and randomly pre-reading file of distributed file system |
| CN112162956A (en) * | 2020-09-11 | 2021-01-01 | 北京浪潮数据技术有限公司 | Skip read pre-reading method, device, equipment and storage medium |
| CN113609093A (en) * | 2021-06-30 | 2021-11-05 | 济南浪潮数据技术有限公司 | Reverse order reading method, system and related device of distributed file system |
| CN113609093B (en) * | 2021-06-30 | 2023-12-22 | 济南浪潮数据技术有限公司 | Reverse order reading method, system and related device of distributed file system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108874310A (en) | The read buffer method, apparatus and computer readable storage medium of distributed storage | |
| CN109951290B (en) | Time service authentication method, device and equipment for chain type account book | |
| US10042576B2 (en) | Method and apparatus for compressing addresses | |
| CN107562915A (en) | Read the method, apparatus and equipment and computer-readable recording medium of small documents | |
| CN107729417A (en) | A kind of quota gives back implementation method, device, equipment and computer-readable storage medium | |
| CN106569963A (en) | Buffering method and buffering device | |
| CN109471843A (en) | A kind of metadata cache method, system and relevant apparatus | |
| CN111966632B (en) | Streaming data storage method, reading method, equipment and readable storage medium | |
| CN110347679B (en) | Data storage method, device and equipment based on receipt | |
| CN107645547A (en) | Data temperature based on storage device determines method, equipment and system | |
| CN106817388A (en) | The system that virtual machine, host obtain the method, device and access data of data | |
| US7660964B2 (en) | Windowing external block translations | |
| CN107122140A (en) | A kind of file intelligent storage method based on metadata information | |
| WO2021128897A1 (en) | Method, apparatus and device for creating thick-provisioned volume, and medium | |
| CN109753446A (en) | Data cached method is brushed under a kind of storage system | |
| CN102902735B (en) | A kind of IPTV IPTV searches for caching method and system | |
| EP3274844B1 (en) | Hierarchical cost based caching for online media | |
| CN107333169A (en) | A kind of client-cache method of adjustment and device | |
| CN115576956A (en) | Data processing method, system, equipment and storage medium | |
| CN110362570A (en) | A kind of date storage method, device and equipment | |
| CN105893424A (en) | Data storage device and method for server | |
| CN108984124A (en) | Reduce the method, apparatus, computer equipment and storage medium for writing amplification | |
| CN105530303B (en) | A kind of network-caching linear re-placement method | |
| CN105740167B (en) | A kind of method and system that file system cache is deleted | |
| CN108932104A (en) | A kind of data processing method, device and processing server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181123 |