CN108182151B - Method, apparatus, storage medium and the terminal device of sequential flow caching and detection - Google Patents
Method, apparatus, storage medium and the terminal device of sequential flow caching and detection Download PDFInfo
- Publication number
- CN108182151B CN108182151B CN201810068469.7A CN201810068469A CN108182151B CN 108182151 B CN108182151 B CN 108182151B CN 201810068469 A CN201810068469 A CN 201810068469A CN 108182151 B CN108182151 B CN 108182151B
- Authority
- CN
- China
- Prior art keywords
- data
- sequential flow
- address
- caching
- sequential
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
- G06F12/0615—Address space extension
- G06F12/063—Address space extension for I/O modules, e.g. memory mapped I/O
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
Abstract
The present invention proposes method, apparatus, storage medium and the terminal device of a kind of sequential stream detection and caching, which comprises receives the access request of I/O data, and records the address information of the I/O data in the skip list of address；According to the address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；Judge whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value；And if so, be recorded in the attribute information in the fixed length array according to the address skip list and the sequential flow, the I/O data for the sequential flow being currently received is cached by benchmark piecemeal of IO queue depth.Using the present invention, sequence flow data can be efficiently identified and cached by benchmark piecemeal of IO queue depth.
Description
Technical field
Method, apparatus, storage the present invention relates to field of computer technology more particularly to a kind of sequential flow caching with detection
Medium and terminal device.
Background technique
Within the storage system, IO (Input/Output, input/output) scheduler module needs pre-read current IO,
And targetedly handled and optimized, to allow IO performance to be optimal as far as possible using disk performance.Among these to sequence
The optimization of IO is extremely important, especially following several scenes:
(1) to a large amount of small optimization for being sequentially written in IO；
(2) optimization of IO is read in a large amount of small sequence；
(3) to the optimization for being sequentially written in IO of non-address alignment.
Wherein, biggish for a data volume to write IO by taking scene 1 as an example, it may be because of service logic, operating system
Or a variety of causes such as processing in file system are split as the soft small letter IO of multiple data volumes, but if disk directly one by one
If handling write request, the amplification for writing IO will cause, and reduce disk performance.With scene 2 citing, for a data volume compared with
IO is greatly read, may be split as because of a variety of causes such as processing in service logic, operating system or file system
Multiple smaller IO of data volume will cause reading IO and put if disk IO smaller to each data volume directly handles reading request
Greatly, disk performance is reduced.With the citing of scene 3, for continuously writing IO, can understand because of service logic, operating system or file
The a variety of causes such as the processing in system are misaligned with disk block address if writing IO after cutting, are actually write by cutting again
When IO, disk can read the data of head and the tail block first, and after merging in memory with the data of write-in, be then written to disk, this
Sample will lead to additional reading IO, and causes and write IO amplification, and reduce disk performance.These types of scene requires scheduler module energy
Under the scene of various complexity, especially in the case where Multi-Client Concurrency Access and a large amount of sequences and random IO mix, identification
The IO sequential flow of multipath concurrence out.Therefore it provides a kind of general method that multipath concurrence sequence IO is detected and dispatched
It is very important.And the scheme for the detection IO sequential flow that the prior art provides has the following deficiencies:
1, the purposes of the identification of the sequential flow IO of the prior art is mainly that IO is pre-read, and there is no the write-ins of optimization write-in IO
Journey；
2, IO queue depth is not supported to detect, reason is mainly that the purposes of existing technical solution is to pre-read, but for writing
IO, if IO depth be 1 if (IO be sequence initiate rather than it is parallel), not can be carried out caching actually, otherwise will cause
IO obstruction；
3, can not detection ordering stream IO when terminate, be that will lead to pre-read in vain for reading IO, for writing IO
It is that tail portion IO is caused to block or generate biggish delay；
4, most of sequential flow IO recognition methods is all based on the piecemeal of regular length (primarily to looking by address Hash
Look for), then this type method is to address space overlap, non-critical dull continuous (such as continuous address the I/O sequence majority IO list in address
Adjust a small amount of address IO inverted sequence in the case where being incremented by), the endless Total continuity in address (being spaced among such as part IO) situations such as write
Enter to handle existing defects.
Summary of the invention
The embodiment of the present invention provides method, apparatus, storage medium and the terminal device of a kind of speech polling, to solve or delay
Solve the above technical problem in the prior art.
In a first aspect, the embodiment of the invention provides a kind of methods of sequential stream detection and caching, comprising:
The access request of I/O data is received, and records the address information of the I/O data in the skip list of address；
According to the address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record in
Fixed length array；Wherein, the attribute information includes the IO quantity of the sequential flow, and the IO quantity indicates that the sequential flow has connect
It receives but the quantity of uncached I/O data；
Judge whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value；And
If so, the attribute information in the fixed length array is recorded according to the address skip list and the sequential flow, to institute
State that sequential flow has received but uncached I/O data carries out piecemeal caching；The quantity of the I/O data of each piece of caching is not more than institute
State the IO queue depth of sequential flow.
Further, the address information includes the offset address of the I/O data, and the attribute information further includes described
The offset address of renewal time and the sequential flow of the attribute information of sequential flow in the fixed length array；And
It is described according to the address skip list and the I/O data, the attribute letter of the affiliated sequential flow of I/O data described in more new record
It ceases in fixed length array, comprising:
Judge whether the I/O data and the address skip list history I/O data recorded are related；
If so, extracting history I/O data offset ground having the same relevant to the I/O data from the fixed length array
The attribute information of location and renewal time nearest sequential flow, and according to the attribute information of extraction and the I/O data, more new record
The attribute information of the affiliated sequential flow of I/O data is in fixed length array
Further, described according to the address skip list and the I/O data, sequence belonging to I/O data described in more new record
The attribute information of stream is in fixed length array, comprising:
When the I/O data is uncorrelated to the history I/O data that the address skip list is recorded, according to the ground of the I/O data
Location information creates the attribute information of sequential flow belonging to the I/O data in the fixed length array.
Further, the address information includes the data length of the I/O data；And the judgement I/O data
It is whether related to the address skip list history I/O data recorded, comprising:
According to the storage for the history I/O data that the storage address section of the I/O data, judgement are recorded with the address skip list
Address section is with the presence or absence of overlapping or continuous relationship；Wherein, the storage address section by the I/O data offset address
The section obtained with data length computation；
If so, determining that the I/O data is related to the history I/O data that the address skip list is recorded；And
If it is not, determining that the I/O data and the address skip list history I/O data recorded are uncorrelated.
Further, the attribute information further includes data length；And
The attribute information and the I/O data according to extraction, the attribute of the affiliated sequential flow of I/O data described in more new record
Information is in fixed length array, comprising:
One is added to the IO quantity in the attribute information of extraction, and sequence belonging to the I/O data is updated with the numerical value after adding one
The IO quantity of stream；
The I/O data is updated with the sum of the data length in the data length of the I/O data and the attribute information of extraction
The data length of affiliated sequential flow；
The renewal time of the affiliated sequential flow of the I/O data is updated with current time；And
The attribute information of the updated affiliated sequential flow of the I/O data is recorded in the fixed length array.
Further, the attribute information further includes data length, the address information according to the I/O data, is created
The attribute information of sequential flow belonging to the I/O data is in the fixed length array, comprising:
Using numerical value one as the IO quantity of the affiliated sequential flow of the I/O data；
Using the data length of the I/O data as the data length of the affiliated sequential flow of the I/O data；
Using current time as the renewal time of the affiliated sequential flow of the I/O data；
Using the offset address of the I/O data as the offset address of the affiliated sequential flow of the I/O data；And
The attribute information of the affiliated sequential flow of the I/O data generated is recorded in the fixed length array.
Further, the fixed length array includes that leader is known；And record the attribute letter of the affiliated sequential flow of the I/O data
It ceases in the fixed length array, comprising:
The attribute information for recording the affiliated sequential flow of the I/O data is directed toward in the address that the leader is known in the fixed length number
Position in group；And
The cyclic address change that the leader is known.
Further, the method also includes:
When the I/O data and the uncorrelated address skip list history I/O data recorded, monitoring the I/O data is
It is no to be buffered in equipment caching and start to be performed refresh operation in equipment caching；The refresh operation is included in
The I/O data is removed in the equipment caching and the I/O data is written to the operation of disk；
If so, the execution state for recording sequential flow belonging to the I/O data is unfinished, and record the sequential flow
The initial value of concurrent quantity is one；And
When the I/O data, which has executed, completes the refresh operation, the execution state for recording the sequential flow is to complete,
And the IO queue depth is updated according to the numerical value of the concurrent quantity.
Further, the method also includes:
When the I/O data is related to the history I/O data that the address skip list is recorded, judge belonging to the I/O data
Whether the execution state of sequential flow is unfinished；If so, adding one to the concurrent quantity.
It is further, described that the IO queue depth is updated according to the numerical value of the concurrent quantity, comprising:
Judge whether the numerical value of the concurrent quantity is greater than the numerical value of current IO queue depth；And
If so, using the numerical value of the concurrent quantity as the numerical value of the updated IO queue depth.
Further, the method also includes:
When the I/O data of the sequential flow is cached in equipment caching, it is slow that the equipment is inquired according to the address skip list
It whether there is I/O data relevant to the sequential flow in depositing；
If nothing, timing is started to the sequential flow；
When the I/O data being buffered in the equipment caching to the sequential flow executes refresh operation, to the clocking value
It resets；
Judge whether the corresponding clocking value of the sequential flow reaches timing threshold value；And
When the clocking value reaches the timing threshold value, the sequential flow is buffered in all in the equipment caching
I/O data executes refresh operation, and the sequential flow is recorded in all letters in the address skip list and the fixed length array
Breath is removed.
Further, the largest buffered length is the mean data length of I/O data and multiplying for the IO queue depth
Product；The setting condition of the largest buffered quantity of the equipment caching are as follows: no more than the IO number that can be cached in equipment caching
According to maximum quantity and be not more than the IO queue depth；And the method also includes:
Judge whether the quantity for the I/O data that the sequential flow is buffered in the equipment caching reaches the largest buffered
Quantity, and judge the total data length for the I/O data that the sequential flow is buffered in equipment caching whether reach it is described most
Big buffer storage length；And
When the quantity reaches the largest buffered quantity or the total data length reaches the largest buffered length,
All I/O datas being buffered in the equipment caching to the sequential flow execute refresh operation.
Further, the method also includes:
According to the address skip list and the fixed length array, the I/O data that the sequential flow is buffered in equipment caching is judged
Storage address whether across storage equipment memory block；And
If so, being buffered in the equipment caching and storage address to the sequential flow is the outland across memory block
All I/O datas before location execute refresh operation.
Further, the method also includes:
When the I/O data being buffered in the equipment caching to the sequential flow starts to execute refresh operation, according to described
Sequential flow is recorded in the renewal time of the fixed length array, records the refresh time of the sequential flow；And according to the fixed length
Array judges that the offset address for first I/O data for executing the refresh operation and the sequential flow are recorded in the fixed length number
Whether the offset address in group is consistent；
If so, the initial value that the refreshing quantity of the sequential flow is arranged is one；
If it is not, the refreshing quantity to the sequential flow adds one；
At the end of the refresh operation, subtract one to the refreshing quantity of the sequential flow；
Judge it is described refresh quantity numerical value whether be zero, and judge the refresh time in period threshold value whether
There is update；
If no, all I/O datas being buffered in the equipment caching to the sequential flow execute refresh operation.
Second aspect, the embodiment of the present invention provide the device of a kind of sequential stream detection and caching, comprising:
Access request receiving module for receiving the access request of I/O data, and records the address information of the I/O data
In the skip list of address；
Fixed length array update module, for according to the address skip list and the I/O data, I/O data described in more new record
The attribute information of affiliated sequential flow is in fixed length array；Wherein, the attribute information includes the IO quantity of the sequential flow, the IO
Quantity indicates that the sequential flow has received but the quantity of uncached I/O data；
Queue depth's judgment module, for judging whether the IO quantity meets IO queue depth；And
Sequential flow cache module, for if so, being recorded in the fixed length number according to the address skip list and the sequential flow
Attribute information in group, has received the sequential flow but uncached I/O data carries out piecemeal caching；Each piece caching
The quantity of I/O data is not more than the IO queue depth of the sequential flow.
Further, the address information includes the offset address of the I/O data, and the attribute information further includes described
The offset address of renewal time and the sequential flow of the attribute information of sequential flow in the fixed length array；And
The fixed length array update module includes:
Data correlated judgment unit, for judging whether is history I/O data that the I/O data and the address skip list are recorded
It is related；
First attribute updating unit is used for if so, extracting history relevant to the I/O data from the fixed length array
The attribute information of I/O data offset address having the same and renewal time nearest sequential flow, and according to the attribute information of extraction
With the attribute information of the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；And
Second attribute updating unit, for if it is not, being created belonging to the I/O data according to the address information of the I/O data
Sequential flow attribute information in the fixed length array.
The function of described device can also execute corresponding software realization by hardware realization by hardware.It is described
Hardware or software include one or more modules corresponding with above-mentioned function.
It is described to deposit including processor and memory in the structure of sequential stream detection and caching in a possible design
Reservoir is used to store the side for supporting the device of sequential stream detection and caching to execute sequential stream detection and caching in above-mentioned first aspect
The program of method, the processor is configured to for executing the program stored in the memory.The sequential stream detection and slow
The device deposited can also include communication interface, logical for the device of sequential stream detection and caching and other equipment or communication network
Letter.
The third aspect, the embodiment of the invention provides a kind of computer readable storage mediums, for sequential stream detection and delay
Computer software instructions used in the device deposited comprising for executing the side of sequential stream detection and caching in above-mentioned first aspect
Method is program involved in the device of sequential stream detection and caching.
Any one technical solution in above-mentioned technical proposal have the following advantages that or the utility model has the advantages that
The embodiment of the present invention recorded while receiving the access request of I/O data the address information of the I/O data in
In the skip list of address；Then according to address skip list and the self attributes information of the I/O data received come I/O data described in more new record
The attribute information of affiliated sequential flow is in fixed length array, so that the serial relation belonged between the I/O data of the same sequential flow passes through
Address skip list and fixed length array connect, and since fixed-length data updates the quantity of the sequential flow received, so that only
Receive meet certain IO quantity sequential flow can disposable piecemeal be cached, piecemeal is based on IO queue and caches depth.
In addition, above-mentioned skip list and fixed length array can identify that sequential flow facilitates further cache and progress to refresh accordingly into disk
Operation.
Above-mentioned general introduction is merely to illustrate that the purpose of book, it is not intended to be limited in any way.Except foregoing description
Schematical aspect, except embodiment and feature, by reference to attached drawing and the following detailed description, the present invention is further
Aspect, embodiment and feature, which will be, to be readily apparent that.
Detailed description of the invention
In the accompanying drawings, unless specified otherwise herein, otherwise indicate the same or similar through the identical appended drawing reference of multiple attached drawings
Component or element.What these attached drawings were not necessarily to scale.It should be understood that these attached drawings depict only according to the present invention
Disclosed some embodiments, and should not serve to limit the scope of the present invention.
Fig. 1 is the flow diagram of one embodiment of the method for sequential stream detection provided by the invention and caching；
Fig. 2 is one that the attribute information of fixed length array in sequential stream detection provided by the invention and the method for caching updates
The flow diagram of embodiment；
Fig. 3 is the schematic diagram of one embodiment of the index relative of skip list provided by the invention and fixed length array；
Fig. 4 is the flow diagram of the one embodiment for the method that IO queue depth provided by the invention updates；
Fig. 5 is the schematic diagram of another embodiment of the method that IO queue depth provided by the invention updates；
Fig. 6 is the stream of first embodiment of the flush buffers of the method for sequential stream detection provided by the invention and caching
Journey schematic diagram；
Fig. 7 is the stream of second embodiment of the flush buffers of the method for sequential stream detection provided by the invention and caching
Journey schematic diagram；
Fig. 8 is the stream of the third embodiment of the flush buffers of the method for sequential stream detection provided by the invention and caching
Journey schematic diagram；
Fig. 9 is the stream of the 4th embodiment of the flush buffers of the method for sequential stream detection provided by the invention and caching
Journey schematic diagram；
Figure 10 is the structural schematic diagram of one embodiment of the device of sequential stream detection provided by the invention and caching；
Figure 11 is the structural schematic diagram of one embodiment of terminal device provided by the invention.
Specific embodiment
Hereinafter, certain exemplary embodiments are simply just described.As one skilled in the art will recognize that
Like that, without departing from the spirit or scope of the present invention, described embodiment can be modified by various different modes.
Therefore, attached drawing and description are considered essentially illustrative rather than restrictive.
The present invention is directed to propose the mechanism of a kind of sequential stream detection and caching, records each IO received by skip list
The address information of data, and one fixed length array of setting, each of array element represent the attribute letter an of sequential flow
Breath, then whether one I/O data of every reception, related according to the I/O data that address skip list inquires the I/O data to historical reception arrives,
And then determine the sequential flow belonging to it, it updates the attribute information of the sequential flow recorded in fixed length array, i.e., further cache or writes
Sequential flow and corresponding operation can be identified when entering or reading sequential flow by skip list and fixed length array.The embodiment of the present invention can
To be widely used in I/O system, the including but not limited to modules such as local file system, Network File System, disk drive can have
Effect reduces the IO number write out, optimizes IO form, improves the performance of IO, can also be in flash memory (including but not limited to SSD)
When, it can effectively reduce and write amplification number, improve the service life of flash memory, and when being used for file system, can effectively subtract
The quantity of few disk fragments.Technical solution of the present invention is described into a point embodiment below:
Embodiment one
Referring to Fig. 1, the embodiment of the invention provides a kind of method of sequential stream detection and caching, by disk scheduling module
It executes, including step S110 is to step S140, specific as follows:
S110, receives the access request of I/O data, and records the address information of I/O data in the skip list of address.
In order to realize the record and caching of I/O data, present invention uses the information of fixed length storage of array history I/O data,
The actually attribute information of sequential flow, and use address skip list as the row of index record I/O data offset address offset
Sequence information, address skip list are ranked up with the offset address offset of I/O data for key, the ground of the I/O data of same serial relation
Location information arrangement is specific as shown in Figure 3 after the address information of previous I/O data.
S120, according to address skip list and I/O data, the attribute information of the affiliated sequential flow of the current I/O data of more new record is in fixed
Long array；Wherein, the attribute information includes the IO quantity of sequential flow.IO quantity indicates that the sequential flow has received but do not delay
The quantity for the I/O data deposited.
Wherein, each element of fixed length array is that (such as sequential flow 1 is ctx_ for the record of the attribute information of a sequential flow
1), which includes the time (up_time) of update of the attribute information in the fixed length array, is currently received
Data length (the seq_ of the IO quantity (seq_num) of the I/O data of the sequential flow, the I/O data for the sequential flow being currently received
The information such as len).
So, for a specific example of step S120, as shown in Fig. 2, can be such that
S121 judges whether the I/O data being currently received and the address skip list history I/O data recorded are related.
Since skip list record in address has the offset address offset and data length len of each I/O data, then accordingly
The address section (offset_begin, offset_end) of each available I/O data, offset_begin are the IO number
According to offset address offset, offset_end is the offset address offset of the I/O data plus the ground after data length len
Location.In turn, whether related to history I/O data in the I/O data for judging to be currently received, associated sequential flow can be searched,
Specifically:
According to the storage address for the history I/O data that the storage address section of current I/O data, judgement are recorded with address skip list
Section is with the presence or absence of overlapping or continuous relationship；If so, determining the history I/O data phase that current I/O data is recorded with address skip list
It closes；If it is not, determining that current I/O data and the history I/O data that address skip list is recorded are uncorrelated.So for address space weight
Situations such as endless Total continuity in the non-critical dull continuous, address in folded, address, can also identify, correspondingly be recorded.So,
It is as follows for the subsequent operation for updating fixed length array:
S122, if determining related, the extraction history I/O data tool relevant to the I/O data from the fixed length array
There is the attribute information of identical offset address and renewal time nearest sequential flow, and according to the attribute information of extraction and the IO
Data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array.
S123, if it is determined that it is uncorrelated, according to the address information of the I/O data, create sequential flow belonging to the I/O data
Attribute information in the fixed length array.
In fact, lower half is fixed length array, and fixed length array also will record sequential flow as shown in figure 3, the upper half is skip list
Offset address, the address is identical as the offset address of first I/O data of sequential flow, and address skip list also record have it is each
The offset address of a I/O data, then can be by the index for address information of the I/O data recorded in the skip list of address by offset address
To the attribute information of sequential flow.Assume that current I/O data is related to the I/O data of history due to aforementioned, then jumping according to address
Table can extract the address information of all relevant I/O datas, and the address of the sequential flow of another aspect fixed length array record is always
It is the offset address of first I/O data, extracts the attribute information with the address information extracted with same offset address, be
The attribute information of the current affiliated sequential flow of I/O data, then according to extraction and recent renewal attribute information and current IO number
According to, it can it is updated the attribute information of sequential flow.Assuming that current I/O data is relevant premise to the I/O data of history
Under, for this renewal process, specifically can be such that
One is added to the IO quantity in the attribute information of extraction, and sequence belonging to current I/O data is updated with the numerical value after adding one
The IO quantity of stream；
Current I/O data is updated with the sum of the data length in the data length of current I/O data and the attribute information of extraction
The data length of affiliated sequential flow；
The renewal time of the affiliated sequential flow of current I/O data is updated with current time；
Using the offset address in the attribute information of extraction as the offset address of the affiliated sequential flow of current I/O data；And
The attribute information of the updated affiliated sequential flow of current I/O data is recorded in the fixed length array.
Wherein, fixed length array has a head to identify (head_index), for recording time nearest IO serial number, this when
Between can be absolute time, be also possible to a relative time, known by this leader, it is equivalent that fixed-length data is changed into one
The queue of first in first out (FIFO).So the above-mentioned fixed length array that is recorded in can be with are as follows: record the affiliated sequential flow of current I/O data
Attribute information is directed toward the position in the fixed length array in the address that the leader is known；And the address known to the leader adds
One.
On the other hand, it is assumed that current I/O data is uncorrelated to the I/O data of history, then it is actually regarded as one
First I/O data of new sequential flow, then the specific implementation process of step S123 can be such that
Using numerical value one as the IO quantity of the affiliated sequential flow of the I/O data；
Using the data length of the I/O data as the data length of the affiliated sequential flow of the I/O data；
Using current time as the renewal time of the affiliated sequential flow of the I/O data；
Using the offset address of the I/O data as the offset address of the affiliated sequential flow of the I/O data；And
The attribute information of the affiliated sequential flow of the I/O data generated is recorded in the fixed length array.Optionally, it records
The attribute information of the affiliated sequential flow of current I/O data is directed toward the position in the fixed length array in the address that the leader is known；And
The cyclic address change that the leader is known.
S130, judges whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value.
In the present embodiment, the I/O data number of requests of current sequential flow meets preset quantity threshold value or the present count
When measuring the integral multiple of threshold value, it can cache the I/O data for the sequential flow being currently received.The preset quantity threshold
The integral multiple of value or preset quantity threshold value equipment according to the performance of different disk or interface.It can during caching
To identify the data for the sequential flow to be cached to the sequential flow to be cached by address skip list above-mentioned and fixed-length data, i.e.,
Step S140.On the other hand, since fixed length array also will record the data length for the sequential flow being currently received, thus can also lead to
The integral multiple for judging whether the data length of record meets preset data length threshold or the preset data length threshold is crossed, if
It is to meet, step S140 also can be performed.
S140 is recorded in the attribute information in the fixed length array according to the address skip list and the sequential flow, to institute
State that sequential flow has received but uncached I/O data carries out piecemeal caching, the quantity of the I/O data of each piece of caching is not more than institute
State the IO queue depth of sequential flow.Wherein, IO queue depth indicate to execute the I/O data that has cached in sequential flow refresh operation,
But the quantity of the I/O data of refresh operation is not completed.
In a specific example, for the same sequential flow, it is assumed that preset quantity threshold value is 10, is currently received
The continuous IO quantity of the sequential flow be 10, then the IO quantity of current record is 10 or 10 integral multiples, and false
If the IO queue depth detected is 5, then for the I/O data of the 10 same sequential flows received, with every piece or every group not
Each piece or each group of I/O data are cached on the basis of greater than 5 I/O datas.
Embodiment two
Referring to Fig. 4, depending on IO queue depth, i.e. IO team during the caching sequential flow that previous embodiment is related to
The detection process of column depth be also for the caching for writing IO it is quite important, the present embodiment will be described in the detection of IO queue depth
Process:
Firstly, can execute monitoring process below to first I/O data of each sequential flow:
S210, if current I/O data and the address skip list history I/O data recorded are uncorrelated, i.e. the current data
For first I/O data of a new sequential flow, monitor whether first I/O data is buffered in equipment caching and in institute
It states and starts to be performed refresh operation in equipment caching；The refresh operation be included in equipment caching remove I/O data and
By the operation of I/O data write-in disk；
S220, if first I/O data is buffered in equipment caching and starts to be performed brush in equipment caching
New operation, then it is unfinished for recording the execution state of sequential flow belonging to first I/O data, and record the sequential flow and
The initial value for sending out quantity is one；And
S230, if the first I/O data, which has executed, completes the refresh operation, the execution state for recording the sequential flow is to complete,
And the IO queue depth is updated according to the numerical value of the concurrent quantity.
S240, if current I/O data is related to the history I/O data that address skip list is recorded, that is, the current data is with the
The connected I/O data of the affiliated sequential flow of one I/O data, judges whether the execution state of sequential flow belonging to the I/O data is not complete
At；If so, adding one to the concurrent quantity.
It is selectively, described that the IO queue depth is updated according to the numerical value of the concurrent quantity, comprising:
Judge whether the numerical value of the concurrent quantity is greater than the numerical value of current IO queue depth；And
If so, using the numerical value of the concurrent quantity as the numerical value of the updated IO queue depth.
Citing description is carried out to the present embodiment below with reference to Fig. 5:
Firstly, the attribute information of the first I/O data IO_1 of a new sequential flow of the most junior one is identified as (IO-ctx_1),
State is not complete (done=false), and concurrent quantity is 1 (co_num=1), then (including but not limited to storage service
Disk, iSCSI interface etc.) initiate IO_1 request；
Then, when being reached with the continuous IO_2 of the first I/O data IO_1, it will the information of more new record (IO-ctx_1),
If state is still unfinished (done=false), concurrent quantity adds 1 (co_num+1), it is assumed that continues continuous n-th of IO and reaches
Then ctx_1 is unfinished state, then concurrent quantity is n；
And when IO_1 requests to complete, the information of ctx_1 will be updated, state is set to complete (done=true),
The concurrent quantity (co_num) detected at this time can regard as IO queue depth (io_depth)；
In addition, if it find that state is not complete (done=true), then can be reset when new sequential flow reaches
State is not complete (done=false), and concurrent quantity is 1 (co_num=1), carries out next round detection, if next round is visited
The IO queue depth (io_depth) of survey is greater than original value, then will be updated the value of IO queue depth (io_depth).
Embodiment three
Refer to Fig. 6, the present embodiment description is cached in equipment caching after by equipment cache in I/O data flush to
Process in specific disk or interface can be in a sequential flow of current cache for the I/O data for being written or reading
After data reach time threshold, then can the execution refresh operation of this current sequential flow be removed it and be cached in equipment
In data and be stored in the memory of disk or interface, it is specific as follows:
S310 inquires the equipment according to the address skip list when the I/O data of sequential flow is cached in equipment caching
It whether there is I/O data relevant to the sequential flow in caching；
S320 starts timing to the sequential flow if nothing；
S330, when the I/O data being buffered in the equipment caching to the sequential flow executes refresh operation, to the meter
Duration is reset；
Since refresh process is by one piece of one piece flushes in disk of data block, then it is too long to there is sequential flow, several times
Refresh operation flushes in disk, then immediately non-triggering timing threshold value and the refresh operation that executes should also carry out clocking value
It resets.
S340, judges whether the corresponding clocking value of the sequential flow reaches timing threshold value；And
S350 is buffered in all IO numbers in the equipment caching to the sequential flow when clocking value reaches timing threshold value
It is clear according to execution refresh operation, and all information that the sequential flow is recorded in the address skip list and the fixed length array
It removes.
Example IV
Fig. 7 is referred to, the present embodiment can provide the strategy of second of flush buffers on the basis of previous embodiment:
Firstly, setting largest buffered length is the mean data length of I/O data and the product of the IO queue depth, institute
The largest buffered quantity for stating caching is the maximum quantity for the I/O data that can be cached in caching no more than the equipment and is not more than
The IO queue depth.
It, can be with for the spatial cache in equipment caching: setting the mean data length of I/O data in the present embodiment
(avg_len) multiplied by the value of IO queue depth (io_depth) as cache size, for example, for reading IO, it can be directly with caching
Size (cache_size) press address align after as read buffer value (rd_cache_size) progress flush buffers operation,
The operation of flush buffers can be carried out in the way of writing IO.For writing IO, writing IO due to caching will lead to I/O latency
(latency) elongated, and then IO throughput is caused to decline, it is therefore desirable to the performance of I/O device is tested, to obtain a conjunction
Suitable maximum write buffer quantity (wr_cache_num) and maximum write buffer length (wr_cache_size), but each sequential flow
Maximum write buffer quantity (wr_cache_num) is not greater than the most capitalization of IO queue depth (io_depth) or each sequential flow
Buffer storage length is not greater than the mean data length of I/O data and the product of the IO queue depth, thus reasoning IO queue depth
IO stream for 1 (io_depth=1) should not be cached.
Then in the present embodiment, the strategy of flush buffers can be with are as follows:
S410, judges whether the quantity for the I/O data that the sequential flow is buffered in the equipment caching reaches the maximum
Quantity is cached, and judges whether the total data length for the I/O data that the sequential flow is buffered in the equipment caching reaches institute
State largest buffered length；And
S420, when the quantity reach the largest buffered quantity or the total data length to reach the largest buffered long
When spending, all I/O datas being buffered in the equipment caching to the sequential flow execute refresh operation.
Embodiment five
Fig. 8 is referred to, the present embodiment can provide the strategy of the third flush buffers on the basis of previous embodiment:
S510 judges that the sequential flow is buffered in the equipment caching according to the address skip list and the fixed length array
In I/O data storage address whether across storage equipment memory block；And
S520, if so, being buffered in the equipment caching and storage address to the sequential flow is across memory block
All I/O datas before boundary address execute refresh operation.
In a specific example, a certain sequential flow is buffered in the storage address of the I/O data in the equipment caching are as follows:
[4,90], since the disk of write-in is usually to carry out piecemeal by 64, if an IO spans the storage of 64 or 64 integral multiple
Boundary address can be then split, then for for the sequential flow for the current cache that storage address is [4,90], in the sequential flow
The I/O data of storage address [4,64] can be performed refresh operation, flush in disk from caching.
Embodiment six
Fig. 9 is referred to, the present embodiment can provide the strategy of the 4th kind of flush buffers on the basis of previous embodiment:
S610, when the I/O data being buffered in the equipment caching to the sequential flow starts to execute refresh operation, root
It is recorded in the renewal time of the fixed length array according to the sequential flow, records the refresh time of the sequential flow；And according to institute
Fixed length array is stated, it is described to judge that the offset address for first I/O data for executing the refresh operation and the sequential flow are recorded in
Whether the offset address in fixed length array is consistent；
S620, if so, the initial value that the refreshing quantity of the sequential flow is arranged is one；
S630, if it is not, the refreshing quantity to the sequential flow adds one；
S640 subtracts one to the refreshing quantity of the sequential flow at the end of the refresh operation；
S650 judges whether the numerical value for refreshing quantity is zero, and judges the refresh time in period threshold value
Inside whether there is update；
S660, if no, all I/O datas being buffered in the equipment caching to the sequential flow execute refresh operation.
In the present embodiment, for some sequential flow, I/O data can be buffered in equipment caching by block successively,
It and in equipment caching is also that one piece one piece of data block flushes in disk or interface, thus the same sequential flow is come
It says, the initial value for refreshing quantity is one, if start to execute a refresh operation of the sequential flow, the refresh operation
The offset address of first I/O data is consistent with the offset address that the sequential flow is recorded in the fixed length array, then explanation should
Refresh operation is first refresh operation of the sequential flow, and the initial value that the refreshing quantity of the sequential flow is arranged is one；If no
Unanimously, then illustrate the refresh operation not and be first refresh operation of the sequential flow but other refresh operations, then at this time
To the refreshing quantity of the sequential flow carry out plus one.And every refresh operation for having executed aforementioned sequence stream, i.e., it is suitable to this
The refreshing quantity of sequence stream subtracts one, to when being zero by the numerical value for the refreshing quantity for determining the sequential flow and every start to hold
When the renewal time that the refresh operation of one sequential flow of row is recorded also does not change, illustrate current order stream probably
Terminated to cache or the quantity of the I/O data of the current order stream in equipment caching or data length do not meet and execute brush
The condition newly operated, but also come in without subsequent I/O data caching.Executing refresh operation at this time can be to avoid a upper sequence
A biggish delay is generated between stream and the refresh operation of next sequential flow, and the time of the delay can be by previous embodiment
Timing threshold value determine.
The refresh process provided through the foregoing embodiment can effectively realize the merging and alignment for realizing IO sequential flow
Boundary address between disk block is written.
Embodiment seven
Referring to Fig. 10, the embodiment of the present invention provides the device of a kind of sequential stream detection and caching, comprising:
Access request receiving module 10 for receiving the access request of I/O data, and records the address letter of the I/O data
It ceases in the skip list of address；
Fixed length array update module 20, for according to the address skip list and the I/O data, IO number described in more new record
According to the attribute information of affiliated sequential flow in fixed length array；Wherein, the attribute information includes the IO quantity of the sequential flow, described
IO quantity indicates that the sequential flow has received but the quantity of uncached I/O data；
Queue depth's judgment module 30, for judging whether the IO quantity meets preset quantity threshold value or the present count
Measure the integral multiple of threshold value；And
Sequential flow cache module 40, for if so, being recorded in the fixed length according to the address skip list and the sequential flow
Attribute information in array, has received the sequential flow but uncached I/O data carries out piecemeal caching；Each piece slow
The quantity for the I/O data deposited is not more than the IO queue depth of the sequential flow.
Further, the address information includes the offset address of the I/O data, and the attribute information further includes described
The offset address of renewal time and the sequential flow of the attribute information of sequential flow in the fixed length array；And
The fixed length array update module includes:
Data correlated judgment unit, for judging whether is history I/O data that the I/O data and the address skip list are recorded
It is related；
First attribute updating unit is used for if so, extracting history relevant to the I/O data from the fixed length array
The attribute information of I/O data offset address having the same and renewal time nearest sequential flow, and according to the attribute information of extraction
With the attribute information of the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；And
Second attribute updating unit, for if it is not, being created belonging to the I/O data according to the address information of the I/O data
Sequential flow attribute information in the fixed length array.
The function of described device can also execute corresponding software realization by hardware realization by hardware.It is described
Hardware or software include one or more modules corresponding with above-mentioned function.
It is described to deposit including processor and memory in the structure of sequential stream detection and caching in a possible design
Reservoir is used to store the side for supporting the device of sequential stream detection and caching to execute sequential stream detection and caching in above-mentioned first aspect
The program of method, the processor is configured to for executing the program stored in the memory.The sequential stream detection and slow
The device deposited can also include communication interface, logical for the device of sequential stream detection and caching and other equipment or communication network
Letter.
Embodiment eight
The embodiment of the present invention also provides a kind of terminal device, and as shown in figure 11, which includes: memory 21 and processor
22, the computer program that can be run on processor 22 is stored in memory 21.Processor 22 executes the computer program
The method of sequential stream detection and caching in Shi Shixian above-described embodiment.The quantity of memory 21 and processor 22 can be one
Or it is multiple.
The equipment further include:
Communication interface 23, for the communication between processor 22 and external equipment.
Memory 21 may include high speed RAM memory, it is also possible to further include nonvolatile memory (non-volatile
Memory), a for example, at least magnetic disk storage.
If memory 21, processor 22 and the independent realization of communication interface 23, memory 21, processor 22 and communication are connect
Mouth 23 can be connected with each other by bus and complete mutual communication.The bus can be industry standard architecture
(ISA, Industry Standard Architecture) bus, external equipment interconnection (PCI, Peripheral
Component) bus or extended industry-standard architecture (EISA, Extended Industry Standard
Component) bus etc..The bus can be divided into address bus, data/address bus, control bus etc..For convenient for expression, Figure 11
In only indicated with a thick line, it is not intended that an only bus or a type of bus.
Optionally, in specific implementation, if memory 21, processor 22 and communication interface 23 are integrated in chip piece
On, then memory 21, processor 22 and communication interface 23 can complete mutual communication by internal interface.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.Moreover, particular features, structures, materials, or characteristics described
It may be combined in any suitable manner in any one or more of the embodiments or examples.In addition, without conflicting with each other, this
The technical staff in field can be by the spy of different embodiments or examples described in this specification and different embodiments or examples
Sign is combined.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic." first " is defined as a result, the feature of " second " can be expressed or hidden
It include at least one this feature containing ground.In the description of the present invention, the meaning of " plurality " is two or more, unless otherwise
Clear specific restriction.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes
It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, to execute function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The instruction fetch of row system, device or equipment and the system executed instruction) it uses, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium ", which can be, any may include, stores, communicates, propagates or pass
Defeated program is for instruction execution system, device or equipment or the dress used in conjunction with these instruction execution systems, device or equipment
It sets.
Computer-readable medium described in the embodiment of the present invention can be computer-readable signal media or computer can
Read storage medium either the two any combination.The more specific example of computer readable storage medium is at least (non-poor
Property list to the greatest extent) include the following: there is the electrical connection section (electronic device) of one or more wirings, portable computer diskette box (magnetic
Device), random access memory (RAM), read-only memory (ROM), erasable edit read-only storage (EPROM or flash
Memory), fiber device and portable read-only memory (CDROM).In addition, computer readable storage medium even can be with
It is the paper or other suitable media that can print described program on it, because can be for example by paper or the progress of other media
Optical scanner is then edited, interpreted or is handled when necessary with other suitable methods and is described electronically to obtain
Program is then stored in computer storage.
In embodiments of the present invention, computer-readable signal media may include in a base band or as carrier wave a part
The data-signal of propagation, wherein carrying computer-readable program code.The data-signal of this propagation can use a variety of
Form, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media is also
It can be any computer-readable medium other than computer readable storage medium, which can send, pass
It broadcasts or transmits for instruction execution system, input method or device use or program in connection.Computer can
The program code for reading to include on medium can transmit with any suitable medium, including but not limited to: wirelessly, electric wire, optical cable, penetrate
Frequently (Radio Frequency, RF) etc. or above-mentioned any appropriate combination.
It should be appreciated that each section of the invention can be realized with hardware, software, firmware or their combination.Above-mentioned
In embodiment, software that multiple steps or method can be executed in memory and by suitable instruction execution system with storage
Or firmware is realized.It, and in another embodiment, can be under well known in the art for example, if realized with hardware
Any one of column technology or their combination are realized: having a logic gates for realizing logic function to data-signal
Discrete logic, with suitable combinational logic gate circuit specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are understood that realize all or part of step that above-described embodiment method carries
It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable storage medium
In matter, which when being executed, includes the steps that one or a combination set of embodiment of the method.
It, can also be in addition, each functional unit in each embodiment of the present invention can integrate in a processing module
It is that each unit physically exists alone, can also be integrated in two or more units in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized and when sold or used as an independent product in the form of software function module, also can store in a computer
In readable storage medium storing program for executing.The storage medium can be read-only memory, disk or CD etc..
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in its various change or replacement,
These should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with the guarantor of the claim
It protects subject to range.
Claims (18)
1. a kind of method of sequential stream detection and caching characterized by comprising
The access request of I/O data is received, and records the address information of the I/O data in the skip list of address；
According to the address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length
Array；Wherein, the attribute information includes the IO quantity of the sequential flow, and the IO quantity indicates that the sequential flow has received
But the quantity of uncached I/O data；
Judge whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value；And
If so, the attribute information in the fixed length array is recorded according to the address skip list and the sequential flow, to described suitable
Sequence stream has received but uncached I/O data carries out piecemeal caching；The quantity of the I/O data of each piece of caching is no more than described suitable
The IO queue depth of sequence stream.
2. the method for sequential stream detection as described in claim 1 and caching, which is characterized in that the address information includes described
The offset address of I/O data, the attribute information further include update of the attribute information of the sequential flow in the fixed length array
The offset address of time and the sequential flow；And
It is described according to the address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record in
Fixed length array, comprising:
Judge whether the I/O data and the address skip list history I/O data recorded are related；
If so, extracted from the fixed length array relevant to I/O data history I/O data offset address having the same and
The attribute information of renewal time nearest sequential flow, and according to the attribute information of extraction and the I/O data, described in more new record
The attribute information of the affiliated sequential flow of I/O data is in fixed length array.
3. the method for sequential stream detection as claimed in claim 2 and caching, which is characterized in that described according to the address skip list
With the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array, comprising:
When the I/O data is uncorrelated to the history I/O data that the address skip list is recorded, believed according to the address of the I/O data
Breath, creates the attribute information of sequential flow belonging to the I/O data in the fixed length array.
4. the method for sequential stream detection as claimed in claim 2 and caching, which is characterized in that the address information includes described
The data length of I/O data；And it is described judge history I/O data that the I/O data is recorded with the address skip list whether phase
It closes, comprising:
According to the storage address for the history I/O data that the storage address section of the I/O data, judgement are recorded with the address skip list
Section is with the presence or absence of overlapping or continuous relationship；Wherein, the storage address section by the I/O data offset address sum number
The section obtained according to length computation；
If so, determining that the I/O data is related to the history I/O data that the address skip list is recorded；And
If it is not, determining that the I/O data and the address skip list history I/O data recorded are uncorrelated.
5. the method for sequential stream detection as claimed in claim 2 and caching, which is characterized in that the attribute information further includes number
According to length；And
The attribute information and the I/O data according to extraction, the attribute information of the affiliated sequential flow of I/O data described in more new record
In fixed length array, comprising:
One is added to the IO quantity in the attribute information of extraction, and the affiliated sequential flow of the I/O data is updated with the numerical value after adding one
IO quantity；
It is updated belonging to the I/O data with the sum of the data length in the data length of the I/O data and the attribute information of extraction
The data length of sequential flow；
The renewal time of the affiliated sequential flow of the I/O data is updated with current time；And
The attribute information of the updated affiliated sequential flow of the I/O data is recorded in the fixed length array.
6. the method for sequential stream detection as claimed in claim 3 and caching, which is characterized in that the attribute information further includes number
According to length, the address information according to the I/O data creates the attribute information of sequential flow belonging to the I/O data in institute
State fixed length array, comprising:
Using numerical value one as the IO quantity of the affiliated sequential flow of the I/O data；
Using the data length of the I/O data as the data length of the affiliated sequential flow of the I/O data；
Using current time as the renewal time of the affiliated sequential flow of the I/O data；
Using the offset address of the I/O data as the offset address of the affiliated sequential flow of the I/O data；And the institute that record generates
The attribute information of the affiliated sequential flow of I/O data is stated in the fixed length array.
7. such as the method for sequential stream detection described in claim 5 or 6 and caching, which is characterized in that the fixed length array includes
Leader is known；And the attribute information of the affiliated sequential flow of the I/O data is recorded in the fixed length array, comprising:
The attribute information for recording the affiliated sequential flow of the I/O data is directed toward in the fixed length array in the address that the leader is known
Position；And
The cyclic address change that the leader is known.
8. the method for sequential stream detection as claimed in claim 2 and caching, which is characterized in that further include:
When the I/O data and the uncorrelated address skip list history I/O data recorded, monitor the I/O data whether by
It is buffered in equipment caching and starts to be performed refresh operation in equipment caching；The refresh operation is included in described
The I/O data is removed in equipment caching and the I/O data is written to the operation of disk；
If so, the execution state for recording sequential flow belonging to the I/O data is unfinished, and record the concurrent of the sequential flow
The initial value of quantity is one；And
When the I/O data, which has executed, completes the refresh operation, the execution state of the sequential flow is recorded to complete, and
The IO queue depth is updated according to the numerical value of the concurrent quantity.
9. the method for sequential stream detection as claimed in claim 8 and caching, which is characterized in that further include:
When the I/O data is related to the history I/O data that the address skip list is recorded, sequence belonging to the I/O data is judged
Whether the execution state of stream is unfinished；If so, adding one to the concurrent quantity.
10. the method for sequential stream detection as claimed in claim 8 and caching, which is characterized in that described according to the number of concurrent
The numerical value of amount updates the IO queue depth, comprising:
Judge whether the numerical value of the concurrent quantity is greater than the numerical value of current IO queue depth；And
If so, using the numerical value of the concurrent quantity as the numerical value of the updated IO queue depth.
11. the method for sequential stream detection as claimed in claim 2 and caching, which is characterized in that the method also includes:
When the I/O data of the sequential flow is cached in equipment caching, inquired in the equipment caching according to the address skip list
With the presence or absence of I/O data relevant to the sequential flow；
If nothing, timing is started to the sequential flow；
It is clear to the clocking value when the I/O data being buffered in the equipment caching to the sequential flow executes refresh operation
Zero；
Judge whether the corresponding clocking value of the sequential flow reaches timing threshold value；And
When the clocking value reaches the timing threshold value, all IO numbers in the equipment caching are buffered in the sequential flow
It is clear according to execution refresh operation, and all information that the sequential flow is recorded in the address skip list and the fixed length array
It removes.
12. the method for sequential stream detection as claimed in claim 8 and caching, which is characterized in that largest buffered length is IO number
According to mean data length and the IO queue depth product；The setting condition of the largest buffered quantity of the equipment caching
Are as follows: no more than the maximum quantity of I/O data that can be cached in equipment caching and no more than the IO queue depth；And
The method also includes:
Judge whether the quantity for the I/O data that the sequential flow is buffered in the equipment caching reaches the largest buffered quantity,
And judge whether the total data length for the I/O data that the sequential flow is buffered in the equipment caching reaches described maximum slow
Deposit length；And
When the quantity reaches the largest buffered quantity or the total data length reaches the largest buffered length, to institute
It states all I/O datas that sequential flow is buffered in the equipment caching and executes refresh operation.
13. the method for sequential stream detection as claimed in claim 9 and caching, which is characterized in that the method also includes:
According to the address skip list and the fixed length array, depositing for the I/O data that the sequential flow is buffered in equipment caching is judged
Store up the memory block whether address crosses over storage equipment；And
If so, to the sequential flow be buffered in equipment caching and storage address be across memory block boundary address it
Preceding all I/O datas execute refresh operation.
14. such as the method for claim 11~13 described in any item sequential stream detections and caching, which is characterized in that the method
Further include:
When the I/O data being buffered in the equipment caching to the sequential flow starts to execute refresh operation, according to the sequence
Stream is recorded in the renewal time of the fixed length array, records the refresh time of the sequential flow；And according to the fixed length array,
Judgement executes the offset address of first I/O data of the refresh operation and the sequential flow is recorded in the fixed length array
Offset address it is whether consistent；
If so, the initial value that the refreshing quantity of the sequential flow is arranged is one；
If it is not, the refreshing quantity to the sequential flow adds one；
At the end of the refresh operation, subtract one to the refreshing quantity of the sequential flow；
Judge whether the numerical value for refreshing quantity is zero, and judges whether the refresh time has more in period threshold value
Newly；
If no, all I/O datas being buffered in the equipment caching to the sequential flow execute refresh operation.
15. the device of a kind of sequential stream detection and caching characterized by comprising
Access request receiving module for receiving the access request of I/O data, and records the address information of the I/O data in ground
In the skip list of location；
Fixed length array update module, for according to the address skip list and the I/O data, belonging to I/O data described in more new record
The attribute information of sequential flow is in fixed length array；Wherein, the attribute information includes the IO quantity of the sequential flow, the IO quantity
Indicate that the sequential flow has received but the quantity of uncached I/O data；
Queue depth's judgment module, for judging whether the IO quantity meets preset quantity threshold value or the preset quantity threshold value
Integral multiple；And
Sequential flow cache module, for if so, being recorded in the fixed length array according to the address skip list and the sequential flow
Attribute information, the sequential flow is had received but uncached I/O data carries out piecemeal caching；The IO number of each piece of caching
According to quantity be not more than the sequential flow IO queue depth.
16. the device of sequential stream detection as claimed in claim 15 and caching, which is characterized in that the address information includes institute
State the offset address of I/O data, the attribute information further include the sequential flow attribute information in the fixed length array more
The offset address of new time and the sequential flow；And
The fixed length array update module includes:
Data correlated judgment unit, for judge history I/O data that the I/O data is recorded with the address skip list whether phase
It closes；
First attribute updating unit is used for if so, extracting history IO number relevant to the I/O data from the fixed length array
According to offset address having the same and the attribute information of renewal time nearest sequential flow, and according to the attribute information of extraction and institute
I/O data is stated, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；And
Second attribute updating unit, for if it is not, being created suitable belonging to the I/O data according to the address information of the I/O data
The attribute information of sequence stream is in the fixed length array.
17. a kind of terminal device for realizing sequential stream detection and caching, which is characterized in that the terminal device includes:
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are executed by one or more of processors, so that one or more of processors
The method for realizing the sequential stream detection and caching as described in any in claim 1-14.
18. a kind of computer readable storage medium, is stored with computer program, which is characterized in that the program is held by processor
The method of the sequential stream detection and caching as described in any in claim 1-14 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810068469.7A CN108182151B (en) | 2018-01-24 | 2018-01-24 | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810068469.7A CN108182151B (en) | 2018-01-24 | 2018-01-24 | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108182151A CN108182151A (en) | 2018-06-19 |
| CN108182151B true CN108182151B (en) | 2019-02-01 |
Family
ID=62551376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810068469.7A Active CN108182151B (en) | 2018-01-24 | 2018-01-24 | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108182151B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274550B (en) * | 2018-09-07 | 2020-07-31 | 电信科学技术第五研究所有限公司 | iSCSI self-adaptive IO queue depth matching method |
| CN113923506B (en) * | 2020-07-10 | 2023-07-18 | 成都鼎桥通信技术有限公司 | Video data processing method, device, edge computing gateway and storage medium |
| KR102568662B1 (en) * | 2021-01-22 | 2023-08-22 | 성균관대학교산학협력단 | Zipper compaction method and apparatus for compacting the plural of skiplists |
| CN115543937B (en) * | 2022-03-22 | 2023-07-11 | 荣耀终端有限公司 | File defragmentation method and electronic device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023931B (en) * | 2010-12-17 | 2015-02-04 | 曙光信息产业(北京)有限公司 | Self-adaption cache pre-fetching method |
| CN102298508B (en) * | 2011-09-07 | 2014-08-06 | 记忆科技(深圳)有限公司 | Stream-based method and device for prereading solid state disk |
| CN103927136B (en) * | 2014-04-21 | 2017-02-22 | 华为技术有限公司 | Identification method and device for input and output IO types |
| CN106294211B (en) * | 2016-08-08 | 2019-05-28 | 浪潮(北京)电子信息产业有限公司 | A kind of detection method and device of multichannel sequential flow |
| CN106557431B (en) * | 2016-11-25 | 2020-07-24 | 苏州浪潮智能科技有限公司 | Pre-reading method and device for multi-path sequential stream |
| CN107589911A (en) * | 2017-09-05 | 2018-01-16 | 郑州云海信息技术有限公司 | A kind of I O process method and device of SSD cachings |
-
2018
- 2018-01-24 CN CN201810068469.7A patent/CN108182151B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108182151A (en) | 2018-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108182151B (en) | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection | |
| CN104335175B (en) | The method and system of thread is identified and migrated between system node based on system performance metric | |
| CN104699422B (en) | Data cached determination method and device | |
| CN101730885A (en) | Cache for a multi thread and multi core system and methods thereof | |
| WO2011031355A1 (en) | Cache prefill on thread migration | |
| CN106547476A (en) | For the method and apparatus of data-storage system | |
| TWI827754B (en) | Mispredict recovery apparatus and method for recovering from mispredicts | |
| CN106546960B (en) | A kind of real-time acquisition analysis system of Radar IF Echo multi-mode and method | |
| CN108920096A (en) | A kind of data storage method of SSD, device, computer equipment and storage medium | |
| US6647463B2 (en) | Cache update method and cache update control system employing non-blocking type cache | |
| CN103927136B (en) | Identification method and device for input and output IO types | |
| CN101639763A (en) | IO dispatching method and device | |
| CN109426623A (en) | A kind of method and device reading data | |
| CN107870874A (en) | A kind of data write-in control method and storage device | |
| CN101840310B (en) | Data read-write method and disk array system using same | |
| CN106445472B (en) | A kind of character manipulation accelerated method, device, chip, processor | |
| CN104991745B (en) | A kind of memory system data wiring method and system | |
| CN117472815A (en) | Storage module conversion interface under AXI protocol and conversion method thereof | |
| CN103870204B (en) | Data write-in and read method, cache controllers in a kind of cache | |
| CN110597482B (en) | Method for searching latest effective data packet in FIFO by serial port | |
| CN101594305A (en) | A kind of message processing method and device | |
| JP2013041414A (en) | Storage control system and method, and replacement system and method | |
| CN115712388A (en) | Data storage method, device and equipment of solid-state disk and storage medium | |
| CN112395011B (en) | Method for returning command response information, return control device and electronic equipment | |
| CN112767978B (en) | DDR command scheduling method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |