CN105677756A - Method and apparatus for effectively using cache in file system - Google Patents
Method and apparatus for effectively using cache in file system Download PDFInfo
- Publication number
- CN105677756A CN105677756A CN201511020378.9A CN201511020378A CN105677756A CN 105677756 A CN105677756 A CN 105677756A CN 201511020378 A CN201511020378 A CN 201511020378A CN 105677756 A CN105677756 A CN 105677756A
- Authority
- CN
- China
- Prior art keywords
- page
- page request
- management system
- request
- file system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and an apparatus for effectively using cache in a file system. The method comprises: classifying page management systems of the file system; in accordance with the classified management systems, acquiring page requests of the file system; determining whether acquiring page requests are successful, and if successful, executing page requests, and if not successful, recycling pages having same attributes in a cache space so as to execute the page requests. The method and apparatus of the invention can effectively manage page cache of the file system, avoid recycling of effective cached data caused by fight for cache among read and write operation, and increase effective utility rate of cache.
Description
Technical field
The present invention relates to the technical field utilizing buffer memory, particularly relate to the method and device that effectively utilize buffer memory in a kind of file system.
Background technology
The page management of current file system is that the overall situation is unique, and when system call internal memory is not enough, that can trigger dirty page cache returns the recovery of brush with clean page cache. Return after the dirty page after brush turns into the clean page and can be recovered, it is also possible to discharge the clean page by file system, give system again to call.
Internal memory in order to avoid the reasons such as disk hardware fault to cause is not enough, file system often limits to be write buffer memory and can not exceed some amount (in CENTOS6.5 acquiescence be the 40% of total amount), but the management of the page of above-mentioned file system then can exist certain problem.
For write operation requests, when write operation triggers cache garbage collection, by the cache garbage collection that reading before operates, but this batch of page can not be used because of the quantity limitation of the page of write operation, causing the invalid of recovery, the page that originally may be used for reading buffer memory is by the unmanned use of recovery in vain. Reduce the cache hit of reading operation. For reading operation, under most scene, the priority of read request is often higher than write request, and it is unacceptable that reading buffer memory is disabled. When reading operation triggers cache garbage collection, compulsory buffer memory before, the dirty page that do not reach back brush condition being returned and brushed to disk, the cache hit rate reducing write operation is polymerized with writing, and write operation is very unfriendly by this.
Summary of the invention
For above the deficiencies in the prior art, the present invention provides the method and device that effectively utilize buffer memory in a kind of file system.
On the one hand, embodiments provide a kind of method effectively utilizing buffer memory in file system, comprising:
The page management system of file system is classified;
The page request of system is obtained according to the management system of classification;
Whether successful judge to obtain page request:
If success, perform page request,
Otherwise, spatial cache reclaims the page of same alike result and then performs page request.
Can selection of land, the described page management system by file system classification comprise:
The page management system of file system is divided into and reads operating system and write operation system.
Can selection of land, described according to classification management system obtain page request comprise:
If page management system is for reading operating system, obtains and read page request;
If page management system is write operation system, obtains and write page request.
Can selection of land, described in spatial cache, reclaim same alike result the page so that perform page request and comprise:
If reading page request, when obtaining page request failure, only reclaim reading in the buffer and operate the relevant page;
If writing page request, when obtaining page request failure, only reclaim the relevant page of write operation in the buffer.
Can selection of land, described also comprised before only reclaiming the relevant page of write operation in the buffer: return the dirty page of brush.
In yet another aspect, embodiments providing the device effectively utilizing buffer memory in a kind of file system, this device comprises:
Page management classification module, for classifying the page management system of file system;
Obtain page request module, for obtaining corresponding page request according to the management system of classification;
Judge execution module, whether successful for judging to obtain page request:
If success, perform page request,
Otherwise, spatial cache reclaims the page of same alike result and then performs page request.
Can selection of land, described page management classification module comprises:
Reading operation taxon, for being divided into reading operating system by the page management system of file system;
Write operation taxon, for being divided into write operation system by the page management system of file system.
Can selection of land, described acquisition page request module comprises:
Obtain and read page request unit, for when page management system is for reading operating system, obtaining and read page request;
Page request unit is write in acquisition, for when page management system is write operation system, obtaining and write page request.
Can selection of land, described judgement execution module comprises:
Read page actuator unit, when reading page request failure for obtaining, only reclaim reading in the buffer and operate the relevant page;
Write page actuator unit, when writing page request failure for obtaining, only reclaim the relevant page of write operation in the buffer.
Can selection of land, described in write page actuator unit and also comprise:
Return the dirty units of pages of brush, for returning the dirty page of brush before only reclaiming the relevant page of write operation in the buffer.
Useful effect is as follows:
The present invention provides the method and device that effectively utilize buffer memory in a kind of file system, the method and device can the page caches of effective managing file system, avoid between read-write operation, fighting for buffer memory and cause effectively data cached being recovered, it is to increase the effective rate of utilization of buffer memory.
Accompanying drawing explanation
Specific embodiments of the invention are described below with reference to accompanying drawings, wherein:
Fig. 1 shows the method flow schematic diagram effectively utilizing buffer memory in the embodiment of the present invention in a kind of file system;
Fig. 2 shows and reads page request method schema in the embodiment of the present invention;
Fig. 3 shows and writes page request method embodiment 1 schema in the embodiment of the present invention;
Fig. 4 shows and writes page request method embodiment 2 schema in the embodiment of the present invention;
Fig. 5 shows the apparatus structure schematic diagram effectively utilizing buffer memory in the embodiment of the present invention in a kind of file system;
Fig. 6 shows page management classification modular structure schematic diagram;
Fig. 7 shows and obtains page request modular structure schematic diagram;
Fig. 8 shows and judges execution module structural representation.
Embodiment
Technical scheme and advantage in order to make the present invention are clearly understood, below in conjunction with accompanying drawing, the exemplary embodiment of the present invention is described in more detail, obviously, described embodiment is only a part of embodiment of the present invention, instead of all embodiments is exhaustive. And when not conflicting, the embodiment in this explanation and the feature in embodiment can be combined with each other.
Contriver notices in invention process: the read-write operation of file system exists the insufficient problem of the utilization of resources, when reading operation triggers cache garbage collection, compulsory buffer memory before, the dirty page that do not reach back brush condition can be returned and brush to disk, the cache hit rate reducing write operation is polymerized with writing, and is very unfriendly for write operation.And when cache garbage collection is triggered for write operation, by the cache garbage collection that reading before operates, but this batch of page can not be used because of the quantity limitation of the page of write operation, cause the invalid of recovery, originally may be used for reading the page of buffer memory by the unmanned use of recovery in vain, reduce the cache hit of reading operation.
For above-mentioned deficiency, the embodiment of the present invention proposes the method and device that effectively utilize buffer memory in a kind of file system, is described below.
Fig. 1 shows the method effectively utilizing buffer memory in a kind of file system of the embodiment of the present invention, as shown in the figure, comprising:
Step 101, the page management system of file system is classified;
Step 102, the page request obtaining system according to the management system of classification;
Step 103, judge to obtain page request whether success:
If success, perform page request,
Otherwise, spatial cache reclaims the page of same alike result and then performs page request.
In specific implementation process, the page management structure of file system is divided into the reading page management system of physical isolation and writes page management system, read and write page request is assignment page in respective page management system only, when the page distributes unsuccessfully, only reclaiming the page of same alike result in spatial cache, the method can be avoided fighting for buffer memory between read-write operation and cause effectively data cached being recovered.
Hereinafter to reading page request, page request operation illustrates respectively with writing respectively.
Being illustrated in figure 2 and read page request method schema, as shown in the figure, method comprises:
Step 201, call reading operating system;
Step 202, acquisition read page request accordingly;
Whether step 203, judgement obtain the page successful;
If step 204 obtains reads page request success, then continue data operation.
If step 205 obtains reads page request failure, then only reclaim the relevant page of reading operation in the buffer.
In specific implementation process, if when obtaining reading page request failure, when also namely reading operation internal memory is not enough, from file system cache, only reclaims the page that reading page management structure is distributed, do not return the dirty page of brush, also do not reclaim the page write page management structure and distribute.
Being illustrated in figure 3 and write page request method embodiment 1 schema, as shown in the figure, method comprises:
Step 301, call write operation system;
Step 302, acquisition write page request accordingly;
Step 303, judge to obtain page request whether success;
If step 304 obtains writes page request success, then continue data operation.
If step 305 obtains writes page request failure, then only reclaim the relevant page of write operation in the buffer.
In concrete real-time process, when write operation internal memory is not enough, it is necessary to return brush dirty data before reclaiming the relevant page of write operation, from file system cache, reclaim the page write page management structure and distribute, do not attempt back the dirty page of brush, do not reclaim the page write page management structure and distribute yet. Being illustrated in figure 4 and write page request method embodiment 2 schema, method comprises:
Step 401, call write operation system;
Step 402, acquisition write page request accordingly;
Step 403, judge to obtain page request whether success;
If step 404 obtains writes page request success, then continue data operation.
If step 405 obtains writes page request failure, then first return the dirty page of brush, only reclaim the relevant page of write operation in the buffer." returning the dirty page of brush " herein and refer to that file system organizes buffer memory in units of the page, first data are write in the page by write request, and now page marks is " dirty ", represent that stored data and data in magnetic disk are inconsistent. When data write to disk from the page, remove dirty mark.
Useful effect:
The method can the page cache of effective managing file system, avoid between read-write operation, fighting for buffer memory and cause effectively data cached being recovered, it is to increase the effective rate of utilization of buffer memory.
Conceive based on same invention, the embodiment of the present invention additionally provides the device effectively utilizing buffer memory in a kind of file system, the principle dealt with problems due to these equipment is similar to effectively utilizing the method for buffer memory in a kind of file system, therefore the enforcement of these equipment see the enforcement of method, can repeat part and repeat no more.
As shown in Figure 5, device can comprise:
Page management classification module 501, for classifying the page management system of file system;
Obtain page request module 502, for obtaining corresponding page request according to the management system of classification;
Judge execution module 503, whether successful for judging to obtain page request:
If success, perform page request,
Otherwise, spatial cache reclaims the page of same alike result and then performs page request.
Can selection of land, as shown in Figure 6, described page management classification module comprises:
Reading operation taxon 5011, for being divided into reading operating system by the page management system of file system;
Write operation taxon 5012, for being divided into write operation system by the page management system of file system.
Can selection of land, as shown in Figure 7, described acquisition page request module comprises:
Obtain and read page request unit 5021, for when page management system is for reading operating system, obtaining and read page request;
Page request unit 5021 is write in acquisition, for when page management system is write operation system, obtaining and write page request.
Can selection of land, as shown in Figure 8, described judgement execution module comprises:
Read page actuator unit 5031, when reading page request failure for obtaining, only reclaim reading in the buffer and operate the relevant page;
Write page actuator unit 5032, when writing page request failure for obtaining, only reclaim the relevant page of write operation in the buffer.
Can selection of land, described in write page actuator unit and also comprise:
Return the dirty units of pages of brush, for returning the dirty page of brush before only reclaiming the relevant page of write operation in the buffer.
In above-described embodiment, existing Functional Unit device blocks all can be adopted to implement. Such as, processing module can adopt existing data processing components and parts, at least, the location-server adopted just possesses realize this Functional Unit device in existing location technology; As for receiver module, then it is the components and parts that any one possesses the equipment of signal transfer functions and all possesses; Meanwhile, what A, n parameter calculating that processing module carries out, intensity adjustment etc. adopted is all existing technique means, and those skilled in the art design and develop can realize through accordingly.
For convenience of description, each several part of the above device is divided into various module or unit to describe respectively with function. Certainly, the function of each module or unit can be realized in same or multiple software or hardware when implementing of the present invention.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, system or computer program.Therefore, the present invention can adopt the form of complete hardware embodiment, completely software implementation or the embodiment in conjunction with software and hardware aspect. And, the present invention can adopt the form at one or more upper computer program implemented of computer-usable storage medium (including but not limited to multiple head unit, CD-ROM, optical memory etc.) wherein including computer usable program code.
The present invention is that schema and/or skeleton diagram with reference to method according to embodiments of the present invention, equipment (system) and computer program describe. Should understand can by the combination of the flow process in each flow process in computer program instructions flowchart and/or skeleton diagram and/or square frame and schema and/or skeleton diagram and/or square frame. These computer program instructions can be provided to the treater of multi-purpose computer, special purpose computer, Embedded Processor or other programmable data processing device to produce a machine so that the instruction performed by the treater of computer or other programmable data processing device is produced for realizing the device of function specified in schema flow process or multiple flow process and/or skeleton diagram square frame or multiple square frame.
These computer program instructions also can be stored in and can guide in computer-readable memory that computer or other programmable data processing device work in a specific way, making the instruction that is stored in this computer-readable memory produce the manufacture comprising instruction device, this instruction device realizes the function specified in schema flow process or multiple flow process and/or skeleton diagram square frame or multiple square frame.
These computer program instructions also can be loaded in computer or other programmable data processing device, make on computer or other programmable devices, to perform a series of operation steps to produce computer implemented process, thus the instruction performed on computer or other programmable devices is provided for realizing the step of the function specified in schema flow process or multiple flow process and/or skeleton diagram square frame or multiple square frame.
Although having described the preferred embodiments of the present invention, but those skilled in the art once the substantially creative concept of cicada, then these embodiments can be made other change and amendment. Therefore, it is intended that the appended claims shall be construed comprise preferred embodiment and fall into all changes and the amendment of the scope of the invention.
Claims (10)
1. a file system effectively utilizes the method for buffer memory, it is characterised in that, the method comprises:
The page management system of file system is classified;
The page request of system is obtained according to the management system of classification;
Whether successful judge to obtain page request:
If success, perform page request,
Otherwise, spatial cache reclaims the page of same alike result and then performs page request.
2. the method for claim 1, it is characterised in that, the management system classification of the described page by file system comprises:
The page management system of file system is divided into the reading management system of physical isolation and writes management system.
3. method as claimed in claim 2, it is characterised in that, the described management system according to classification obtains page request and comprises:
If page management system is for reading management system, obtains and read page request;
If page management system is for writing management system, obtains and write page request.
4. method as claimed in claim 3, it is characterised in that, described in spatial cache, reclaim the page of same alike result and then perform page request comprise:
If reading page request, when obtaining page request failure, only reclaim reading in the buffer and operate the relevant page;
If writing page request, when obtaining page request failure, only reclaim the relevant page of write operation in the buffer.
5. method as claimed in claim 4, it is characterised in that, described also comprised before only reclaiming the relevant page of write operation in the buffer: return the dirty page of brush.
6. a file system effectively utilizes the device of buffer memory, it is characterised in that, this device comprises:
Page management classification module, for classifying the page management system of file system;
Obtain page request module, for obtaining corresponding page request according to the management system of classification;
Judge execution module, whether successful for judging to obtain page request:
If success, perform page request,
Otherwise, spatial cache reclaims the page of same alike result and then performs page request.
7. device as claimed in claim 6, it is characterised in that, described page management classification module comprises:
Reading operation taxon, for being divided into reading operating system by the page management system of file system;
Write operation taxon, for being divided into write operation system by the page management system of file system.
8. device as claimed in claim 7, it is characterised in that, described acquisition page request module comprises:
Obtain and read page request unit, for when page management system is for reading operating system, obtaining and read page request;
Page request unit is write in acquisition, for when page management system is write operation system, obtaining and write page request.
9. device as claimed in claim 8, it is characterised in that, described judgement execution module comprises:
Read page actuator unit, when reading page request failure for obtaining, only reclaim reading in the buffer and operate the relevant page;
Write page actuator unit, when writing page request failure for obtaining, only reclaim the relevant page of write operation in the buffer.
10. device as claimed in claim 9, it is characterised in that, described in write page actuator unit and also comprise:
Return the dirty units of pages of brush, for returning the dirty page of brush before only reclaiming the relevant page of write operation in the buffer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511020378.9A CN105677756A (en) | 2015-12-28 | 2015-12-28 | Method and apparatus for effectively using cache in file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511020378.9A CN105677756A (en) | 2015-12-28 | 2015-12-28 | Method and apparatus for effectively using cache in file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105677756A true CN105677756A (en) | 2016-06-15 |
Family
ID=56298044
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511020378.9A Pending CN105677756A (en) | 2015-12-28 | 2015-12-28 | Method and apparatus for effectively using cache in file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105677756A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111562983A (en) * | 2020-04-30 | 2020-08-21 | Oppo（重庆）智能科技有限公司 | Memory optimization method and device, electronic equipment and storage medium |
| CN113326214A (en) * | 2021-06-16 | 2021-08-31 | 统信软件技术有限公司 | Page cache management method, computing device and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102411541A (en) * | 2010-10-13 | 2012-04-11 | 微软公司 | Dynamic cache configuration using separate read and write caches |
| CN102521157A (en) * | 2011-12-13 | 2012-06-27 | 曙光信息产业(北京)有限公司 | System and method for realizing onboard storage resource management on FPGA (Field Programmable Gate Array) |
| CN103246611A (en) * | 2012-02-07 | 2013-08-14 | 腾讯科技（深圳）有限公司 | File processing method and system |
| US20130282964A1 (en) * | 2010-05-05 | 2013-10-24 | Microsoft Corporation | Flash memory cache including for use with persistent key-value store |
| CN104899154A (en) * | 2015-06-10 | 2015-09-09 | 山东大学 | Page management method based on embedded system mixed main memory |
| CN105095495A (en) * | 2015-08-21 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Distributed file system cache management method and system |
-
2015
- 2015-12-28 CN CN201511020378.9A patent/CN105677756A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130282964A1 (en) * | 2010-05-05 | 2013-10-24 | Microsoft Corporation | Flash memory cache including for use with persistent key-value store |
| CN102411541A (en) * | 2010-10-13 | 2012-04-11 | 微软公司 | Dynamic cache configuration using separate read and write caches |
| CN102521157A (en) * | 2011-12-13 | 2012-06-27 | 曙光信息产业(北京)有限公司 | System and method for realizing onboard storage resource management on FPGA (Field Programmable Gate Array) |
| CN103246611A (en) * | 2012-02-07 | 2013-08-14 | 腾讯科技（深圳）有限公司 | File processing method and system |
| CN104899154A (en) * | 2015-06-10 | 2015-09-09 | 山东大学 | Page management method based on embedded system mixed main memory |
| CN105095495A (en) * | 2015-08-21 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Distributed file system cache management method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111562983A (en) * | 2020-04-30 | 2020-08-21 | Oppo（重庆）智能科技有限公司 | Memory optimization method and device, electronic equipment and storage medium |
| CN113326214A (en) * | 2021-06-16 | 2021-08-31 | 统信软件技术有限公司 | Page cache management method, computing device and readable storage medium |
| CN113326214B (en) * | 2021-06-16 | 2023-06-16 | 统信软件技术有限公司 | Page cache management method, computing device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11061572B2 (en) | Memory object tagged memory monitoring method and system | |
| CN109697016B (en) | Method and apparatus for improving storage performance of containers | |
| CN110196818B (en) | Data caching method, caching device and storage system | |
| CN105677580A (en) | Method and device for accessing cache | |
| CN105868122A (en) | Data processing method and device for quick flashing storage equipment | |
| CN102576333A (en) | Data caching in non-volatile memory | |
| CN104301360A (en) | Method, log server and system for recording log data | |
| CN102782683A (en) | Buffer pool extension for database server | |
| CN107357650A (en) | A kind of automatic method for releasing of internal memory based on picture resource and system | |
| CN105446814A (en) | Cache recovery method and device | |
| CN103412825B (en) | A kind of memory release method and device | |
| CN105912425B (en) | Snapshot image foundation, rollback method and device | |
| CN111045956A (en) | Solid state disk garbage recycling method and device based on multi-core CPU | |
| CN104423982A (en) | Request processing method and device | |
| CN107817945A (en) | A kind of method for reading data and system for mixing internal storage structure | |
| CN103617260A (en) | Index generation method and device for repeated data deletion | |
| CN111831691B (en) | Data reading and writing method and device, electronic equipment and storage medium | |
| CN105677756A (en) | Method and apparatus for effectively using cache in file system | |
| GB2502076A (en) | Managing memory in a computer system | |
| CN104462940A (en) | Monitoring method and device for computer USB interface | |
| CN103699435A (en) | Load balancing method and device | |
| CN111858612B (en) | Data accelerated access method and device based on graph database and storage medium | |
| CN101587457B (en) | Adaptive cache organization for chip multiprocessors | |
| CN110865869B (en) | Method and system for extracting memory access characteristics of virtual machine on Shenwei architecture | |
| CN106202456B (en) | Send the method and device of picture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160615 |