CN109086220A - A kind of method and apparatus recycling memory space - Google Patents
A kind of method and apparatus recycling memory space Download PDFInfo
- Publication number
- CN109086220A CN109086220A CN201810643419.7A CN201810643419A CN109086220A CN 109086220 A CN109086220 A CN 109086220A CN 201810643419 A CN201810643419 A CN 201810643419A CN 109086220 A CN109086220 A CN 109086220A
- Authority
- CN
- China
- Prior art keywords
- file
- recycling
- posix
- memory space
- small documents
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a kind of method and apparatus for recycling memory space, method includes: to obtain portable operating system interface POSIX file to be processed, when receiving deletion instruction, first object small documents in POSIX file are identified as and have been deleted, and recycle the memory space of first object small documents, obtain pre- recycling POSIX file, when recycling POSIX file meets default recycling condition in advance, for each small documents in pre- recycling POSIX file, judge whether the small documents are identified as to have deleted, if the small documents are not identified as having deleted, the small documents are then copied into default memory space, and recycle the memory space of the small documents.Based on above-mentioned processing, it can merge and delete the file space that small documents generate, reduce the quantity in file cavity, and then reduce the influence for deleting small documents to the system performance of server.
Description
Technical field
The present invention relates to technical field of data storage, more particularly to a kind of method and apparatus for recycling memory space.
Background technique
With the high speed development of internet, it is literary that picture and document of enormous amount (hundreds of millions of) etc. is produced in internet
Part.These files are usually smaller (generally in 100KB or less), to can be described as small documents.The method for merging storage is by enormous amount
The contents of small documents be stored in POSIX (Portable Operating System Interface of UNIX, portable
Operating system interface) file.The common practice of these small documents memory spaces is using Copy- in current recycling POSIX file
Commit (duplication-submission) method.It the Ambry of (neck English) and washes in a pan for example, the Haystack of Facebook (facebook), LinkedIn
Precious TFS (Taobao File System, Taobao's file system).
In the prior art, after the content of the small documents of enormous amount is stored in POSIX file by server, it is also necessary to record
The index information, is then added in index file by small documents offset in big file, that is, index information.Work as service
When device receives the removal request for a certain small documents, can according to the index information of the small documents, by the small documents from
It is deleted in POSIX file, and recycles memory space shared by the small documents.
However, inventor has found in the implementation of the present invention, at least there are the following problems for the prior art:
Since when deleting each small documents, the corresponding memory space of the small documents will become file cavity.Work as deletion
In POSIX file when more small documents, will lead in POSIX file there are more file cavity.Server needs to distribute
Each file of system material resource maintenance cavity, and then the system performance of server can be reduced.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of method and apparatus for recycling memory space, can reduce to service
The influence of the system performance of device.Specific technical solution is as follows:
In a first aspect, in order to achieve the above objectives, it is described the embodiment of the invention discloses a kind of method for recycling memory space
Method includes:
Obtain portable operating system interface POSIX file to be processed；
When receiving deletion instruction, first object small documents in the POSIX file are identified as and have been deleted, and are recycled
The memory space of the first object small documents obtains pre- recycling POSIX file；
When the pre- recycling POSIX file meets default recycling condition, for every in the pre- recycling POSIX file
A small documents, judge whether the small documents are identified as to have deleted；
If the small documents are not identified as having deleted, which is copied into default memory space, and recycling should
The memory space of small documents.
Optionally, each small documents in the pre- recycling POSIX file, judge whether the small documents are marked
Know to have deleted, comprising:
The pre- recycling POSIX file is segmented according to preset mode, obtains multiple file sections, the file section
Including multiple small documents；
For each file section in the multiple file section, the empty density of this document section is calculated, the cavity is close
Degree indicates the ratio of the memory space being recovered in this document section and this document section memory space；
According to the empty density of each file section in the multiple file section, according to the size of the empty density to institute
Multiple file sections are stated to be ranked up；
According to the corresponding sequence of the multiple file section, the small documents that file section to be processed includes are obtained, for acquisition
The each small documents arrived, judge whether the small documents are identified as to have deleted.
Optionally, the pre- recycling POSIX file meets default recycling condition, comprising:
The read-write delay of the pre- recycling POSIX file is greater than preset threshold；
Alternatively,
Reach the preset recycling moment.
Optionally, described to be segmented the pre- recycling POSIX file according to preset mode, multiple file sections are obtained, are wrapped
It includes:
The corresponding relationship of memory space and segmented mode that pre- recycling POSIX file according to the pre-stored data occupies, determines
The corresponding segmented mode of the pre- recycling POSIX file, and according to the segmented mode determined to the pre- recycling POSIX file
It is segmented, obtains multiple file sections.
Optionally, pair of the pre- recycling POSIX file according to the pre-stored data occupies memory space and segmented mode
It should be related to, determine the corresponding segmented mode of the pre- recycling POSIX file, and according to the segmented mode determined to described pre- time
It receives POSIX file to be segmented, obtains multiple file sections, comprising:
When the memory space that the pre- recycling POSIX file occupies is in the first default memory space ranges, according to pre-
If file section memory space the pre- recycling POSIX file is segmented, obtain multiple file sections.
Optionally, pair of the pre- recycling POSIX file according to the pre-stored data occupies memory space and segmented mode
It should be related to, determine the corresponding segmented mode of the pre- recycling POSIX file, and according to the segmented mode determined to described pre- time
It receives POSIX file to be segmented, obtains multiple file sections, comprising:
When the memory space that the pre- recycling POSIX file occupies is in the second default memory space ranges, according to pre-
If file section number the pre- recycling POSIX file is segmented, obtain multiple file sections.
Second aspect, in order to achieve the above object, the embodiment of the invention provides a kind of device for recycling memory space, institutes
Stating device includes:
Module is obtained, for obtaining portable operating system interface POSIX file to be processed；
First recycling module, for when receive delete instruction when, by first object small documents mark in the POSIX file
Know to have deleted, and recycle the memory space of the first object small documents, obtains pre- recycling POSIX file；
Judgment module is used for when the pre- recycling POSIX file meets default recycling condition, for the pre- recycling
Each small documents in POSIX file, judge whether the small documents are identified as to have deleted；
Second recycling module copies to the small documents default if being not identified as having deleted for the small documents
Memory space, and recycle the memory space of the small documents.
Optionally, the judgment module, comprising:
Subsection submodule obtains multiple texts for the pre- recycling POSIX file to be segmented according to preset mode
Part section, the file section include multiple small documents；
Computational submodule, for calculating the cavity of this document section for each file section in the multiple file section
Density, the cavity density indicate the ratio of the memory space being recovered in this document section Yu this document section memory space；
Sorting sub-module, for the empty density according to each file section in the multiple file section, according to the sky
The size of hole density is ranked up the multiple file section；
Judging submodule, for according to the corresponding sequence of the multiple file section, obtaining file section to be processed to include
Small documents judge whether the small documents are identified as to have deleted for each small documents got.
Optionally, the pre- recycling POSIX file meets default recycling condition, comprising:
The read-write delay of the pre- recycling POSIX file is greater than preset threshold；
Alternatively,
Reach the preset recycling moment.
Optionally, the subsection submodule, the storage occupied specifically for pre- recycling POSIX file according to the pre-stored data
The corresponding relationship in space and segmented mode, determine the corresponding segmented mode of the pre- recycling POSIX file, and according to determining
Segmented mode is segmented the pre- recycling POSIX file, obtains multiple file sections.
Optionally, the subsection submodule, specifically for the memory space that is occupied when the pre- recycling POSIX file the
When in one default memory space ranges, the pre- recycling POSIX file is segmented according to preset file section memory space,
Obtain multiple file sections.
Optionally, the subsection submodule, specifically for the memory space that is occupied when the pre- recycling POSIX file the
When in two default memory space ranges, the pre- recycling POSIX file is segmented according to preset file section number, is obtained
Multiple file sections.
The third aspect, in order to achieve the above object, the embodiment of the invention also discloses a kind of electronic equipment, the electronics is set
Standby includes processor, communication interface, memory and communication bus, wherein the processor, the communication interface, the storage
Device completes mutual communication by the communication bus:
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes the method and step as described in any of the above-described.
Fourth aspect, in order to achieve the above object, the embodiment of the invention also discloses a kind of computer readable storage medium,
It is stored with computer program in the computer readable storage medium, when the computer program is executed by processor, realizes such as
Any of the above-described method and step.
At the another aspect that the present invention is implemented, the embodiment of the invention also provides a kind of, and the computer program comprising instruction is produced
Product, when run on a computer, so that computer executes any of the above-described method and step.
A kind of recycling memory space method and apparatus provided in an embodiment of the present invention, available POSIX text to be processed
First object small documents in POSIX file are identified as and have deleted, and recycle first object when receiving deletion instruction by part
The memory space of small documents obtains pre- recycling POSIX file, when recycling POSIX file meets default recycling condition in advance, for
Each small documents in pre- recycling POSIX file, judge whether the small documents are identified as to have deleted, if the small documents not by
It is identified as and has deleted, then the small documents are copied into default memory space, and recycle the memory space of the small documents.Based on above-mentioned
Processing can merge and delete the file cavity that small documents generate, reduce the quantity in file cavity, and then reduce and delete small documents pair
The influence of the system performance of server.
Certainly, it implements any of the products of the present invention or method must be not necessarily required to reach all the above excellent simultaneously
Point.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described.
Fig. 1 is a kind of flow chart of method for recycling memory space provided in an embodiment of the present invention；
Fig. 2 is a kind of structure chart of POSIX file provided in an embodiment of the present invention；
Fig. 3 is a kind of structure chart of POSIX file provided in an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram of the treatment process of the pre- recycling POSIX file of duplication provided in an embodiment of the present invention；
Fig. 5 is a kind of signal for the treatment of process recycled to pre- recycling POSIX file provided in an embodiment of the present invention
Figure；
Fig. 6 is a kind of signal for the treatment of process being segmented to pre- recycling POSIX file provided in an embodiment of the present invention
Figure；
Fig. 7 is a kind of structure chart of device for recycling memory space provided in an embodiment of the present invention；
Fig. 8 is the structure chart of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention is described.
In the prior art, when deleting the small documents in POSIX file, the corresponding memory space of the small documents will become text
Part cavity.When deleting more small documents in POSIX file, it will lead in POSIX file that there are more file cavities.Clothes
Business device needs distributing system resource to safeguard each file cavity, and then can reduce the system performance of server.
The embodiment of the invention provides a kind of methods and transposition for recycling memory space, can be applied to server, service
Device can merge when recycling memory space and delete the file cavity that small documents generate, reduce the quantity in file cavity, and then drop
The low influence for deleting small documents to the system performance of server.
Referring to Fig. 1, Fig. 1 is a kind of flow chart of method for recycling memory space provided in an embodiment of the present invention, this method
It may include steps of:
S100 obtains portable operating system interface POSIX file to be processed.
Wherein, POSIX file is made of a large amount of small documents.Application program can generate quantity in the process of running
The files such as huge (hundreds of millions of) picture and document, these files are usually smaller (generally in 100KB or less), to can be described as small text
Part.Server can store these small documents into POSIX file, while small documents are recorded in indexed file in POSIX
Offset in file, that is, the index information of the small documents.When application program needs to read a certain small documents, server
The offset for searching the small documents in indexed file first obtains the small documents further according to the offset found, and should
Small documents are sent to application program.
In an implementation, when application program needs to handle POSIX file, application program can be to server
Small documents processing request is sent, after server receives small documents processing request, POSIX belonging to the available small documents
File obtains the small documents from the POSIX file, and is sent to the application program.For example, the chat when user's using terminal is answered
When with program, user can send the read request of small documents (such as picture), service by the chat application to server
After device receives the read request, the small documents can be obtained from POSIX file, and be sent to chat application.Chat is answered
After receiving the small documents with program, user can be showed, for example, by contact head image, image content, the Yi Jibei of chat
Note information shows user.As shown in Fig. 2, may include multiple small documents in POSIX file, it can also be empty comprising available storage
Between, free memory is the idle storage space for storing small documents.
First object small documents in POSIX file are identified as and have deleted when receiving deletion instruction by S110, and return
The memory space for receiving first object small documents, obtains pre- recycling POSIX file.
In an implementation, when user needs to delete a certain small documents (first object file), user can be in application program
In, selection needs the small documents (i.e. first object small documents) deleted, such as a certain picture or a certain document etc..User can point
The deletion option in the application program is hit, application program can will then delete instruction and be sent to server, and server can then connect
Receive the removal request of corresponding first object small documents.Then, server can in the corresponding index information of POSIX file,
The index information of first object small documents is searched, and then adds mark information in the index information of first object small documents, with
Label first object small documents are to have deleted.Then, server can discharge memory space shared by first object small documents, with
Realize the memory space of recycling first object small documents.Wherein, the POSIX file after deleting first object small documents is pre- returns
POSIX file is received, as shown in figure 3, the occupancy memory space of the small documents can be recovered to be formed when deleting after a certain small documents
File cavity.
It should be noted that the memory space can be used to deposit after memory space shared by a certain small documents is recovered
Store up other new small documents.For example, server can recycle these picture files after user deletes some picture files
Memory space.After user newly adds a part of picture file, server can be stored this newly added part picture file
To the memory space recycled.
S120, when recycling POSIX file meets default recycling condition in advance, for each of pre- recycling POSIX file
Small documents, judge whether the small documents are identified as to have deleted.
Wherein, when user delete small documents quantity it is more, and these small documents memory space it is discontinuous when, will lead to
There are a large amount of file cavities in pre- recycling POSIX file.Since server needs distributing system resource to safeguard each file
Cavity, therefore, the time that will affect server and read pre- recycling POSIX file small file in these files cavity.Based on this,
Default recycling condition can be greater than preset threshold for the read-write delay of pre- recycling POSIX file.For example, preset threshold can be
100ms, when server is greater than 100ms to the read-write delay of pre- recycling POSIX file, then server can be determined that POSIX text
Part meets default recycling condition.Alternatively, the fixed recycling moment can also be rule of thumb arranged in technical staff, as pre- recycling
The default recycling condition of POSIX file.
In an implementation, when reaching the default recycling condition of POSIX file, server can be to the pre- recycling POSIX text
Small documents in part carry out secondary recovery.Server can be multiple since the memory space address that recycling POSIX file originates in advance
Small documents in the pre- recycling POSIX file of system, can also be to a certain section of memory space in pre- recycling POSIX file storage
It is recycled.Specifically, for each small documents in pre- recycling POSIX file, whether server may determine that the small documents
It is identified as to have deleted.
The small documents are copied to default memory space, and return if the small documents are not identified as having deleted by S130
Receive the memory space of the small documents.
In an implementation, for a certain small documents, if server determines the small documents (can be described as the second target small documents)
It is not identified as having deleted, server can copy to the second target small documents in preset free memory, and discharge
The memory space that the second target small documents occupy in recycling POSIX file in advance.Wherein, default free memory can be
It is empty to be also possible to the unemployed available storage of other in server for the free memory of the pre- recycling POSIX file end
Between.Based on above-mentioned processing, server can merge discrete file cavity, and then reduce the quantity in file cavity.Such as Fig. 4 institute
Show, for the treatment process schematic diagram of the pre- recycling POSIX file of duplication provided in an embodiment of the present invention.
As shown in figure 5, for the present invention provides a kind for the treatment of process signals recycled to pre- recycling POSIX file
Figure, specific steps may include S121, S122, S123 and S124.
Pre- recycling POSIX file is segmented according to preset segmented mode, obtains multiple file sections by S121.
Wherein, each file section may include multiple small documents.
In an implementation, as shown in fig. 6, being segmented for one kind provided in an embodiment of the present invention to pre- recycling POSIX file
Treatment process schematic diagram.Preset segmented mode is can store in server, server can be according to preset segmentation side
Formula is segmented pre- recycling POSIX file, obtains multiple file sections.Wherein, segmented mode can be depositing according to file section
Storage space is segmented, and be can also be and is segmented according to the number of file section.For example, recycling POSIX file size is 10G in advance,
If the memory space according to each file section is that 2G is segmented, pre- recycling POSIX file can be divided into 5 by server
Section；If being 2 sections according to the number of total file section to be segmented, pre- recycling POSIX file can be divided into two by server
A memory space is the file section of 5G.
S122 calculates the empty density of this document section for each file section in multiple file sections.
Wherein, empty density can indicate the memory space being recovered in this document section and this document section memory space
Ratio.
In an implementation, there is the memory space that is recovered of part in each file section, server, which can calculate, to be recovered
The ratio of the data volume of the memory space of the data volume and file section of memory space obtains the empty density of this document section.For example,
One file section size is 5G, wherein the memory space being recovered is 2G, then the empty close of this document section is calculated in server
Degree is 2G/5G=0.4.
S123, according to the empty density of each file section in multiple file sections, according to the size of empty density to multiple
File section is ranked up.
In an implementation, server is according to the empty density of each file section being calculated, by each file section according to sky
The sequence of hole density from big to small establishes index information.
S124 obtains the small documents that file section to be processed includes, for acquisition according to the corresponding sequence of multiple file sections
The each small documents arrived, judge whether the small documents are identified as to have deleted.
In an implementation, server can according to the sequence of empty density from big to small, successively to each file section at
Reason.For each small documents in file section to be processed, server may determine that whether the small documents are identified as to have deleted,
If the small documents are identified as to have deleted, server can recycle the memory space of the small documents.
The present invention provides a kind of mode for being segmented pre- recycling POSIX file, concrete processing procedure is as follows:
The corresponding relationship of memory space and segmented mode that pre- recycling POSIX file according to the pre-stored data occupies, determines
It is pre- to recycle the corresponding segmented mode of POSIX file, and pre- recycling POSIX file is divided according to the segmented mode determined
Section, obtains multiple file sections.
In an implementation, the memory space and segmentation side that pre- recycling POSIX file occupies can be previously stored in server
The corresponding relationship of formula, the corresponding relationship can be rule of thumb configured by technical staff.Server can be according to pre- recycling
The memory space that POSIX file occupies, is inquired in the above correspondence relationship, obtains the segmentation of the pre- recycling POSIX file
Mode.Then, the segmented mode that server is obtained according to inquiry, is segmented above-mentioned pre- recycling POSIX file, obtains multiple
File section.For example, server can be according to each file section 1G to pre- recycling POSIX when recycling POSIX file is 10G in advance
File is segmented, and 10 file sections are obtained.When recycling POSIX file is 20G in advance, server can be according to 5 file sections
Mode recycle POSIX file in advance and be segmented, obtain the file section that 5 storage sizes are 4G.
The present invention provides the modes being segmented to pre- recycling POSIX file, specifically there is following two situation.
Situation one, when the memory space that recycling POSIX file occupies in advance is in the first default memory space ranges, according to
Preset file section memory space is segmented pre- recycling POSIX file, obtains multiple file sections.
Wherein, the first default memory space ranges can be rule of thumb configured by technical staff.
In an implementation, it after server determines the pre- size for recycling the memory space that POSIX file occupies, can be deposited in advance
It is inquired in the corresponding relationship of memory space and segmented mode that the pre- recycling POSIX file of storage occupies, determines pre- recycling
The corresponding memory space ranges of memory space that POSIX file occupies, and then determine the corresponding segmentation side of the memory space ranges
Formula.If the memory space that recycling POSIX file occupies in advance is in the first memory space ranges, server can determine segmentation
Mode is to be segmented according to the memory space that file section occupies.Server can be according to the segmented mode to pre- recycling POSIX
File is segmented.For example, the corresponding memory space ranges of memory space that ought recycle POSIX file occupancy in advance are 10~20G
When, server can be segmented pre- recycling POSIX file according to the file section size of every section of 1G, obtain multiple file sections.
Situation two, when the memory space that recycling POSIX file occupies in advance is in the second default memory space ranges, according to
Preset file section number is segmented pre- recycling POSIX file, obtains multiple file sections.
Wherein, the lower limit of the second default memory space ranges can be greater than the first default memory space ranges of situation one kind
The upper limit, specifically, the second default memory space ranges can be rule of thumb configured by technical staff.
In an implementation, it after server determines the pre- size for recycling the memory space that POSIX file occupies, can be deposited in advance
It is inquired in the corresponding relationship of memory space and segmented mode that the pre- recycling POSIX file of storage occupies, determines pre- recycling
The corresponding memory space ranges of memory space that POSIX file occupies, and then determine the corresponding segmentation side of the memory space ranges
Formula.If the memory space that recycling POSIX is occupied in advance is in the second memory space ranges, server can determine segmented mode
To be segmented according to the occupancy memory space of file section.Server can be according to the segmented mode to pre- recycling POSIX file
It is segmented.For example, when the corresponding memory space ranges of memory space that recycling POSIX file occupies in advance are 20~100G,
Server can be 5 sections according to file section number, be segmented to pre- recycling POSIX file, obtain 5 file sections.
A kind of method recycling memory space provided in an embodiment of the present invention, available portable operating system to be processed
First object small documents in POSIX file are identified as and have deleted when receiving deletion instruction by interface POSIX file, and return
The memory space for receiving first object small documents, obtains pre- recycling POSIX file, when recycling POSIX file meets default recycling in advance
When condition, for each small documents in pre- recycling POSIX file, judge whether the small documents are identified as to have deleted, if
The small documents are not identified as having deleted, then the small documents are copied to default memory space, and recycle the storage of the small documents
Space.Based on above-mentioned processing, it can merge and delete the file cavity that small documents generate, reduce the quantity in file cavity, and then drop
The low influence for deleting small documents to the system performance of server.
Corresponding with the embodiment of the method for Fig. 1, referring to Fig. 7, Fig. 7 is that the embodiment of the invention also provides a kind of recycling storages
The structure chart of the device in space, the apparatus may include:
Module 700 is obtained, for obtaining portable operating system interface POSIX file to be processed；
First recycling module 710, for when receive delete instruction when, by the small text of first object in the POSIX file
Part, which is identified as, have been deleted, and recycles the memory space of the first object small documents, obtains pre- recycling POSIX file；
Judgment module 720 is used for when the pre- recycling POSIX file meets default recycling condition, for described pre- time
Each small documents in POSIX file are received, judge whether the small documents are identified as to have deleted；
Second recycling module 730 copies to the small documents pre- if being not identified as having deleted for the small documents
If memory space, and recycle the memory space of the small documents.
Optionally, the judgment module 720, comprising:
Subsection submodule obtains multiple texts for the pre- recycling POSIX file to be segmented according to preset mode
Part section, the file section include multiple small documents；
Computational submodule, for calculating the cavity of this document section for each file section in the multiple file section
Density, the cavity density indicate the ratio of the memory space being recovered in this document section Yu this document section memory space；
Sorting sub-module, for the empty density according to each file section in the multiple file section, according to the sky
The size of hole density is ranked up the multiple file section；
Judging submodule, for according to the corresponding sequence of the multiple file section, obtaining file section to be processed to include
Small documents judge whether the small documents are identified as to have deleted for each small documents got.
Optionally, the pre- recycling POSIX file meets default recycling condition, comprising:
The read-write delay of the pre- recycling POSIX file is greater than preset threshold；
Alternatively,
Reach the preset recycling moment.
Optionally, the subsection submodule, the storage occupied specifically for pre- recycling POSIX file according to the pre-stored data
The corresponding relationship in space and segmented mode, determine the corresponding segmented mode of the pre- recycling POSIX file, and according to determining
Segmented mode is segmented the pre- recycling POSIX file, obtains multiple file sections.
Optionally, the subsection submodule, specifically for the memory space that is occupied when the pre- recycling POSIX file the
When in one default memory space ranges, the pre- recycling POSIX file is segmented according to preset file section memory space,
Obtain multiple file sections.
Optionally, the subsection submodule, specifically for the memory space that is occupied when the pre- recycling POSIX file the
When in two default memory space ranges, the pre- recycling POSIX file is segmented according to preset file section number, is obtained
Multiple file sections.
A kind of device recycling memory space provided in an embodiment of the present invention, available portable operating system to be processed
First object small documents in POSIX file are identified as and have deleted when receiving deletion instruction by interface POSIX file, and return
The memory space for receiving first object small documents, obtains pre- recycling POSIX file, when recycling POSIX file meets default recycling in advance
When condition, for each small documents in pre- recycling POSIX file, judge whether the small documents are identified as to have deleted, if
The small documents are not identified as having deleted, then the small documents are copied to default memory space, and recycle the storage of the small documents
Space.Based on above-mentioned processing, it can merge and delete the file cavity that small documents generate, reduce the quantity in file cavity, and then drop
The low influence for deleting small documents to the system performance of server.
The embodiment of the invention also provides a kind of electronic equipment, as shown in figure 8, include processor 801, communication interface 802,
Memory 803 and communication bus 804, wherein processor 801, communication interface 802, memory 803 are complete by communication bus 804
At mutual communication,
Memory 803, for storing computer program；
Processor 801 when for executing the program stored on memory 803, realizes following steps:
Obtain portable operating system interface POSIX file to be processed；
When receiving deletion instruction, first object small documents in the POSIX file are identified as and have been deleted, and are recycled
The memory space of the first object small documents obtains pre- recycling POSIX file；
When the pre- recycling POSIX file meets default recycling condition, for every in the pre- recycling POSIX file
A small documents, judge whether the small documents are identified as to have deleted；
If the small documents are not identified as having deleted, which is copied into default memory space, and recycling should
The memory space of small documents.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component
Interconnect, abbreviation PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, abbreviation EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..
Only to be indicated with a thick line in figure, it is not intended that an only bus or a type of bus convenient for indicating.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory may include random access memory (Random Access Memory, abbreviation RAM), also may include
Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.Optionally, memory may be used also
To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit,
Abbreviation CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be digital signal processor
(Digital Signal Processing, abbreviation DSP), specific integrated circuit (Application Specific
Integrated Circuit, abbreviation ASIC), field programmable gate array (Field-Programmable Gate Array,
Abbreviation FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware components.
In another embodiment provided by the invention, a kind of computer readable storage medium is additionally provided, which can
It reads to be stored with instruction in storage medium, when run on a computer, so that computer executes any institute in above-described embodiment
The method stated.
In another embodiment provided by the invention, a kind of computer program product comprising instruction is additionally provided, when it
When running on computers, so that computer executes any method in above-described embodiment.
In embodiments of the present invention, available portable operating system interface POSIX file to be processed, is deleted when receiving
When except instruction, first object small documents in POSIX file are identified as and have been deleted, and the storage for recycling first object small documents is empty
Between, pre- recycling POSIX file is obtained, when recycling POSIX file meets default recycling condition in advance, for pre- recycling POSIX text
Each small documents in part, judge whether the small documents are identified as to have deleted, if the small documents are not identified as having deleted,
The small documents are then copied into default memory space, and recycle the memory space of the small documents.Based on above-mentioned processing, can merge
The file cavity that small documents generate is deleted, reduces the quantity in file cavity, and then reduce and delete small documents to the system of server
The influence of performance.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof real
It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.The computer program
Product includes one or more computer instructions.When loading on computers and executing the computer program instructions, all or
It partly generates according to process or function described in the embodiment of the present invention.The computer can be general purpose computer, dedicated meter
Calculation machine, computer network or other programmable devices.The computer instruction can store in computer readable storage medium
In, or from a computer readable storage medium to the transmission of another computer readable storage medium, for example, the computer
Instruction can pass through wired (such as coaxial cable, optical fiber, number from a web-site, computer, server or data center
User's line (DSL)) or wireless (such as infrared, wireless, microwave etc.) mode to another web-site, computer, server or
Data center is transmitted.The computer readable storage medium can be any usable medium that computer can access or
It is comprising data storage devices such as one or more usable mediums integrated server, data centers.The usable medium can be with
It is magnetic medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as solid state hard disk
Solid State Disk (SSD)) etc..
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For electronic equipment, computer readable storage medium and computer program product embodiments, since it is substantially similar to method
Embodiment, so being described relatively simple, the relevent part can refer to the partial explaination of embodiments of method.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention
It is interior.
Claims (13)
1. a kind of method for recycling memory space, which is characterized in that the described method includes:
Obtain portable operating system interface POSIX file to be processed；
When receiving deletion instruction, first object small documents in the POSIX file are identified as and have been deleted, and described in recycling
The memory space of first object small documents obtains pre- recycling POSIX file；
It is small for each of the pre- recycling POSIX file when the pre- recycling POSIX file meets default recycling condition
File, judges whether the small documents are identified as to have deleted；
If the small documents are not identified as having deleted, which is copied into default memory space, and recycles the small text
The memory space of part.
2. the method according to claim 1, wherein described for each of described pre- recycling POSIX file
Small documents, judge whether the small documents are identified as to have deleted, comprising:
The pre- recycling POSIX file is segmented according to preset mode, obtains multiple file sections, the file section includes
Multiple small documents；
For each file section in the multiple file section, the empty density of this document section, the cavity density meter are calculated
Show the ratio for the memory space and this document section memory space being recovered in this document section；
According to the empty density of each file section in the multiple file section, according to the size of the empty density to described more
A file section is ranked up；
According to the corresponding sequence of the multiple file section, the small documents that file section to be processed includes are obtained, for what is got
Each small documents, judge whether the small documents are identified as to have deleted.
3. the method according to claim 1, wherein the pre- recycling POSIX file meets default recycling condition,
Include:
The read-write delay of the pre- recycling POSIX file is greater than preset threshold；
Alternatively,
Reach the preset recycling moment.
4. according to the method described in claim 2, it is characterized in that, it is described by the pre- recycling POSIX file according to preset
Mode is segmented, and multiple file sections are obtained, comprising:
It is according to the pre-stored data it is pre- recycling POSIX file occupy memory space and segmented mode corresponding relationship, determine described in
It is pre- to recycle the corresponding segmented mode of POSIX file, and the pre- recycling POSIX file is carried out according to the segmented mode determined
Segmentation, obtains multiple file sections.
5. according to the method described in claim 4, it is characterized in that, the pre- recycling POSIX file according to the pre-stored data accounts for
The corresponding relationship of memory space and segmented mode, determines the corresponding segmented mode of the pre- recycling POSIX file, and according to
The segmented mode determined is segmented the pre- recycling POSIX file, obtains multiple file sections, comprising:
When the memory space that the pre- recycling POSIX file occupies is in the first default memory space ranges, according to preset
File section memory space is segmented the pre- recycling POSIX file, obtains multiple file sections.
6. according to the method described in claim 4, it is characterized in that, the pre- recycling POSIX file according to the pre-stored data accounts for
The corresponding relationship of memory space and segmented mode, determines the corresponding segmented mode of the pre- recycling POSIX file, and according to
The segmented mode determined is segmented the pre- recycling POSIX file, obtains multiple file sections, comprising:
When the memory space that the pre- recycling POSIX file occupies is in the second default memory space ranges, according to preset
File section number is segmented the pre- recycling POSIX file, obtains multiple file sections.
7. a kind of device for recycling memory space, which is characterized in that described device includes:
Module is obtained, for obtaining portable operating system interface POSIX file to be processed；
First recycling module, for when receiving deletion instruction, first object small documents in the POSIX file to be identified as
It has been deleted that, and recycle the memory space of the first object small documents, obtained pre- recycling POSIX file；
Judgment module is used for when the pre- recycling POSIX file meets default recycling condition, for the pre- recycling POSIX
Each small documents in file, judge whether the small documents are identified as to have deleted；
The small documents are copied to default storage if being not identified as having deleted for the small documents by the second recycling module
Space, and recycle the memory space of the small documents.
8. device according to claim 7, which is characterized in that the judgment module, comprising:
Subsection submodule obtains multiple files for the pre- recycling POSIX file to be segmented according to preset mode
Section, the file section includes multiple small documents；
Computational submodule, for calculating the empty density of this document section for each file section in the multiple file section,
The cavity density indicates the ratio of the memory space being recovered in this document section Yu this document section memory space；
Sorting sub-module, it is close according to the cavity for the empty density according to each file section in the multiple file section
The size of degree is ranked up the multiple file section；
Judging submodule, for obtaining the small text that file section to be processed includes according to the corresponding sequence of the multiple file section
Part judges whether the small documents are identified as to have deleted for each small documents got.
9. device according to claim 7, which is characterized in that the pre- recycling POSIX file meets default recycling condition,
Include:
The read-write delay of the pre- recycling POSIX file is greater than preset threshold；
Alternatively,
Reach the preset recycling moment.
10. device according to claim 8, which is characterized in that the subsection submodule is stored in advance specifically for basis
Pre- recycling POSIX file occupy memory space and segmented mode corresponding relationship, determine the pre- recycling POSIX file pair
The segmented mode answered, and the pre- recycling POSIX file is segmented according to the segmented mode determined, obtain multiple files
Section.
11. device according to claim 10, which is characterized in that the subsection submodule is specifically used for working as described pre- time
When receiving the memory space of POSIX file occupancy in the first default memory space ranges, according to preset file section memory space
The pre- recycling POSIX file is segmented, multiple file sections are obtained.
12. device according to claim 10, which is characterized in that the subsection submodule is specifically used for working as described pre- time
When receiving the memory space of POSIX file occupancy in the second default memory space ranges, according to preset file section number to institute
It states pre- recycling POSIX file to be segmented, obtains multiple file sections.
13. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing
Device, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes any method and step of claim 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810643419.7A CN109086220B (en) | 2018-06-21 | 2018-06-21 | Method and device for recycling storage space |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810643419.7A CN109086220B (en) | 2018-06-21 | 2018-06-21 | Method and device for recycling storage space |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109086220A true CN109086220A (en) | 2018-12-25 |
| CN109086220B CN109086220B (en) | 2023-04-28 |
Family
ID=64840108
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810643419.7A Active CN109086220B (en) | 2018-06-21 | 2018-06-21 | Method and device for recycling storage space |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109086220B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328549A (en) * | 2020-10-29 | 2021-02-05 | 无锡先进技术研究院 | Small file storage method, electronic device and storage medium |
| CN112925745A (en) * | 2021-03-25 | 2021-06-08 | 北京奇艺世纪科技有限公司 | Data cleaning method and device, electronic equipment and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100235365A1 (en) * | 2009-03-13 | 2010-09-16 | Newby Jr Marvon M | PS9110 Linear time sorting and constant time searching algorithms |
| CN105138282A (en) * | 2015-08-06 | 2015-12-09 | 上海七牛信息技术有限公司 | Storage space recycling method and storage system |
| CN106446044A (en) * | 2016-08-31 | 2017-02-22 | 北京小米移动软件有限公司 | Storage space reclaiming method and device |
| CN108090128A (en) * | 2017-11-16 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of merging memory space recovery method, device and electronic equipment |
-
2018
- 2018-06-21 CN CN201810643419.7A patent/CN109086220B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100235365A1 (en) * | 2009-03-13 | 2010-09-16 | Newby Jr Marvon M | PS9110 Linear time sorting and constant time searching algorithms |
| CN105138282A (en) * | 2015-08-06 | 2015-12-09 | 上海七牛信息技术有限公司 | Storage space recycling method and storage system |
| CN106446044A (en) * | 2016-08-31 | 2017-02-22 | 北京小米移动软件有限公司 | Storage space reclaiming method and device |
| CN108090128A (en) * | 2017-11-16 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of merging memory space recovery method, device and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328549A (en) * | 2020-10-29 | 2021-02-05 | 无锡先进技术研究院 | Small file storage method, electronic device and storage medium |
| CN112925745A (en) * | 2021-03-25 | 2021-06-08 | 北京奇艺世纪科技有限公司 | Data cleaning method and device, electronic equipment and computer readable storage medium |
| CN112925745B (en) * | 2021-03-25 | 2023-08-08 | 北京奇艺世纪科技有限公司 | Data cleaning method and device, electronic equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109086220B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2021524091A (en) | Abnormal user identification method, electronic devices and machine-readable storage media | |
| CN107786638B (en) | Data processing method, device and system | |
| KR20160003682A (en) | Hydration and dehydration with placeholders | |
| US8214428B1 (en) | Optimized prepopulation of a client side cache in a deduplication environment | |
| US9940331B1 (en) | Proactive scavenging of file system snaps | |
| CN108984103B (en) | Method and apparatus for deduplication | |
| CN110888837B (en) | Object storage small file merging method and device | |
| US20210011634A1 (en) | Methods and systems for managing key-value solid state drives (kv ssds) | |
| US20170199889A1 (en) | Method and device for identifying junk picture files | |
| CN109086220A (en) | A kind of method and apparatus recycling memory space | |
| CN110147203A (en) | A kind of file management method, device, electronic equipment and storage medium | |
| CN111143113A (en) | Method, electronic device and computer program product for copying metadata | |
| CN105493080A (en) | Method and apparatus for context aware based data de-duplication | |
| CN114035750A (en) | File processing method, device, equipment, medium and product | |
| US9785724B2 (en) | Secondary queue for index process | |
| CN109144403B (en) | Method and equipment for switching cloud disk modes | |
| WO2024082525A1 (en) | File snapshot method and system, electronic device, and storage medium | |
| WO2023185159A1 (en) | Method and apparatus for determining spatial position relationship, and electronic device | |
| CN109298956B (en) | File transmission method and device and terminal equipment | |
| CN115757269A (en) | Data recovery method and device, electronic equipment and storage medium | |
| CN114968558A (en) | Memory cleaning method and related equipment | |
| CN116362955A (en) | Graph data storage, access and processing methods, training methods, equipment and media | |
| CN110083482A (en) | The method, apparatus and electronic equipment of correcting and eleting codes processing are carried out to document storage system | |
| US20130246956A1 (en) | Information processing apparatus, non-transitory computer readable medium, and information processing method | |
| CN114528258B (en) | Asynchronous file processing method, device, server, medium, product and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |