CN109408202B - LMDB extension method and device - Google Patents
LMDB extension method and device Download PDFInfo
- Publication number
- CN109408202B CN109408202B CN201811142236.3A CN201811142236A CN109408202B CN 109408202 B CN109408202 B CN 109408202B CN 201811142236 A CN201811142236 A CN 201811142236A CN 109408202 B CN109408202 B CN 109408202B
- Authority
- CN
- China
- Prior art keywords
- virtual memory
- memory space
- database file
- target database
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
- G06F9/467—Transactional memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides an LMDB extension method and an LMDB extension device, wherein the method comprises the following steps: when determining that a first virtual memory space of a target process pointed by a pointer of a target database file meets an expansion condition, mapping a second virtual memory space in the target process for the target database file; wherein the length of the second virtual memory space is greater than the length of the first virtual memory space; and pointing the pointer of the target database file to the second virtual memory space. By applying the embodiment of the invention, the access efficiency of the database file can be improved.
Description
Technical Field
The invention relates to the technical field of network communication, in particular to an LMDB expansion method and an LMDB expansion device.
Background
An LMDB (Lightning Memory-Mapped Database) is a Database based on MMAP (a method of mapping a file). The MMAP can map the content of the database file to the inside of a process (virtual memory space), a plurality of processes can map one database file at the same time, and the LMDB realizes the sharing of data among different processes based on the LMDB.
When the remaining space of the virtual memory space mapped by a database file in a process is zero, the process needs to wait for the completion of all transactions accessing the virtual memory space, then calls an interface of the LMDB, cancels the original memory mapping, i.e., releases the original virtual memory space (the virtual memory space with the remaining space of zero), and re-maps a longer virtual memory space for the database file in the process.
However, releasing the original virtual memory space requires waiting for all readers/writers accessing the virtual memory space to finish, and the time for the readers/writers accessing the virtual memory space to finish is uncertain, and waiting for a long time may be required for all readers/writers accessing the virtual memory space, which may cause other readers/writers needing to access the database file through the process to block.
Disclosure of Invention
The invention provides an LMDB extension method and device, which aim to solve the problem of low LMDB extension efficiency in the conventional LMDB implementation scheme.
According to a first aspect of the present invention, there is provided an LMDB extension method, comprising:
when determining that a first virtual memory space of a target process pointed by a pointer of a target database file meets an expansion condition, mapping a second virtual memory space in the target process for the target database file; wherein the length of the second virtual memory space is greater than the length of the first virtual memory space;
and pointing the pointer of the target database file to the second virtual memory space.
With reference to the first aspect, in a first possible implementation manner, before mapping a second virtual memory space in the target process for the target database file, the method further includes:
when the residual space of the first virtual memory space is smaller than a preset threshold value, adding a preset mark in the file header of the target database file;
the determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition includes:
when the preset mark is detected to exist in the file head of the target database file, determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition.
With reference to the first aspect, in a second possible implementation manner, a file header of the target database file records a length of the target database file;
the determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition includes:
when the length of the target database file is detected to be inconsistent with the length of the first virtual memory space, determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition.
With reference to the first aspect, in a third possible implementation manner, after the pointing the pointer of the target database file to the second virtual memory space, the method further includes:
and releasing the first virtual memory space when the first virtual memory space is determined to have no visitor.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the method further includes:
when detecting that the target process calls a transaction starting interface corresponding to the first virtual memory space, increasing a reference count corresponding to the first virtual memory space by a preset value;
when detecting that the target process calls a transaction submission interface corresponding to the first virtual memory space, subtracting the preset value from the reference count corresponding to the first virtual memory space;
the determining that no visitor exists to the first virtual memory space comprises:
and when detecting that the value of the reference count corresponding to the first virtual memory space is an initial value, determining that no visitor exists in the first virtual memory space.
According to a second aspect of the present invention, there is provided an LMDB extension apparatus, comprising:
the system comprises a determining unit, a processing unit and a processing unit, wherein the determining unit is used for determining whether a first virtual memory space of a target process pointed by a pointer of a target database file meets an expansion condition;
the mapping unit is used for mapping a second virtual memory space in a target process for a target database file when the fact that a first virtual memory space of the target process pointed by a pointer of the target database file meets an expansion condition is determined; wherein the length of the second virtual memory space is greater than the length of the first virtual memory space;
and the control unit is used for pointing the pointer of the target database file to the second virtual memory space.
With reference to the second aspect, in a first possible implementation manner, the apparatus further includes:
an adding unit, configured to add a preset flag in a file header of the target database file when a remaining space of the first virtual memory space is smaller than a preset threshold;
the determining unit is specifically configured to determine that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition when it is detected that the preset flag exists at the file header of the target database file.
With reference to the second aspect, in a second possible implementation manner, a file header of the target database file records a length of the target database file;
the determining unit is specifically configured to determine that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition when it is detected that the length of the target database file is inconsistent with the length of the first virtual memory space.
With reference to the second aspect, in a third possible implementation manner, the control unit is further configured to release the first virtual memory space when it is determined that no visitor exists in the first virtual memory space.
With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, the control unit is specifically configured to increase, by a preset value, a reference count corresponding to the first virtual memory space when it is detected that the target process invokes a transaction start interface corresponding to the first virtual memory space; when detecting that the target process calls a transaction submission interface corresponding to the first virtual memory space, subtracting the preset value from the reference count corresponding to the first virtual memory space;
the control unit is further specifically configured to determine that no visitor exists in the first virtual memory space when it is detected that the value of the reference count corresponding to the first virtual memory space is an initial value.
By applying the technical scheme disclosed by the invention, when the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition, the second virtual memory space with longer mapping length is mapped in the target process for the target database file, and the pointer of the target database file points to the second virtual memory space, so that readers/writers who subsequently access the target database file through the target process can directly access the second virtual memory space without waiting for the completion of all readers/writers who access the first virtual memory space, and the access efficiency of the database file is improved.
Drawings
Fig. 1 is a flowchart illustrating an LMDB extension method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an LMDB extension provided by an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an LMDB extension apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of another LMDB expansion device provided by an embodiment of the present invention;
fig. 5 is a schematic hardware structure diagram of an LMDB extension apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions in the embodiments of the present invention better understood and make the above objects, features and advantages of the embodiments of the present invention more comprehensible, the technical solutions in the embodiments of the present invention are described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, a flow chart of an LMDB extension method provided in an embodiment of the present invention is shown, where the LMDB extension method may be applied to a network device (e.g., a switch) running an LMDB, and as shown in fig. 1, the LMDB extension method may include the following steps:
for ease of understanding and description, the following description will use the execution subjects of step 101 to step 102 as network devices as examples.
In the embodiment of the invention, the target database file does not refer to a fixed database file, but refers to any database file in the network equipment;
in addition, the target process does not refer to a fixed process, but may refer to any process in the network device, and the following description of the embodiment of the present invention is not repeated.
In the embodiment of the present invention, when the network device needs to perform space allocation in a virtual memory space (referred to as a first virtual memory space herein) of the target process pointed by the pointer of the target database file, if a write transaction initiated by the target process and directed to the target database file is received, the network device may determine whether the first virtual memory space of the target process pointed by the pointer of the target database file satisfies an extension condition.
In the embodiment of the present invention, in consideration of the uncertainty of the end time of the reader/writer accessing the first virtual memory space, in order to avoid the blocking of other readers/writers that need to access the target database file through the target process, when the network device determines that the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition, if the remaining space of the first virtual memory space is zero, the network device may not release the first virtual memory space first, and map another longer virtual memory space (referred to as a second virtual memory space herein) in the target process for the target database file.
After the network device maps the second virtual memory space in the target process for the target database file, readers/writers who are accessing the first virtual memory space are not affected, and the network device continues to access the first virtual memory space.
In the embodiment of the present invention, in order to enable a subsequent reader/writer to access the target database file through the target process and access the second virtual memory space, after the network device maps the second virtual memory space in the target process for the target database file, the pointer of the target database file pointing to the first virtual memory space may be adjusted to point to the second virtual memory space.
It should be noted that, in the embodiment of the present invention, when a target database file maps a virtual memory space in a target process for the first time, a pointer pointing to the virtual memory space in the target process may be added to the target database file, and for an access request initiated by the target process and directed to the target database file, the access request may be located in the virtual memory space in the target process according to the pointer, which is not described herein in detail.
It can be seen that, in the method flow shown in fig. 1, when the first virtual memory space in the target process pointed by the pointer of the target database file meets the expansion condition, without releasing the first virtual memory space, a second virtual memory space with a larger length is mapped for the target database file in the target process, and the pointer of the target database file pointing to the first virtual memory space is adjusted to point to the second virtual memory space, so that readers/writers who subsequently access the target database file through the target process can directly access the second virtual memory space without waiting for all readers/writers who are accessing the first virtual memory space to end, thereby improving the access efficiency of the database file.
Further, in the embodiment of the present invention, it is considered that, for a write transaction, there may be any multiple delete (delete)/add (add)/modify (modify) operations from the beginning of the transaction (begin) to the commit of the transaction (commit), and these operations all need to allocate space from the virtual memory space, that is, all may result in zero remaining space in the virtual memory space, and if the remaining space in the virtual memory space before the commit of the write transaction is zero, the network device needs to cancel the operations (delete, add, and/or modify operations) that have been performed by the write transaction, and after the expansion of the virtual memory space is completed, re-execute all operations of the write transaction, thereby further reducing the access efficiency of the database file.
Therefore, in order to improve the access efficiency of the database file, it is necessary to avoid the remaining space of the virtual memory space as zero as possible.
Accordingly, in an embodiment of the present invention, before mapping the second virtual memory space in the target process for the target database file, the method may further include:
when the residual space of the first virtual memory space is smaller than a preset threshold value, adding a preset mark in a file header of a target database file;
the determining that the pointer of the target database file points to the first virtual memory space of the target process and meets the expansion condition may include:
when the file head of the target database file is detected to have the preset mark, determining that a first virtual memory space of a target process pointed by a pointer of the target database file meets an expansion condition.
In this embodiment, to avoid reduction of the database file access efficiency due to zero remaining space of the virtual memory space, the network device may perform virtual memory space expansion when the remaining space of the virtual memory space is smaller than a preset threshold (which may be set according to an actual scene, such as 10M or 20M).
In this embodiment, when the network device allocates a space for the write transaction from the first virtual memory space, the remaining space of the first virtual memory space may be detected, and when the remaining space of the first virtual memory space is smaller than a preset threshold, a flag (referred to as a preset flag herein) for identifying that the virtual memory space needs to be expanded may be added to the file header of the database file.
When there is a transaction accessing the target database file through the target process, the network device may detect the file header of the target database file to determine whether the file header of the target database file has the preset flag, and if so, determine that the first virtual memory space meets the expansion condition, and perform processing according to the manner described in the above step 101 to step 102.
It should be noted that, in the embodiment of the present invention, the size of the space that is increased when performing virtual memory expansion may be fixed, for example, each time performing virtual memory expansion, the space is increased by 20M; alternatively, the space to be increased when the virtual memory is expanded may not be fixed.
In addition, for different processes, the preset thresholds may be different, that is, for virtual memory spaces mapped in different processes, the virtual memory space expansion may be triggered when the remaining space in the virtual memory space decreases to different values.
Furthermore, after the network device maps the second virtual memory space in the target process for the target database file, the preset flag in the file header of the target database file may be deleted, so that the visitor can normally access the second virtual memory space, and the specific implementation thereof is not repeated here.
It can be seen that, in this embodiment, when the remaining space of the virtual memory space is less than the preset threshold, the preset flag is added in the header of the corresponding database file to indicate the network device to expand the virtual memory space, so that occurrence of a situation that the remaining space of the virtual memory space is zero is reduced, reduction of database file access efficiency caused by the fact that the remaining space of the virtual memory space is zero is avoided, and the database file access efficiency is improved.
Further, in the embodiment of the present invention, in order to ensure that the lengths of the database files in the processes are consistent after the virtual memory space is expanded, considering that the same database file may be mapped to multiple processes, when the virtual memory space in any process is expanded, the length of the database file may be recorded at the file header of the database file, and when the virtual memory space in any process is expanded, the length of the database file recorded at the file header of the corresponding database file may be updated (the length of the database file is matched with the length of the corresponding virtual memory space), so that other processes mapped with the same database file may adaptively perform virtual memory space expansion.
Accordingly, in an embodiment of the present invention, the determining that the first virtual memory space of the target process pointed by the pointer of the target database file satisfies the extension condition includes:
and when the length of the target database file is detected to be inconsistent with the length of the first virtual memory space, determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition.
In this embodiment, when the reader/writer accesses the target database file through the target process, for example, when the target process calls the transaction start interface corresponding to the first virtual memory space, the network device may read the length of the target database file recorded in the file header of the target database file, and determine whether the length of the target database file is consistent with the length of the first virtual memory space, and if not, may determine that virtual memory space expansion is performed on other processes to which the target database file is mapped, and at this time, the network device may also expand the first virtual memory space mapped in the target process.
In this embodiment, the length of the second virtual memory space is the same as the length of the target database file recorded in the file header of the target database file.
Further, in the embodiment of the present invention, after mapping the second virtual memory space in the target process for the target database file by the network device, the pointer of the target database file pointing to the first virtual memory space is adjusted to point to the second virtual memory space, so that a reader/writer who subsequently accesses the target database file through the target process accesses the second virtual memory space, that is, no newly added reader/writer occurs in the first virtual memory space, and therefore, in order to improve the resource utilization rate, resource recycling processing needs to be performed on the first virtual memory space at an appropriate time.
Accordingly, in an embodiment of the present invention, after the pointing the pointer of the target database file to the second virtual memory space, the method may further include:
and releasing the first virtual memory space when the first virtual memory space is determined to have no visitor.
In this embodiment, after the network device points the pointer of the target database file to the second virtual memory space, the first virtual memory space may be released and resource recovery may be performed when no visitor exists in the first virtual memory space, that is, when all visitors (readers/writers) currently accessing the first virtual memory space end, so as to improve resource utilization.
In one example, in order to enable the network device to timely know whether the virtual memory space has a visitor, a reference count may be newly added to each virtual memory space, and for any virtual memory space, when a transaction for accessing the virtual memory space starts, the reference count is increased by a preset value; when a transaction for an access to the virtual memory space commits, the reference count is subtracted by the preset value, so that the number of visitors accessing the virtual memory space can be determined according to the reference count.
Correspondingly, when the network device detects that the target process calls the transaction start interface corresponding to the first virtual memory space, the reference count corresponding to the first virtual memory space can be increased by a preset value;
and when detecting that the target process calls a commit interface of the transaction corresponding to the first virtual memory space, subtracting the preset value from the reference count corresponding to the first virtual memory space.
The determining that no visitor exists in the first virtual memory space may include:
and when detecting that the value of the reference count corresponding to the first virtual memory space is an initial value, determining that no visitor exists in the first virtual memory space.
For example, if the initial value of the reference count is 0 and the preset value is 1, when the network device detects that the target process calls the transaction start interface corresponding to the first virtual memory space, the reference count corresponding to the first virtual memory space may be increased by 1; when it is detected that the target process calls the transaction commit interface corresponding to the first virtual memory space, the reference count corresponding to the first virtual memory space may be decremented by 1, and further, when the network device detects that the value of the reference count corresponding to the first virtual memory space is 0, it may be determined that no visitor exists in the first virtual memory space, and at this time, the network device may release the first virtual memory space.
It should be appreciated that the implementation manner for determining whether the virtual memory space has the visitor through the reference count is only a specific example of determining whether the virtual memory space has the visitor in the embodiment of the present invention, and is not limited to the scope of the present invention, that is, in the embodiment of the present invention, whether the virtual memory space has the visitor may also be determined in other manners, for example, a user may detect whether the virtual memory space has the visitor, and notify the network device of the detection result; or, the longest time for the visitor to access the virtual memory space may be determined according to an empirical value of the time for the visitor to access the virtual memory space, and a preset time (the preset time is greater than or equal to the longest time) is set according to the longest time, and the network device may start timing when the pointer pointing to the first virtual memory space of the target database file is adjusted to point to the second virtual memory space, and determine that the visitor does not exist in the first virtual memory space when the timing duration reaches the preset time, which is not described herein in detail.
In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present invention, the technical solutions provided by the embodiments of the present invention are described below with reference to specific examples.
In this embodiment, it is assumed that the network device maps a virtual memory space 1 for the database file 1 in the process a, and the length of the virtual memory space 1 is 10M.
When the transaction start interface corresponding to the virtual memory space 1 is called, the network device may detect whether a preset flag exists in a file header of the database file 1, and if the preset flag does not exist, it is determined that the virtual memory space 1 does not need to be expanded, and at this time, the virtual memory space 1 may be accessed according to an existing processing flow.
When a write transaction for accessing the database file 1 through the process a starts, delete/add/modify operations and the like all trigger the space allocation of the virtual memory space 1.
After the network device allocates a space from the virtual memory space 1 each time, it may detect whether the remaining space of the virtual memory space 1 is greater than or equal to a preset threshold (assumed to be 1M). If it is detected that the remaining space of the virtual memory space 1 is less than 1M, a preset mark may be added to the file header of the database file 1.
When the transaction start interface corresponding to the virtual memory space 1 is called and the network device detects that the file header of the database file 1 has the preset mark, the network device may map a longer virtual memory space (hereinafter referred to as a virtual memory space 2) for the database file 1 in the process a, and adjust the pointer of the database file 1 pointing to the virtual memory space 1 to point to the virtual memory space 2, and thereafter, a visitor accessing the database file 1 through the process a may access the virtual memory space 2, which is schematically illustrated in fig. 2.
Here, assuming that the size of the space to be added each time the virtual memory space is expanded is 20M, the virtual memory space 2 is 30M (10M +20M is 30M).
It should be noted that, after mapping the virtual memory space 2 for the database file 1 in the process a, the network device may delete the preset mark of the file header of the database file 1, and a subsequent visitor may normally access the virtual memory space 2.
In addition, in this embodiment, after mapping the virtual memory space 2 for the database file 1 in the process a, the network device may further update the length of the database file recorded in the file header of the database file 1 to 30M (from 10M to 30M).
Further, in this embodiment, in order to improve the resource utilization, a reference count may be newly added to the virtual memory space, where an initial value of the reference count is 0, and when a transaction start interface corresponding to the virtual memory space is called, the reference count of the virtual memory space is + 1; when the transaction commit interface corresponding to the virtual memory space is called, the reference count of the virtual memory space is-1.
Correspondingly, when the network device detects that the reference count of the virtual memory space 1 is 0, it may be determined that no visitor exists in the virtual memory space 1, and at this time, the network device may release the virtual memory space 1 to implement resource recycling.
In this embodiment, assuming that the network device also maps a virtual memory space (hereinafter referred to as virtual memory space 3, assuming that the length of the virtual memory space is 10M) for the database file 1 in the process B, when the transaction start interface corresponding to the virtual memory space 3 is called, the network device may read the length (30M) of the database file 1 recorded in the file header of the database file 1.
Since the length (30M) of the database file 1 is not consistent with the length (10M) of the virtual memory space 3, the network device may map a longer virtual memory space (hereinafter referred to as a virtual memory space 4, whose length is 30M) for the database file 1 in the process B, so as to ensure that the lengths of the same database file mapped by different processes are the same.
After the network device maps the virtual memory space 4 for the database file 1 in the process B, the pointer of the database file 1 pointing to the virtual memory space 3 may also be adjusted to point to the virtual memory space 4.
For the virtual memory space 3, resource recovery may be performed in the manner described above, and specific implementation thereof is not described herein again.
As can be seen from the above description, in the technical solution provided in the embodiment of the present invention, when it is determined that the first virtual memory space of the target process to which the pointer of the target database file points satisfies the extension condition, the second virtual memory space with a longer length is mapped in the target process for the target database file, and the pointer of the target database file points to the second virtual memory space, so that readers/writers who subsequently access the target database file through the target process can directly access the second virtual memory space without waiting for all readers/writers who are accessing the first virtual memory space to end, thereby improving the access efficiency of the database file.
Referring to fig. 3, a schematic structural diagram of an LMDB extension apparatus is provided for an embodiment of the present invention, where the apparatus may be applied to a network device in the foregoing method embodiment, and as shown in fig. 3, the LMDB extension apparatus may include:
a determining unit 310, configured to determine whether a first virtual memory space of a target process pointed by a pointer of a target database file meets an extension condition;
a mapping unit 320, configured to map, when it is determined that a first virtual memory space of a target process pointed by a pointer of a target database file meets an extension condition, a second virtual memory space in the target process for the target database file; wherein the length of the second virtual memory space is greater than the length of the first virtual memory space;
the control unit 330 is configured to point the pointer of the target database file to the second virtual memory space.
Referring to fig. 4 together, a schematic structural diagram of another LMDB extension apparatus according to an embodiment of the present invention is shown in fig. 4, where, on the basis of the LMDB extension apparatus shown in fig. 3, the LMDB extension apparatus shown in fig. 4 may further include:
an adding unit 340, configured to add a preset flag in a file header of the target database file when a remaining space of the first virtual memory space is smaller than a preset threshold;
the determining unit 310 is specifically configured to determine that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition when it is detected that the preset flag exists in the file header of the target database file.
In an optional embodiment, the file header of the target database file records the length of the target database file;
the determining unit is specifically configured to determine that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition when it is detected that the length of the target database file is inconsistent with the length of the first virtual memory space.
In an optional embodiment, the control unit 330 is further configured to release the first virtual memory space when it is determined that there is no visitor to the first virtual memory space.
In an optional embodiment, the control unit 330 is specifically configured to, when it is detected that the target process invokes the transaction start interface corresponding to the first virtual memory space, increase a reference count corresponding to the first virtual memory space by a preset value; when detecting that the target process calls a transaction submission interface corresponding to the first virtual memory space, subtracting the preset value from the reference count corresponding to the first virtual memory space;
the control unit 330 is further specifically configured to determine that no visitor exists in the first virtual memory space when it is detected that the value of the reference count corresponding to the first virtual memory space is an initial value.
Fig. 5 is a schematic diagram of a hardware structure of an LMDB extension apparatus according to an embodiment of the present invention. The LMDB extension apparatus may include a processor 501, a machine-readable storage medium 502 having machine-executable instructions stored thereon. The processor 501 and the machine-readable storage medium 502 may communicate via a system bus 503. Also, the processor 501 may perform the LMDB extension method described above by reading and executing machine-executable instructions in the machine-readable storage medium 502 corresponding to the LMDB extension logic.
The machine-readable storage medium 502 referred to herein may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be volatile memory, non-volatile memory, or similar storage medium. In particular, the machine-readable storage medium 502 may be a RAM (random Access Memory), a flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., a compact disk, a DVD, etc.), or similar storage medium, or a combination thereof.
Embodiments of the present invention also provide a machine-readable storage medium, such as machine-readable storage medium 502 in fig. 5, comprising machine-executable instructions that are executable by processor 501 in an LMDB expansion device to implement the LMDB expansion method described above.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
As can be seen from the foregoing embodiments, when it is determined that the first virtual memory space of the target process to which the pointer of the target database file points satisfies the extension condition, a second virtual memory space with a longer mapping length is mapped in the target process for the target database file, and the pointer of the target database file points to the second virtual memory space, so that readers/writers who subsequently access the target database file through the target process can directly access the second virtual memory space without waiting for all readers/writers who are accessing the first virtual memory space to end, thereby improving the access efficiency of the database file.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (8)
1. A lightning memory mapping database LMDB expansion method is characterized by comprising the following steps:
when determining that a first virtual memory space of a target process pointed by a pointer of a target database file meets an expansion condition, mapping a second virtual memory space in the target process for the target database file; wherein the length of the second virtual memory space is greater than the length of the first virtual memory space; not releasing the first virtual memory space for the target database before mapping the second virtual memory space in the target process;
pointing the pointer of the target database file to the second virtual memory space;
wherein, after the pointing the pointer of the target database file to the second virtual memory space, the method further includes:
and releasing the first virtual memory space when the first virtual memory space is determined to have no visitor.
2. The method of claim 1, wherein before mapping a second virtual memory space for the target database file in the target process, further comprising:
when the residual space of the first virtual memory space is smaller than a preset threshold value, adding a preset mark in the file header of the target database file;
the determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition includes:
when the preset mark is detected to exist in the file head of the target database file, determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition.
3. The method according to claim 1, wherein the file header of the target database file records the length of the target database file;
the determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets the expansion condition includes:
when the length of the target database file is detected to be inconsistent with the length of the first virtual memory space, determining that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition.
4. The method of claim 1, further comprising:
when detecting that the target process calls a transaction starting interface corresponding to the first virtual memory space, increasing a reference count corresponding to the first virtual memory space by a preset value;
when detecting that the target process calls a transaction submission interface corresponding to the first virtual memory space, subtracting the preset value from the reference count corresponding to the first virtual memory space;
the determining that no visitor exists to the first virtual memory space comprises:
and when detecting that the value of the reference count corresponding to the first virtual memory space is an initial value, determining that no visitor exists in the first virtual memory space.
5. A lightning memory mapped database LMDB extension apparatus, comprising:
the system comprises a determining unit, a processing unit and a processing unit, wherein the determining unit is used for determining whether a first virtual memory space of a target process pointed by a pointer of a target database file meets an expansion condition;
the mapping unit is used for mapping a second virtual memory space in a target process for a target database file when the fact that a first virtual memory space of the target process pointed by a pointer of the target database file meets an expansion condition is determined; wherein the length of the second virtual memory space is greater than the length of the first virtual memory space; not releasing the first virtual memory space for the target database before mapping the second virtual memory space in the target process;
the control unit is used for pointing the pointer of the target database file to the second virtual memory space;
the control unit is further configured to release the first virtual memory space when it is determined that no visitor exists in the first virtual memory space.
6. The apparatus of claim 5, further comprising:
an adding unit, configured to add a preset flag in a file header of the target database file when a remaining space of the first virtual memory space is smaller than a preset threshold;
the determining unit is specifically configured to determine that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition when it is detected that the preset flag exists at the file header of the target database file.
7. The apparatus according to claim 5, wherein the file header of the target database file records the length of the target database file;
the determining unit is specifically configured to determine that the first virtual memory space of the target process pointed by the pointer of the target database file meets an expansion condition when it is detected that the length of the target database file is inconsistent with the length of the first virtual memory space.
8. The apparatus of claim 5,
the control unit is specifically configured to increase a reference count corresponding to the first virtual memory space by a preset value when it is detected that the target process calls a transaction start interface corresponding to the first virtual memory space; when detecting that the target process calls a transaction submission interface corresponding to the first virtual memory space, subtracting the preset value from the reference count corresponding to the first virtual memory space;
the control unit is further specifically configured to determine that no visitor exists in the first virtual memory space when it is detected that the value of the reference count corresponding to the first virtual memory space is an initial value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811142236.3A CN109408202B (en) | 2018-09-28 | 2018-09-28 | LMDB extension method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811142236.3A CN109408202B (en) | 2018-09-28 | 2018-09-28 | LMDB extension method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109408202A CN109408202A (en) | 2019-03-01 |
| CN109408202B true CN109408202B (en) | 2020-11-06 |
Family
ID=65465537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811142236.3A Active CN109408202B (en) | 2018-09-28 | 2018-09-28 | LMDB extension method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109408202B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902459A (en) * | 2012-12-25 | 2014-07-02 | 华为技术有限公司 | Method and associated equipment for determining management mode of shared virtual memory page |
-
2018
- 2018-09-28 CN CN201811142236.3A patent/CN109408202B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902459A (en) * | 2012-12-25 | 2014-07-02 | 华为技术有限公司 | Method and associated equipment for determining management mode of shared virtual memory page |
| CN107402891A (en) * | 2012-12-25 | 2017-11-28 | 华为技术有限公司 | Determine the method and relevant device of shared virtual memory page management pattern |
Non-Patent Citations (2)
| Title |
|---|
| 认真分析mmap：是什么 为什么 怎么用 https://www.cnblogs.com/huxiao-tee/p/4660352.html#4008787;胡潇;《认真分析mmap：是什么 为什么 怎么用 博客园》;20150720;正文1-6页，附图1 * |
| 请问溢出mmap()分配的共享空间要怎么办 https://www.169it.com/tech-qa-linux/article-15164213648962554143.html;互联网;《请问溢出mmap()分配的共享空间要怎么办 169IT》;20150809;正文第1页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109408202A (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180173620A1 (en) | Data erasure method for solid state drive, and apparatus | |
| US9921955B1 (en) | Flash write amplification reduction | |
| US20150317246A1 (en) | Memory Reclamation Method and Apparatus | |
| CN109213696B (en) | Method and apparatus for cache management | |
| CN108959118B (en) | Data writing method and device | |
| CN111723058B (en) | Pre-read data caching method, device, equipment and storage medium | |
| CN108874688B (en) | Message data caching method and device | |
| US10789170B2 (en) | Storage management method, electronic device and computer readable medium | |
| CN106201652B (en) | Data processing method and virtual machine | |
| CN109558456A (en) | A kind of file migration method, apparatus, equipment and readable storage medium storing program for executing | |
| CN105138481A (en) | Stored data processing method and apparatus and system | |
| US8484620B2 (en) | Implementing performance impact reduction of watched variables | |
| US10901914B2 (en) | Method for writing multiple copies into storage device, and storage device | |
| CN109408202B (en) | LMDB extension method and device | |
| WO2018171266A1 (en) | Video monitoring data storage method and device | |
| CN110647476B (en) | Method, device and equipment for writing data in solid state disk and storage medium | |
| CN104899158A (en) | Memory access optimization method and memory access optimization device | |
| CN108334457B (en) | IO processing method and device | |
| CN108170380B (en) | Method for improving sequential reading performance of solid state disk and solid state disk | |
| CN110795031A (en) | Data deduplication method, device and system based on full flash storage | |
| CN111880735B (en) | Data migration method, device, equipment and storage medium in storage system | |
| CN106202262B (en) | Information processing method and electronic equipment | |
| CN109840217B (en) | Cache resource allocation and device | |
| CN112148220B (en) | Method, device, computer storage medium and terminal for realizing data processing | |
| CN110825652B (en) | Method, device and equipment for eliminating cache data on disk block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |