CN110321367B - Data storage method, data query method, related device and electronic equipment - Google Patents
Data storage method, data query method, related device and electronic equipment Download PDFInfo
- Publication number
- CN110321367B CN110321367B CN201910573663.5A CN201910573663A CN110321367B CN 110321367 B CN110321367 B CN 110321367B CN 201910573663 A CN201910573663 A CN 201910573663A CN 110321367 B CN110321367 B CN 110321367B
- Authority
- CN
- China
- Prior art keywords
- stored
- key
- value
- meta information
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data storage method, a data query method, a related device and electronic equipment. When the key value pair is stored into the cache, acquiring a key to be stored and a value to be stored of the key value pair to be stored; storing the value to be stored in a local system memory; acquiring meta information of the stored value to be stored; and correspondingly storing the key to be stored and the meta information to the application memory. When the key value pairs in the cache are inquired, acquiring a key to be inquired; inquiring the meta information stored corresponding to the key to be inquired in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be inquired; and acquiring the stored value represented by the meta information in the memory of the local system as a value corresponding to the key to be inquired.
Description
Technical Field
The present invention relates to the field of storage and query technologies, and in particular, to a data storage method, a data query method, a related apparatus, and an electronic device.
Background
In a distributed system, there is a large amount of data storage and querying that needs to rely on caching.
When the application memory is used as the cache, the cache and the application program share one memory, the application program does not need to pass through network IO when inquiring the cache, and the inquiry speed is higher because the application program is very close to the cache. However, the application memory is used as a specific storage area partitioned from the system memory, and the storage space of the application memory is small, so that once the size of the cache data is increased, the memory space required by the running of the application program is invaded, the running rate of the application program is reduced, and even the application program is crashed.
If the system memory is used as the cache, although large-scale cache data can be stored without influencing the running speed of the application program, the cache data is stored in the system memory, so that the distance between the application program and the cache is increased, and the cache query speed is reduced.
Disclosure of Invention
The embodiment of the invention aims to provide a data storage method, a data query method, a related device and electronic equipment, so as to reduce the adverse effect of cache on the performance of an application program without reducing the query rate. The specific technical scheme is as follows:
to achieve the above object, an embodiment of the present invention provides a data storage method, including:
acquiring a key to be stored and a value to be stored of a key value pair to be stored;
storing the value to be stored in a local system memory;
acquiring the stored meta-information of the value to be stored;
and correspondingly storing the key to be stored and the meta information to an application memory.
Further, the step of correspondingly storing the key to be stored and the meta information to an application memory includes:
calculating the storage position of the key to be stored in a hash bucket in the application memory according to a hash algorithm;
and storing the key to be stored and the meta information to the storage position.
Further, before the step of storing the value to be stored in the local system memory, the method further includes:
serializing the value to be stored into binary stream, and acquiring serialization mode information as a part of meta-information of the value to be stored;
the storing the value to be stored into a local system memory includes:
and storing the binary stream obtained after the value to be stored is serialized into a local system memory.
Further, the method further comprises:
when the stored value of the stored key-value pair needs to be updated, acquiring the meta-information corresponding to the stored key of the stored key-value pair in the application memory as the original meta-information;
updating the stored value represented by the original meta information in the memory of the local system into an updated value, and acquiring new meta information of the updated value;
and changing the original meta information in the application memory into the new meta information.
Further, the meta information includes a storage address of the value to be stored in the local system memory.
The embodiment of the invention provides a data query method, which comprises the following steps:
acquiring a key to be queried;
inquiring the meta information stored corresponding to the key to be inquired in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be inquired;
and acquiring the stored value represented by the meta information in a local system memory as a value corresponding to the key to be queried.
Further, the key to be queried is stored in a hash bucket in the application memory;
the step of querying the meta information stored corresponding to the key to be queried in the application memory includes:
calculating the storage position of the key to be queried in a hash bucket in the application memory according to a hash algorithm;
and acquiring the meta information from the storage position as the meta information stored corresponding to the key to be queried.
Further, the meta information includes serialization mode information in which the value corresponding to the key to be queried is serialized into a binary stream;
after the obtaining the stored value represented by the meta information in the local system memory, further comprising:
acquiring the serialization mode information from the meta information;
and performing deserialization on the value corresponding to the key to be queried by using the serialization mode information.
Further, the meta information includes a storage address of the value corresponding to the key to be queried in the local system memory.
An embodiment of the present invention further provides a data storage device, including:
the first acquisition module is used for acquiring a key to be stored and a value to be stored of the key value pair to be stored;
the first storage module is used for storing the value to be stored into a local system memory;
the second acquisition module is used for acquiring the stored meta information of the value to be stored;
and the second storage module is used for correspondingly storing the key to be stored and the meta information to an application memory.
Further, the second storage module is specifically configured to calculate, according to a hash algorithm, a storage location of the key to be stored in a hash bucket in the application memory; and storing the key to be stored and the meta information to the storage position.
Further, the apparatus further comprises:
the serialization module is used for serializing the value to be stored into a binary stream before the first storage module stores the value to be stored into a local system memory, and acquiring serialization mode information as a part of meta information of the value to be stored;
the first storage module is specifically configured to store the binary stream obtained by serializing the value to be stored in the local system memory.
Further, the apparatus further comprises:
a third obtaining module, configured to obtain, when a stored value of a stored key-value pair needs to be updated, meta information corresponding to the stored key of the stored key-value pair in an application memory, as original meta information;
the first storage module is further configured to update a stored value represented by the original meta information in a local system memory to an updated value, and acquire new meta information of the updated value;
the second storage module is further configured to change the original meta information in the application memory to the new meta information.
Further, the meta information includes a storage address of the value to be stored in the local system memory.
An embodiment of the present invention further provides a data query apparatus, including:
the key acquisition module is used for acquiring a key to be queried;
the query module is used for querying the meta information stored corresponding to the key to be queried in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be queried;
and the value acquisition module is used for acquiring the stored value represented by the meta information in a local system memory as the value corresponding to the key to be inquired.
Further, the key to be queried is stored in a hash bucket in the application memory;
the query module is specifically used for calculating the storage position of the key to be queried in a hash bucket in the application memory according to a hash algorithm; and acquiring the meta information from the storage position as the meta information stored corresponding to the key to be queried.
Further, the meta information includes serialization mode information in which the value corresponding to the key to be queried is serialized into a binary stream;
the device further comprises:
the deserialization module is used for acquiring the serialization mode information from the meta-information; and performing deserialization on the value corresponding to the key to be queried by using the serialization mode information.
Further, the meta information includes a storage address of the value corresponding to the key to be queried in the local system memory.
The embodiment of the invention also provides electronic equipment which is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any data storage method step when executing the program stored in the memory.
The embodiment of the invention also provides electronic equipment which is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing the steps of any data query method when executing the program stored in the memory.
The present invention also provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above data storage methods.
The present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps of any of the above data query methods.
Embodiments of the present invention also provide a computer program product containing instructions, which when run on a computer, cause the computer to perform any of the above data storage methods.
Embodiments of the present invention further provide a computer program product containing instructions, which when run on a computer, cause the computer to execute any of the above data query methods.
The invention has the beneficial effects that:
the embodiment of the invention provides a data storage method and a data query method, wherein when key-value pairs are stored into a cache, keys to be stored of the key-value pairs to be stored and the values to be stored are obtained; storing the value to be stored in a local system memory; acquiring meta information of the stored value to be stored; and correspondingly storing the key to be stored and the meta information to the application memory. When the key value pairs in the cache are inquired, acquiring a key to be inquired; inquiring the meta information stored corresponding to the key to be inquired in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be inquired; and acquiring the stored value represented by the meta information in the memory of the local system as a value corresponding to the key to be inquired. By adopting the method provided by the embodiment of the invention, all information of the key value pair is not stored in the application memory, but the value of the key value pair is stored in the system memory, and the key information of the key value pair and the meta information of the value are stored in the application memory, so that the cache information stored in the application memory is reduced, and the adverse effect of the cache on the performance of the application program is reduced while the query speed is not reduced.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a schematic flowchart of a data storage method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a data query method according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data storage device according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a data query device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a data storage method according to an embodiment of the present invention, which may include the following steps:
And 102, storing the value to be stored into a local system memory.
Specifically, before executing this step, the value to be stored may be serialized into a binary stream, and the serialization manner information may be acquired as a part of the meta information of the value to be stored;
the to-be-stored value is stored in the local system memory, and specifically, the binary stream obtained by serializing the to-be-stored value is stored in the local system memory.
And 103, acquiring the stored meta information of the value to be stored.
Specifically, the meta information may include: in practical applications, other information of the key-value pair to be stored may also be used as a part of the meta information, for example, the meta information may also include an expiration time of the key-value pair to be stored.
And 104, correspondingly storing the key to be stored and the meta information to the application memory.
In steps S103 to S104, the expiration time of the key value pair to be stored is the storage time limit set for the key value pair to be stored, and may be specified when storing the meta information in the application memory, or may directly use the default expiration time. If an expiration time needs to be specified, the specified expiration time may also be stored in the meta-information. Here, the application memory may be a memory area of a fixed size partitioned from the system memory, and when partitioned, the partial memory area will mainly serve the application.
After the key to be stored and the meta information are stored in the application memory correspondingly, the stored key value pair can be monitored according to an LRU (Least Recently Used) algorithm or an expiration mechanism, and once the storage time exceeds the set expiration time, the value exceeding the storage time limit and the corresponding key and meta information are deleted together.
Specifically, the step S104 may be executed in the following manner:
calculating the storage position of the key to be stored in a hash bucket in the application memory according to a hash algorithm;
and storing the key to be stored and the meta information to the storage position.
Further, when the stored value of the stored key-value pair needs to be updated, the meta information corresponding to the stored key of the stored key-value pair can be acquired in the application memory as the original meta information;
updating the stored value represented by the meta information in the memory of the local system into an updated value, and acquiring new meta information of the updated value;
and changing the original meta information in the application memory into new meta information.
By adopting the data storage method provided by the embodiment of the invention, all information of the key value pair is not stored in the application memory, but the value of the key value pair is stored in the system memory, and the key information of the key value pair and the meta information of the value are stored in the application memory, so that the cache information stored in the application memory is reduced, and the adverse effect of the cache on the performance of the application program is reduced while the query rate is not reduced.
The data storage method shown in fig. 1 will be described in detail below by way of example.
For example, the key to be stored of the key-value pair is person age lyy, which indicates the age of a person named lyy, and the value to be stored of the key-value pair is 18, which indicates the age of 18 years;
firstly, 18 can be serialized into a string of binary streams, and serialization mode information is obtained;
storing the binary stream of 18 into a local system memory, and acquiring a storage address of the binary stream in the local system memory;
taking the storage address and the serialization mode information of the binary stream of 18 in the local system memory as a part of meta information of the value to be stored;
calculating the storage position of person | Lily in the hash bucket in the application memory according to the hash algorithm;
storing the person information and meta information of the value to be stored in the storage location in the hash bucket.
When the existing key value pair needs to be updated, for example, the value of the key person age only needs to be updated to 19, the primitive information stored corresponding to the key person age only can be acquired from the application memory;
updating the binary stream of the stored value 18 represented by the storage address in the local system memory in the original meta-information into the binary stream of 19, and using the serialization mode information of the binary stream of 19 and the storage address in the local system memory as a part of the new meta-information;
and updating the original meta information in the application memory into new meta information.
It can be understood that, because languages such as Java have a garbage collection mechanism, the usage of the application memory will be scanned regularly. If the application memory is used as the cache, the data stored in the application memory is continuously increased, which also results in an increase in the time for scanning the application memory by using a garbage collection mechanism, further causing a decrease in the running rate of the application program.
Correspondingly, an embodiment of the present invention provides a data query method, as shown in fig. 2, which specifically includes the following steps:
Specifically, the key to be queried may be stored in a hash bucket in the application memory.
This step may be performed in the following manner:
calculating the storage position of the key to be queried in a hash bucket in the application memory according to a hash algorithm;
and acquiring the meta information from the storage position as the meta information stored corresponding to the key to be queried.
Specifically, the meta information may include: in practical applications, the meta information may further include other information of the key-value pair to be queried, for example, the meta information may further include an expiration time of the key-value pair to be queried.
In this step, the value corresponding to the key to be queried may be specifically obtained according to the storage address of the value corresponding to the key to be queried in the meta information in the local system memory.
Specifically, after the step, serialization mode information can be acquired from the meta information;
and performing deserialization on the value corresponding to the key to be queried by using the serialization mode information.
The data storage method shown in fig. 2 will be described in detail below by way of example.
For example, to query for the age of Lily, the to-be-queried key person age Lily may be obtained first;
calculating the storage position of person | Lily in the hash bucket in the application memory according to the hash algorithm;
acquiring meta information stored in correspondence with person | age _ Lily from the storage location;
according to the storage address of the local system memory in the meta information, acquiring a binary stream stored at the storage address, for example, a binary stream of 18;
the binary stream of 18 is deserialized into 18 according to the serialization manner information of the binary stream in the meta information.
By adopting the data storage method provided by the embodiment of the invention, the application program queries the key to be queried in the application memory, the distance is short, the query speed is high, further, the value corresponding to the key to be queried is queried in the local system memory according to the meta information, the query speed is high without network IO (input/output), and meanwhile, the value to be queried is not required to be queried in the application memory, so that the query frequency of the application memory is reduced, and the adverse effect on the application program is reduced.
Based on the same inventive concept, according to the data storage method provided by the above embodiment of the present invention, correspondingly, another embodiment of the present invention further provides a data storage device, a schematic structural diagram of which is shown in fig. 3, and the data storage device specifically includes:
a first obtaining module 301, configured to obtain a key to be stored and a value to be stored of a key-value pair to be stored;
a first storage module 302, configured to store a value to be stored in a local system memory;
a second obtaining module 303, configured to obtain meta information of a stored value to be stored;
the second storage module 304 is configured to correspondingly store the key to be stored and the meta information to the application memory.
Further, the second storage module 304 is specifically configured to calculate, according to a hash algorithm, a storage location of the key to be stored in a hash bucket in the application memory; and storing the key to be stored and the meta information to the storage position.
Further, the apparatus further comprises:
a serialization module 305, configured to serialize the value to be stored into a binary stream before the first storage module 302 stores the value to be stored into the local system memory, and acquire serialization manner information as a part of meta information of the value to be stored;
the first storage module 302 is specifically configured to store a binary stream obtained by serializing a value to be stored to a local system memory.
Further, the apparatus further comprises:
a third obtaining module 306, configured to obtain, when the stored value of the stored key-value pair needs to be updated, meta information corresponding to the stored key of the stored key-value pair in the application memory, as original meta information;
the first storage module 302 is further configured to update a stored value represented by meta information in a local system memory to an updated value, and acquire new meta information of the updated value;
the second storage module 304 is further configured to change the original meta information in the application memory into new meta information.
Further, the meta information includes a storage address of the value to be stored in the local system memory.
Based on the same inventive concept, according to the data query method provided in the above embodiment of the present invention, correspondingly, another embodiment of the present invention further provides a data query device, a schematic structural diagram of which is shown in fig. 4, and the data query device specifically includes:
a key obtaining module 401, configured to obtain a key to be queried;
a query module 402, configured to query, in an application memory, meta information stored in correspondence with a key to be queried, where the meta information is meta information of a value corresponding to the key to be queried;
a value obtaining module 403, configured to obtain the stored value represented by the meta information in the local system memory, as the value corresponding to the key to be queried.
Further, the key to be queried is stored in a hash bucket in the application memory;
a query module 402, configured to calculate, according to a hash algorithm, a storage location of a key to be queried in a hash bucket in an application memory; and acquiring the meta information from the storage position as the meta information stored corresponding to the key to be queried.
Further, the meta information includes serialization mode information in which the value corresponding to the key to be queried is serialized into a binary stream;
the device also includes:
a deserializing module 404, configured to obtain serialization manner information from the meta information; and performing deserialization on the value corresponding to the key to be queried by using the serialization mode information.
Further, the meta information includes a storage address of a value corresponding to the key to be queried in the local system memory.
Based on the same inventive concept, according to the data storage method provided by the above-mentioned embodiment of the present invention, an electronic device is further provided in the embodiment of the present invention, as shown in fig. 5, which includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 are communicated with each other via the communication bus 504,
a memory 503 for storing a computer program;
the processor 501, when executing the program stored in the memory 503, implements the following steps:
acquiring a key to be stored and a value to be stored of a key value pair to be stored;
storing the value to be stored in a local system memory;
acquiring meta information of the stored value to be stored;
and correspondingly storing the key to be stored and the meta information to the application memory.
Based on the same inventive concept, according to the data query method provided by the above embodiment of the present invention, an embodiment of the present invention further provides an electronic device, as shown in fig. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604,
a memory 603 for storing a computer program;
the processor 601 is configured to implement the following steps when executing the program stored in the memory 603:
acquiring a key to be queried;
inquiring the meta information stored corresponding to the key to be inquired in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be inquired;
and acquiring the stored value represented by the meta information in a local system memory as a value corresponding to the key to be inquired.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In yet another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above data storage methods.
In another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above data query methods.
In yet another embodiment, the present invention further provides a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the data storage methods of the above embodiments.
In yet another embodiment, a computer program product containing instructions is provided, which when run on a computer, causes the computer to perform any one of the above-described data query methods.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus, the electronic device, the computer-readable storage medium, and the computer program product embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and in relation to them, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (20)
1. A method of storing data, comprising:
acquiring a key to be stored and a value to be stored of a key value pair to be stored;
storing the value to be stored in a local system memory;
acquiring the stored meta-information of the value to be stored;
and correspondingly storing the key to be stored and the meta information to an application memory.
2. The method according to claim 1, wherein the step of storing the key to be stored and the meta information in the application memory in a corresponding manner comprises:
calculating the storage position of the key to be stored in a hash bucket in the application memory according to a hash algorithm;
and storing the key to be stored and the meta information to the storage position.
3. The method of claim 1, wherein prior to the step of storing the value to be stored to local system memory, the method further comprises:
serializing the value to be stored into binary stream, and acquiring serialization mode information as a part of meta-information of the value to be stored;
the storing the value to be stored into a local system memory includes:
and storing the binary stream obtained after the value to be stored is serialized into a local system memory.
4. The method of claim 1, further comprising:
when the stored value of the stored key-value pair needs to be updated, acquiring the meta-information corresponding to the stored key of the stored key-value pair in the application memory as the original meta-information;
updating the stored value represented by the original meta information in the memory of the local system into an updated value, and acquiring new meta information of the updated value;
and changing the original meta information in the application memory into the new meta information.
5. The method according to any one of claims 1 to 4, wherein the meta information comprises a storage address of the value to be stored in the local system memory.
6. A method for querying data, comprising:
acquiring a key to be queried;
inquiring the meta information stored corresponding to the key to be inquired in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be inquired;
and acquiring the stored value represented by the meta information in a local system memory as a value corresponding to the key to be queried.
7. The method of claim 6, wherein the key to be queried is stored in a hash bucket in the application memory;
the step of querying the meta information stored corresponding to the key to be queried in the application memory includes:
calculating the storage position of the key to be queried in a hash bucket in the application memory according to a hash algorithm;
and acquiring the meta information from the storage position as the meta information stored corresponding to the key to be queried.
8. The method according to claim 6, wherein the meta information includes serialization mode information in which the value corresponding to the key to be queried is serialized into a binary stream;
after the obtaining the stored value represented by the meta information in the local system memory, further comprising:
acquiring the serialization mode information from the meta information;
and performing deserialization on the value corresponding to the key to be queried by using the serialization mode information.
9. The method according to any one of claims 6 to 8, wherein the meta information includes a storage address of a value corresponding to the key to be queried in the local system memory.
10. A data storage device, comprising:
the first acquisition module is used for acquiring a key to be stored and a value to be stored of the key value pair to be stored;
the first storage module is used for storing the value to be stored into a local system memory;
the second acquisition module is used for acquiring the stored meta information of the value to be stored;
and the second storage module is used for correspondingly storing the key to be stored and the meta information to an application memory.
11. The apparatus according to claim 10, wherein the second storage module is specifically configured to calculate, according to a hashing algorithm, a storage location of the key to be stored in a hash bucket in the application memory; and storing the key to be stored and the meta information to the storage position.
12. The apparatus of claim 10, further comprising:
the serialization module is used for serializing the value to be stored into a binary stream before the first storage module stores the value to be stored into a local system memory, and acquiring serialization mode information as a part of meta information of the value to be stored;
the first storage module is specifically configured to store the binary stream obtained by serializing the value to be stored in the local system memory.
13. The apparatus of claim 10, further comprising:
a third obtaining module, configured to obtain, when a stored value of a stored key-value pair needs to be updated, meta information corresponding to the stored key of the stored key-value pair in an application memory, as original meta information;
the first storage module is further configured to update a stored value represented by the original meta information in a local system memory to an updated value, and acquire new meta information of the updated value;
the second storage module is further configured to change the original meta information in the application memory to the new meta information.
14. The apparatus of any of claims 10 to 13, wherein the meta information comprises a storage address of the value to be stored in the local system memory.
15. A data query apparatus, comprising:
the key acquisition module is used for acquiring a key to be queried;
the query module is used for querying the meta information stored corresponding to the key to be queried in the application memory, wherein the meta information is the meta information of the value corresponding to the key to be queried;
and the value acquisition module is used for acquiring the stored value represented by the meta information in a local system memory as the value corresponding to the key to be inquired.
16. The apparatus of claim 15, wherein the key to be queried is stored in a hash bucket in the application memory;
the query module is specifically used for calculating the storage position of the key to be queried in a hash bucket in the application memory according to a hash algorithm; and acquiring the meta information from the storage position as the meta information stored corresponding to the key to be queried.
17. The apparatus according to claim 15, wherein the meta information includes serialization manner information in which the value corresponding to the key to be queried is serialized into a binary stream;
the device further comprises:
the deserialization module is used for acquiring the serialization mode information from the meta-information; and performing deserialization on the value corresponding to the key to be queried by using the serialization mode information.
18. The apparatus according to any one of claims 16 to 17, wherein the meta information comprises a storage address of a value corresponding to the key to be queried in the local system memory.
19. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 5 when executing a program stored in the memory.
20. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 6 to 9 when executing a program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910573663.5A CN110321367B (en) | 2019-06-28 | 2019-06-28 | Data storage method, data query method, related device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910573663.5A CN110321367B (en) | 2019-06-28 | 2019-06-28 | Data storage method, data query method, related device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110321367A CN110321367A (en) | 2019-10-11 |
| CN110321367B true CN110321367B (en) | 2021-12-17 |
Family
ID=68120556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910573663.5A Active CN110321367B (en) | 2019-06-28 | 2019-06-28 | Data storage method, data query method, related device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110321367B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112800303B (en) * | 2019-11-13 | 2023-05-05 | 马上消费金融股份有限公司 | Browser data storage and reading method and related device |
| CN112988737B (en) * | 2019-12-16 | 2023-09-15 | 北京奇艺世纪科技有限公司 | Data storage method and device, data acquisition method and device and electronic equipment |
| CN112416940B (en) * | 2020-11-27 | 2024-05-28 | 深信服科技股份有限公司 | Key value pair storage method, device, terminal equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202548A (en) * | 2016-07-25 | 2016-12-07 | 网易（杭州）网络有限公司 | Date storage method, lookup method and device |
| US9703788B1 (en) * | 2014-03-31 | 2017-07-11 | EMC IP Holding Company LLC | Distributed metadata in a high performance computing environment |
| CN107659626A (en) * | 2017-09-11 | 2018-02-02 | 上海交通大学 | Towards the separate-storage method of temporary metadata |
| CN109521959A (en) * | 2018-11-01 | 2019-03-26 | 西安交通大学 | One kind being based on SSD-SMR disk mixing key assignments memory system data method for organizing |
-
2019
- 2019-06-28 CN CN201910573663.5A patent/CN110321367B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9703788B1 (en) * | 2014-03-31 | 2017-07-11 | EMC IP Holding Company LLC | Distributed metadata in a high performance computing environment |
| CN106202548A (en) * | 2016-07-25 | 2016-12-07 | 网易（杭州）网络有限公司 | Date storage method, lookup method and device |
| CN107659626A (en) * | 2017-09-11 | 2018-02-02 | 上海交通大学 | Towards the separate-storage method of temporary metadata |
| CN109521959A (en) * | 2018-11-01 | 2019-03-26 | 西安交通大学 | One kind being based on SSD-SMR disk mixing key assignments memory system data method for organizing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110321367A (en) | 2019-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321367B (en) | Data storage method, data query method, related device and electronic equipment | |
| US11176099B2 (en) | Lockless synchronization of LSM tree metadata in a distributed system | |
| US11074225B2 (en) | Synchronization of index copies in an LSM tree file system | |
| CN110928904A (en) | Data query method and device and related components | |
| US9384236B2 (en) | Method and system for operating on database queries | |
| CN109450969B (en) | Method and device for acquiring data from third-party data source server and server | |
| CN109815240B (en) | Method, apparatus, device and storage medium for managing index | |
| CN111506604B (en) | Method, apparatus and computer program product for accessing data | |
| CN110737682A (en) | cache operation method, device, storage medium and electronic equipment | |
| US9734178B2 (en) | Searching entity-key associations using in-memory objects | |
| CN113377289B (en) | Cache management method, system, computing device and readable storage medium | |
| WO2022156650A1 (en) | Data access method and apparatus | |
| WO2020192663A1 (en) | Data management method and related device | |
| CN110321298B (en) | Time interval determination method and device, electronic equipment and medium | |
| CN111563199A (en) | Data processing method and device | |
| CN111382206A (en) | Data storage method and device | |
| CN111651424A (en) | Data processing method and device, data node and storage medium | |
| CN111597259A (en) | Data storage system, method, device, electronic equipment and storage medium | |
| CN111046106A (en) | Cache data synchronization method, device, equipment and medium | |
| CN106156050B (en) | Data processing method and device | |
| CN110427538B (en) | Data query method, data storage method, data query device, data storage device and electronic equipment | |
| US7752194B2 (en) | LDAP revision history | |
| US10956386B2 (en) | Methods and apparatuses for automated performance tuning of a data modeling platform | |
| US7251660B2 (en) | Providing mappings between logical time values and real time values in a multinode system | |
| CN114268608A (en) | Address segment retrieval method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |