CN115993942B - Data caching method, device, electronic equipment and computer readable medium - Google Patents
Data caching method, device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN115993942B CN115993942B CN202310287702.1A CN202310287702A CN115993942B CN 115993942 B CN115993942 B CN 115993942B CN 202310287702 A CN202310287702 A CN 202310287702A CN 115993942 B CN115993942 B CN 115993942B
- Authority
- CN
- China
- Prior art keywords
- data
- determining
- candidate
- request
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the disclosure discloses a data caching method, a data caching device, electronic equipment and a computer readable medium. One embodiment of the method comprises the following steps: determining a set of data requests for the target data received within the target time period; in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, for each data request in the set of data requests, performing the following processing steps: determining the number of cache nodes corresponding to the data request; determining a routing address set corresponding to the data request; forwarding the data request to a cache node set corresponding to the route address set; and responding to the successful forwarding, and caching the data to be requested corresponding to the data request into the cache nodes included in the cache node set. According to the embodiment, aiming at the situation that the cache node storing part of data is down, the data reading performance can be ensured.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a data caching method, apparatus, electronic device, and computer readable medium.
Background
Data caching refers to a technique of caching data stored in a medium with low read-write performance into a medium with high read-write performance. At present, with the increase of data access, in order to improve the data reading performance, the following methods are generally adopted: and respectively caching part of the high-frequency access data through a plurality of caching nodes.
However, the inventors found that when the above manner is adopted, there are often the following technical problems:
firstly, when a cache node is down in a plurality of cache nodes, the data in the down cache node is lost, so that the data reading performance is reduced;
secondly, in different access periods, the access frequency of the same data often generates difference, and the data in the cache node is not released in time, so that the waste of cache resources is caused.
The above information disclosed in this background section is only for enhancement of understanding of the background of the inventive concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose data caching methods, apparatuses, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a data caching method, the method comprising: determining a set of data requests for the target data received within the target time period; in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, for each data request in the set of data requests, performing the following processing steps: determining the number of cache nodes corresponding to the data request; determining a routing address set corresponding to the data request, wherein the routing address in the routing address set is an address of a cache node for caching data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes; forwarding the data request to a cache node set corresponding to the routing address set; and in response to successful forwarding, caching the data to be requested corresponding to the data request into a cache node included in the cache node set.
In a second aspect, some embodiments of the present disclosure provide a data caching apparatus, the apparatus including: a determining unit configured to determine a set of data requests for target data received within a target period of time; an execution unit configured to, in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, perform, for each data request in the set of data requests, the following processing steps: determining the number of cache nodes corresponding to the data request; determining a routing address set corresponding to the data request, wherein the routing address in the routing address set is an address of a cache node for caching data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes; forwarding the data request to a cache node set corresponding to the routing address set; and in response to successful forwarding, caching the data to be requested corresponding to the data request into a cache node included in the cache node set.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors causes the one or more processors to implement the method described in any of the implementations of the first aspect above.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.
The above embodiments of the present disclosure have the following advantageous effects: by the data caching method of some embodiments of the present disclosure, the reading performance of data is improved. Specifically, the reason why the data reading performance is degraded is that: when a cache node is down in the plurality of cache nodes, the data in the down cache node is lost, so that the data reading performance is reduced. Based on this, the data caching method of some embodiments of the present disclosure first determines a set of data requests for target data received within a target time period. By determining the set of data requests within the target time period, it is thereby possible to determine whether the data request is a high frequency access request based on the number of data requests within the set of data requests. Then, in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, for each data request in the set of data requests, performing the following processing steps: the first step is to determine the number of cache nodes corresponding to the data request. Thereby, the number of cache nodes corresponding to the data request for caching the data to be requested is determined. And determining a routing address set corresponding to the data request, wherein the routing address in the routing address set is the address of a cache node for caching the data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes. By determining the routing address, request forwarding of the data request is facilitated. And thirdly, forwarding the data request to a cache node set corresponding to the routing address set. And fourthly, responding to successful forwarding, and caching the data to be requested corresponding to the data request into a cache node included in the cache node set. In this way, even if the cache node storing part of the data is down, the data reading performance can be ensured.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a flow chart of some embodiments of a data caching method according to the present disclosure;
FIG. 2 is a schematic diagram of some embodiments of a data caching apparatus according to the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Referring to fig. 1, a flow 100 of some embodiments of a data caching method according to the present disclosure is shown. The data caching method comprises the following steps:
In some embodiments, an executing body (e.g., computing device) of the data caching method may determine a set of data requests for target data received within a target time period. The target period may be a preset unit period. In practice, the target time period may be 100 milliseconds. The target data comprises data to be requested corresponding to each data request in the data request set. The data requests in the set of data requests may be requests sent by different clients for data access. In practice, the target data may be constituted of data to be accessed a, data to be accessed B, and data to be accessed C. The data request a may be a request to access the data a to be accessed. The data request B may be a request to access the data B to be accessed. The data request C may be a request to access the data C to be accessed. Meanwhile, the data request a, the data request B, and the data request C are all data requests received within a target period. Thus, the data request set may include: data request a, data request B, and data request C.
The computing device may be hardware or software. When the computing device is hardware, the computing device may be implemented as a distributed cluster formed by a plurality of servers or terminal devices, or may be implemented as a single server or a single terminal device. When the computing device is embodied as software, it may be installed in the hardware devices listed above. It may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein. Further, it should be appreciated that the number of computing devices may have any number of computing devices, as desired for implementation.
In some optional implementations of some embodiments, the executing entity determining the set of data requests for the target data received within the target time period may include the steps of:
and the first step, receiving the data access request sent by at least one client in the target time period as a candidate data request to obtain a candidate data request set.
And secondly, carrying out data grouping on the candidate data request set according to the data to be requested corresponding to the candidate data request in the candidate data request set to obtain a candidate data request group set. The data to be requested corresponding to each candidate data request in the candidate data request group may be each part of data in a complete data section.
And thirdly, for each candidate data request group in the candidate data request group set, according to the request receiving time of the candidate data request in the candidate data request group, carrying out request ordering on the candidate data request in the candidate data request group to obtain a candidate data request sequence.
In practice, the execution body may perform request ordering on the candidate data requests in the candidate data request group according to the sequence of the request receiving time, so as to obtain a candidate data request sequence. For example, the request receipt time for the 1 st candidate data in the candidate data request sequence is the smallest.
And step four, screening candidate data request sequences meeting screening conditions from the obtained candidate data request sequence group to be used as the data request set.
Wherein, the screening conditions are as follows: the candidate data request sequence includes a number of candidate data requests consistent with the target number, and the candidate data request sequence includes a request receipt time of the candidate data requests for the target location less than the target time. In practice, the target number is the number of candidate data requests in the candidate data request sequence containing the most candidate data requests in the candidate data request sequence group. The target location may be a location of the 1 st candidate data request in the candidate data request sequence. The target time may be a mean time of request receipt times corresponding to respective candidate data requests in the sequence of candidate data requests.
in step 1021, the number of cache nodes corresponding to the data request is determined.
In some embodiments, the execution body may determine the number of cache nodes corresponding to the data request. The preset request threshold may be a preset threshold for determining whether the data to be accessed, which is requested to be accessed by the data, is high-frequency data to be accessed. The number of cache nodes may be the number of cache nodes for caching data to be accessed for data request access. In practice, the cache node may be hardware or software. For example, the cache node may be a cache server. As another example, the cache node may be a virtual cache node built within a cache server.
As an example, the execution body may determine the preset number as the number of cache nodes.
In some optional implementations of some embodiments, the determining, by the execution body, the number of cache nodes corresponding to the data request may include the following steps:
the first step is to determine the number of nodes to be requested corresponding to the data request.
In practice, for example, the number of nodes to be requested may be a preset number. For another example, the number of nodes to be requested may be the number of nodes to be accessed by the data request obtained by performing request analysis on the data request.
And secondly, determining the number of redundant nodes according to the request splitting probability.
Wherein the request splitting probability characterizes a probability of splitting the request. In practice, the request split probability may be characterized by the probability of whether or not to set a redundant node. For example, the request split probability may be set manually.
As an example, the above-described execution body may determine a value of (1-request split probability)/request split probability +1 as the number of redundant nodes. In practice, the result calculated by (1-request split probability)/request split probability+1 may have a decimal case, so the result may be rounded up again to obtain the above-mentioned number of redundant nodes.
And thirdly, generating the number of the cache nodes according to the number of the nodes to be requested and the number of the redundant nodes.
The execution body may determine the sum of the number of nodes to be requested and the number of redundant nodes as the number of cache nodes.
By controlling two variables of the number of nodes to be requested and the request splitting probability, flexible control of the number of cache nodes can be realized.
In some embodiments, the execution body may determine a set of routing addresses corresponding to the data request. The routing address in the routing address set is an address of a cache node for caching data to be requested corresponding to the data request. The number of the routing addresses in the routing address set is consistent with the number of the cache nodes. In practice, first, the execution body may allocate the number of cache nodes. And then, determining the address of the cache node as a routing address to obtain a routing address set.
In some optional implementations of some embodiments, the executing body determining the set of routing addresses corresponding to the data request may include the following steps:
the first step, determining the cache node in a non-full occupation state as a candidate cache node to obtain a candidate cache node set.
In practice, first, the execution body may determine the cache resource usage rate of the cache node in the working state. And then, determining the cache nodes with the cache resource utilization rate smaller than the preset utilization rate as cache nodes in a non-full occupation state as candidate cache nodes, and obtaining a candidate cache node set.
Second, for each candidate cache node in the candidate cache node set, the following route address determining step is performed:
a first sub-step of determining whether cached data exists in the candidate cache node.
In practice, the executing entity may scan the candidate cache nodes to determine whether cached data exists.
And a second sub-step of determining a routing address in the routing address set from the node communication addresses of the candidate cache nodes in response to absence.
A third sub-step, responsive to the presence, of determining a data lifecycle of the cached data.
Wherein the data lifecycle characterizes a storage period of cached data within the candidate cache node.
And a fourth sub-step of releasing the buffer space corresponding to the buffered data in the candidate buffer node and determining the current available space size of the candidate buffer node after releasing the data in response to determining that the data life cycle is greater than a preset life cycle.
And a fifth substep, in response to determining that the data life cycle is less than or equal to a preset life cycle, determining a current available space size of the candidate cache node.
And a sixth substep, in response to determining that the current available space size is greater than or equal to a preset space size, determining a routing address in the routing address set from the node communication addresses of the candidate cache nodes.
By the method, on the basis of determining the routing address, when the cached data in the cache node is longer in caching time, resource release can be realized on the cached data with longer caching time, so that the purpose of improving the utilization rate of available space of the cache node is achieved.
In some optional implementations of some embodiments, before forwarding the data request to the set of cache nodes corresponding to the set of routing addresses, the execution body may further perform the following processing steps:
first, determining a storage mode of the cached data in the cache node.
The storage mode characterizes a storage mode of the cached data in the cache node. In practice, the storage modes may include a first storage mode and a second storage mode. Wherein the first storage mode characterizes data fragmentation storage. The second storage mode characterizes the non-fragmented storage of data. In practice, in order to improve the storage space utilization of the cache node, data fragmentation storage is often adopted. In order to improve the data retrieval efficiency of the cache node, data non-fragmentation storage is often adopted.
And a second step of releasing the buffer space corresponding to the buffered data and re-storing the buffered data in the candidate buffer node in block form in response to determining that the storage mode is the first storage mode.
And storing the block data in a second storage mode, and re-storing the cached data in the candidate cache nodes to improve the retrieval efficiency of the data.
And thirdly, responding to the fact that the storage mode is determined to be the second storage mode, and optimizing the cache position of the cached data in the candidate cache node.
In practice, the execution body may update the cache location of the cached data in the candidate cache node, that is, store the cached data in the candidate cache node before, thereby further achieving the purpose of improving the data retrieval efficiency.
The first step to the third step realize effective space carding of the cache nodes by determining different storage modes. In practice, the data fragmentation storage divides the cache space in the cache node into a plurality of discontinuous storage areas, thereby causing fragmentation of the cache space and affecting the data storage efficiency. Therefore, when the storage mode is the first storage mode, the method and the device can restore the cached data stored in a fragmentation mode, so that the fragmentation condition of the cache space of the corresponding cache node is reduced. In addition, in practice, even if the data is stored in the second storage mode, i.e. in a non-fragmented manner, when the interval length of the storage space is longer, the search time is still required for indexing to the cache space corresponding to the cached data, so when the storage mode is the second storage mode, the present disclosure optimizes the cache position of the cached data in the candidate cache node, thereby achieving the purpose of reducing the search time.
In some embodiments, the execution body may forward the data request to a set of cache nodes corresponding to the set of routing addresses. In practice, the execution body may forward the data request to the cache node set corresponding to the routing address set by means of address routing.
In some optional implementations of some embodiments, the foregoing execution body forwarding the data request to the cache node set corresponding to the routing address set may include the following steps:
for each routing address in the set of routing addresses, performing the following request forwarding step:
and firstly, performing address mapping on the routing address to determine the offset of the routing address in a pre-constructed buffer node position ring.
The buffer node position ring contains the relative positions of the buffer nodes. In practice, the execution body may perform address mapping on the routing address through a murmur hash function to determine an offset of the routing address in a pre-constructed cache node location ring.
And a second step of determining the cache node identification corresponding to the routing address according to the offset.
As an example, the execution body may determine the cache node identification sequentially according to the value of the offset. For example, the offset may be 12, and the cache node identification may be "L".
As yet another example, the execution body may determine the cache node identifier sequentially according to an interval in which the value of the offset is located. For example, the offset may be 12, between 1 and 33, and the cache node identification may be "a".
And thirdly, sending the data request to a cache node corresponding to the cache node identifier.
The conventional request forwarding by means of a routing table can lead to a drastic increase in the size of the routing table as the number of cache nodes increases. Meanwhile, in the conventional manner of addressing the routing table, as the volume of the routing table increases, the addressing time increases, thereby increasing the time cost of forwarding the request to the cache node. Thus, the present disclosure determines the relative location of the cache node to receive the request by mapping the location of the cache node to a ring of cache node locations by calculating an offset. The forwarding efficiency of the request is greatly improved, and the method has extremely excellent effect especially for the situations of high concurrency and massive requests.
In step 1024, in response to the forwarding success, the data to be requested corresponding to the data request is cached into the cache nodes included in the cache node set.
In some embodiments, in response to the forwarding success, the execution body may cache the data to be requested corresponding to the data request into a cache node included in the cache node set. In practice, for example, in order to improve the storage space utilization efficiency, the first storage mode may be adopted to cache the data to be requested corresponding to the data request into the cache nodes included in the cache node set. For another example, in order to improve the retrieval efficiency, the second storage mode may be adopted to cache the data to be requested corresponding to the data request into the cache nodes included in the cache node set.
In some optional implementations of some embodiments, after the buffering, in response to the forwarding success, the data to be requested corresponding to the data request into a buffering node included in the buffering node set, the execution body may further execute the following processing steps:
the first step, for each cache node in the cache node set, responding to the successful caching of the data to be requested corresponding to the data request, sending the data to be requested to the client corresponding to the data request, and starting a cache timer.
The cache timer may be a timer for timing a time during which the data to be requested is stored in the cache node. In practice, the above-mentioned buffer timer may be counted down.
And a second step of determining a first accessed number of the stored data to be requested in a first unit time of each cache node in the cache node set.
And thirdly, determining a first accessed frequency mean value according to the obtained first accessed frequency set.
In practice, the executing body may determine an arithmetic average value of each first accessed number in the obtained first accessed number set as the first accessed number average value.
And step four, updating the timer change rate of the buffer timer corresponding to the first target buffer node to be the first rate.
The first target cache node is a cache node corresponding to a cache node with the corresponding accessed times smaller than the average value of the first accessed times in the cache node set. In practice, the first rate may be a 1.25 times rate.
And fifthly, determining a second accessed number of the stored data to be requested in a second unit time of each cache node in the cache node set.
And step six, determining a second accessed frequency mean value according to the obtained second accessed frequency set.
In practice, the executing body may determine an arithmetic average value of the respective second accessed times in the obtained second accessed times set as the second accessed times average value.
And seventh, updating the timer change rate of the buffer timer corresponding to the second target buffer node to be the second rate.
The second target cache node is a cache node corresponding to a cache node with the corresponding accessed times smaller than the average value of the second accessed times in the cache node set. The second rate is greater than the first rate. In practice, the second rate of change may be a 1.5-fold rate.
And eighth step, in response to determining that the corresponding buffer timer in the buffer node set has the same timing value as the target timing value, releasing the data to be requested stored in the buffer node with the corresponding buffer timer having the same timing value as the target timing value.
In practice, the target timing value may be 0, that is, when the cache timer returns to zero, the data to be requested stored in the cache node whose corresponding cache timer timing value is the same as the target timing value is released.
The first to eighth steps are taken as an invention point of the present disclosure, and solve the second technical problem mentioned in the background art, that is, "in different access periods, access frequencies for the same data will often generate differences, and the data in the cache node is not released in time, which may cause waste of cache resources. In practice, the cache node belongs to a storage medium with high read-write performance. Therefore, it is important to improve the space utilization efficiency of the cache node, so that in order to ensure that the high-frequency accessed data can be written into the cache node timely and effectively, the data with reduced access times needs to be released timely. Based on the above, the present disclosure determines the release time of the data to be requested by setting the buffer timer, and considers that the same data to be requested in the present application often needs to be buffered in multiple buffer nodes, and the access hit probabilities of the data to be requested stored in different buffer nodes often have differences, so the present disclosure determines the access times of the data to be requested in each buffer node corresponding to the same data to be requested in unit time, and determines the average value of the access times, so as to determine whether to update the change rate of the counter for the data to be requested in each buffer node. Meanwhile, in consideration of the fact that the access frequency of the same data always generates differences in different access periods, a first unit time and a second unit time are set, and the average value of the access times is successively determined twice, so that whether the data to be requested in each cache node are updated in the counter change rate is determined. At the same time, the first rate and the second rate are set to further increase the rate of timer change of the buffer timer. By the method, the release of the data stored in the cache node and corresponding to the reduced access times can be effectively promoted, and the use efficiency of the cache resources is greatly improved.
The above embodiments of the present disclosure have the following advantageous effects: by the data caching method of some embodiments of the present disclosure, the reading performance of data is improved. Specifically, the reason why the data reading performance is degraded is that: when a cache node is down in the plurality of cache nodes, the data in the down cache node is lost, so that the data reading performance is reduced. Based on this, the data caching method of some embodiments of the present disclosure first determines a set of data requests for target data received within a target time period. By determining the set of data requests within the target time period, it is thereby possible to determine whether the data request is a high frequency access request based on the number of data requests within the set of data requests. Then, in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, for each data request in the set of data requests, performing the following processing steps: the first step is to determine the number of cache nodes corresponding to the data request. Thereby, the number of cache nodes corresponding to the data request for caching the data to be requested is determined. And determining a routing address set corresponding to the data request, wherein the routing address in the routing address set is the address of a cache node for caching the data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes. By determining the routing address, request forwarding of the data request is facilitated. And thirdly, forwarding the data request to a cache node set corresponding to the routing address set. And fourthly, responding to successful forwarding, and caching the data to be requested corresponding to the data request into a cache node included in the cache node set. In this way, even if the cache node storing part of the data is down, the data reading performance can be ensured.
With further reference to fig. 2, as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of a data caching apparatus, which correspond to those method embodiments shown in fig. 1, and which may be applied in particular in various electronic devices.
As shown in fig. 2, the data caching apparatus 200 of some embodiments includes: a determination unit 201 and an execution unit 202. Wherein the determining unit 201 is configured to determine a set of data requests for target data received within a target period of time; an execution unit 202 configured to, in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, perform the following processing steps for each data request in the set of data requests: determining the number of cache nodes corresponding to the data request; determining a routing address set corresponding to the data request, wherein the routing address in the routing address set is an address of a cache node for caching data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes; forwarding the data request to a cache node set corresponding to the routing address set; and in response to successful forwarding, caching the data to be requested corresponding to the data request into a cache node included in the cache node set.
It will be appreciated that the elements described in the data caching apparatus 200 correspond to the various steps of the method described with reference to fig. 1. Thus, the operations, features and advantages described above for the method are equally applicable to the data caching apparatus 200 and the units contained therein, and are not described herein.
Referring now to fig. 3, a schematic diagram of an electronic device (e.g., computing device) 300 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 3 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 3, the electronic device 300 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 301 that may perform various suitable actions and processes in accordance with programs stored in a read-only memory 302 or programs loaded from a storage 308 into a random access memory 303. In the random access memory 303, various programs and data necessary for the operation of the electronic device 300 are also stored. The processing means 301, the read only memory 302 and the random access memory 303 are connected to each other by a bus 304. An input/output interface 305 is also connected to the bus 304.
In general, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 308 including, for example, magnetic tape, hard disk, etc.; and communication means 309. The communication means 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. While fig. 3 shows an electronic device 300 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 3 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 309, or from storage device 308, or from read only memory 302. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing means 301.
It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: determining a set of data requests for the target data received within the target time period; in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, for each data request in the set of data requests, performing the following processing steps: determining the number of cache nodes corresponding to the data request; determining a routing address set corresponding to the data request, wherein the routing address in the routing address set is an address of a cache node for caching data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes; forwarding the data request to a cache node set corresponding to the routing address set; and in response to successful forwarding, caching the data to be requested corresponding to the data request into a cache node included in the cache node set.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes a determination unit and an execution unit. Where the names of the units do not constitute a limitation of the unit itself in some cases, for example, the determining unit may also be described as "a unit that determines a set of data requests for target data received within a target period of time".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (5)
1. A data caching method, comprising:
determining a set of data requests for the target data received within the target time period;
in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, for each data request in the set of data requests, performing the following processing steps:
determining the number of cache nodes corresponding to the data request;
determining a routing address set corresponding to the data request, wherein the routing addresses in the routing address set are addresses of cache nodes for caching data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes;
forwarding the data request to a cache node set corresponding to the route address set;
in response to successful forwarding, caching the data to be requested corresponding to the data request into a cache node included in the cache node set, wherein,
the determining the number of the cache nodes corresponding to the data request includes:
determining the number of nodes to be requested corresponding to the data request;
determining the number of redundant nodes according to the request splitting probability;
Generating the number of cache nodes according to the number of nodes to be requested and the number of redundant nodes, wherein,
the determining the routing address set corresponding to the data request includes:
determining a cache node in a non-full occupation state as a candidate cache node to obtain a candidate cache node set;
for each candidate cache node in the set of candidate cache nodes, performing the following routing address determination step:
determining whether cached data exists in the candidate cache node;
determining node communication addresses of the candidate cache nodes as routing addresses in the routing address set in response to the absence;
determining a data lifecycle of the cached data in response to the presence;
responsive to determining that the data lifecycle is greater than a preset lifecycle, releasing a cache space corresponding to the cached data in the candidate cache node, and determining a current available space size of the candidate cache node after releasing the data;
determining the current available space size of the candidate cache node in response to determining that the data lifecycle is less than or equal to a preset lifecycle;
and determining the node communication address of the candidate cache node as a routing address in the routing address set in response to determining that the current available space size is greater than or equal to a preset space size.
2. The method of claim 1, wherein the determining the set of data requests for the target data received within the target time period comprises:
receiving a data access request sent by at least one client in the target time period as a candidate data request to obtain a candidate data request set;
according to the data to be requested corresponding to the candidate data request in the candidate data request set, carrying out data grouping on the candidate data request set to obtain a candidate data request group set;
for each candidate data request group in the candidate data request group set, according to the request receiving time of the candidate data request in the candidate data request group, carrying out request sequencing on the candidate data request in the candidate data request group to obtain a candidate data request sequence;
screening candidate data request sequences meeting screening conditions from the obtained candidate data request sequence group to be used as the data request set, wherein the screening conditions are as follows: the candidate data request sequence includes a number of candidate data requests consistent with the target number, and the candidate data request sequence includes a request receipt time of the candidate data requests for the target location less than the target time.
3. A data caching apparatus, comprising:
a determining unit configured to determine a set of data requests for target data received within a target period of time;
an execution unit configured to, in response to determining that the number of data requests in the set of data requests is greater than a preset request threshold, perform, for each data request in the set of data requests, the following processing steps: determining the number of cache nodes corresponding to the data request; determining a routing address set corresponding to the data request, wherein the routing addresses in the routing address set are addresses of cache nodes for caching data to be requested corresponding to the data request, and the number of the routing addresses in the routing address set is consistent with the number of the cache nodes; forwarding the data request to a cache node set corresponding to the route address set; in response to successful forwarding, caching the data to be requested corresponding to the data request into a cache node included in the cache node set, wherein,
the determining the number of the cache nodes corresponding to the data request includes:
determining the number of nodes to be requested corresponding to the data request;
Determining the number of redundant nodes according to the request splitting probability;
generating the number of cache nodes according to the number of nodes to be requested and the number of redundant nodes, wherein,
the determining the routing address set corresponding to the data request includes:
determining a cache node in a non-full occupation state as a candidate cache node to obtain a candidate cache node set;
for each candidate cache node in the set of candidate cache nodes, performing the following routing address determination step:
determining whether cached data exists in the candidate cache node;
determining node communication addresses of the candidate cache nodes as routing addresses in the routing address set in response to the absence;
determining a data lifecycle of the cached data in response to the presence;
responsive to determining that the data lifecycle is greater than a preset lifecycle, releasing a cache space corresponding to the cached data in the candidate cache node, and determining a current available space size of the candidate cache node after releasing the data;
determining the current available space size of the candidate cache node in response to determining that the data lifecycle is less than or equal to a preset lifecycle;
And determining the node communication address of the candidate cache node as a routing address in the routing address set in response to determining that the current available space size is greater than or equal to a preset space size.
4. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-2.
5. A computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310287702.1A CN115993942B (en) | 2023-03-23 | 2023-03-23 | Data caching method, device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310287702.1A CN115993942B (en) | 2023-03-23 | 2023-03-23 | Data caching method, device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115993942A CN115993942A (en) | 2023-04-21 |
| CN115993942B true CN115993942B (en) | 2023-06-06 |
Family
ID=85995334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310287702.1A Active CN115993942B (en) | 2023-03-23 | 2023-03-23 | Data caching method, device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115993942B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107181775A (en) * | 2016-03-10 | 2017-09-19 | 北京大学 | Route selection method and routing arrangement in content center network |
| CN113760178A (en) * | 2021-01-29 | 2021-12-07 | 北京京东拓先科技有限公司 | Cache data processing method and device, electronic equipment and computer readable medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581052B (en) * | 2012-08-02 | 2017-07-21 | 华为技术有限公司 | A kind of data processing method, router and NDN system |
| US8817796B2 (en) * | 2012-08-29 | 2014-08-26 | International Business Machines Corporation | Cached routing table management |
| US9262323B1 (en) * | 2012-11-26 | 2016-02-16 | Amazon Technologies, Inc. | Replication in distributed caching cluster |
| CN106230953B (en) * | 2016-08-05 | 2019-07-05 | 北京邮电大学 | A kind of D2D communication means and device based on distributed storage |
| CN111338567B (en) * | 2020-02-13 | 2021-09-17 | 苏州浪潮智能科技有限公司 | Mirror image caching method based on Protocol Buffer |
-
2023
- 2023-03-23 CN CN202310287702.1A patent/CN115993942B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107181775A (en) * | 2016-03-10 | 2017-09-19 | 北京大学 | Route selection method and routing arrangement in content center network |
| CN113760178A (en) * | 2021-01-29 | 2021-12-07 | 北京京东拓先科技有限公司 | Cache data processing method and device, electronic equipment and computer readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115993942A (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112637287B (en) | Load balancing method and equipment | |
| CN110430142B (en) | Method and device for controlling flow | |
| CN112416632B (en) | Event communication method and device, electronic equipment and computer readable medium | |
| CN112035529A (en) | Caching method and device, electronic equipment and computer readable storage medium | |
| CN112256733A (en) | Data caching method and device, electronic equipment and computer readable storage medium | |
| CN113760536A (en) | Data caching method and device, electronic equipment and computer readable medium | |
| CN112015816A (en) | Data synchronization method, device, medium and electronic equipment | |
| CN109918381B (en) | Method and apparatus for storing data | |
| CN111309366B (en) | Method, device, medium and electronic equipment for managing registration core | |
| CN115993942B (en) | Data caching method, device, electronic equipment and computer readable medium | |
| CN114785770A (en) | Mirror layer file sending method and device, electronic equipment and computer readable medium | |
| CN112418389A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN116226189A (en) | Cache data query method, device, electronic equipment and computer readable medium | |
| CN111459893B (en) | File processing method and device and electronic equipment | |
| CN112507676B (en) | Method and device for generating energy report, electronic equipment and computer readable medium | |
| CN114115941A (en) | Resource sending method, page rendering method, device, electronic equipment and medium | |
| CN113553206A (en) | Data event execution method and device, electronic equipment and computer readable medium | |
| CN113064704A (en) | Task processing method and device, electronic equipment and computer readable medium | |
| CN113342837B (en) | Data transmission method, device, electronic equipment and computer readable medium | |
| CN113760178A (en) | Cache data processing method and device, electronic equipment and computer readable medium | |
| CN112015746A (en) | Data real-time processing method, device, medium and electronic equipment | |
| CN116361254B (en) | Image storage method, apparatus, electronic device, and computer-readable medium | |
| CN111209042B (en) | Method, device, medium and electronic equipment for establishing function stack | |
| CN111314457B (en) | Method and device for setting virtual private cloud | |
| CN113438284B (en) | Request processing method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |